Designing a future-proof health monitoring system

Contents

Designing a future-proof health monitoring system#

You are a fresh software engineer in a healthcare facility. The facility operates a health monitoring system which has been running over a decade on the local servers in the basement. The system will collect, process, and analyze data from wearable devices (e.g., heart rate, blood pressure, sleep patterns).

Due to the aging population in your country, the number of patients have been rising and likewise the complaints about the system. For example, the medical staff complains about the laggy patient portal and an in-house scientist mentioned that the disk space is not enough to run experiments.

The system administrator (admin) presents you the system. Your observations:

  • The system consists of:

    • a computing server with 64 cores and 512G RAM

    • a network-attached storage (NAS) with four hard disks

  • Some services provided by the system are Nginx webserver, PostgreSQL database, Nextcloud, Matrix.

  • The system runs an OS and the OS manages the services as systemd services.

  • If a systemd service fails after several automatic restarts, then the admin gets an email. Especially the new PostgreSQL version lead to many failures in the previous week.

  • Webserver and the database communicate via Unix sockets

  • The admin checks for updates every week and installs updates manually.

  • Most of the configuration is inside /etc and the admin backups the old configuration files with the suffix .old.

  • The NAS and the computing server filesystems are backuped every hour using btrbk

You propose the admin to virtualize the system. But the admin is already stressed with the daily maintenance of the server. You ask the admin:

Don’t you think that you should at least install the security updates automatically?

The admin replies:

You know, I have a limited contract here and if I automate the processes, then I would make myself unnecessary. At some point, there will be less work and my contract may not be extended.

You are surprised and cannot reply.

The next day during lunch, your line manager tells you that the section got new funding to renew the system and tasks you with designing a future-proof backend for the health monitoring system. When you ask to unpack future-proof, she replies:

Your goal is to ensure the system can:

  • Scale to handle data from thousands of devices and users so that we can handle the changing number of service users

  • Ensure reliability and availability for critical health data, such as emergency alerts.

  • Support future expansion to integrate with new devices and external healthcare providers.

  • Should

First you start with a guideline document. What will you pay attention to? You should at least answer:

  • How would you ensure that all dependencies are explicitly managed in your system? For example, how do you ensure that an OS update to libc won’t cause any failure to the system?

  • Imagine at some point you have to scale your database up. How would you design the connection to the database?

  • Imagine you are leaving the company and a new hire takes over the maintenance of the system. How does the user experience of maintaining the system look like?

Preparation#

Read the development branch of The Twelve Factors (and not the 2017 released version at 12factor.net).

Contents