HealthCam engineering

Engineering decisions behind a local-first AI health platform.

HealthCam is not only a webcam health-monitoring app. It is a modular platform for running computer vision analyzers locally, orchestrating services, ingesting telemetry, processing raw data, and generating dashboard-ready insights.

Local-first AIMicroservicesPlugin architectureTelemetry pipelinesDockerized analyzers

Local machine

Webcam capture
AI inference
Analyzer services
Local orchestration

Cloud platform

Authentication
Telemetry storage
SQL transformations
Dashboards

Plugin ecosystem

Manifest
Service definition
Telemetry schema
Dashboard definitions

The main architectural idea is simple: sensitive and expensive computation stays close to the user’s hardware, while the cloud provides the product layer around it. Plugins extend the system without forcing the core platform to be rewritten every time a new analyzer is added.

Architecture choices

Five decisions that shaped the platform

These decisions explain why HealthCam is structured as a platform instead of a single-purpose application. Each one solved a concrete engineering problem that appeared while moving from thesis prototype to a more extensible product architecture.

Engineering decision

Local-first AI architecture

HealthCam keeps sensitive video capture and AI inference on the user’s machine, while the cloud handles authentication, dashboards, configuration, telemetry storage, and product experience.

Context

HealthCam processes sensitive webcam data and runs computer vision analyzers that may require significant compute. Sending raw video to the cloud would increase privacy risk, infrastructure cost, and latency.

Decision

I designed the platform so video capture and AI inference run locally on the user’s machine. The cloud is responsible for user accounts, dashboards, configuration, telemetry storage, and the overall product experience.

Why

Privacy: raw webcam and video data does not need to leave the user’s machine.
Scalability: inference compute is distributed across user devices instead of centralized on the server.
Cost: the cloud does not need to continuously process expensive video workloads.
Flexibility: different analyzers can run close to the hardware and use local resources.

Trade-offs

The local agent must handle different operating systems, hardware, dependencies, and device permissions.
Users need to install and run a local agent instead of using a fully cloud-based app.
Not every analyzer works on every device. For example, CUDA-dependent modules cannot run on a Mac without NVIDIA GPU support.

Result

This architecture makes HealthCam more privacy-preserving and economically scalable, while turning the local agent into a core part of the product rather than a simple helper script.

Engineering decision

Plugin-based platform architecture

Analyzers are treated as self-contained platform extensions, so new computer vision capabilities can be added without modifying the core application.

Context

Traditional software often requires modifying the core application whenever a new feature is introduced. In HealthCam, every new computer vision analyzer would otherwise require changes across service installation, orchestration, telemetry ingestion, cloud data processing, dashboard integration, and user interfaces.

Decision

I designed HealthCam around a plugin-based architecture where analyzers are treated as self-contained platform extensions rather than built-in features. Each plugin exposes a manifest describing how it integrates with the platform, including its installation source, service definition, lifecycle configuration, telemetry schema, SQL transformation scripts, dashboard definitions, and metadata.

When a user installs a plugin, the local agent automatically downloads it, the orchestrator discovers and manages the new service, the analyzer’s telemetry becomes part of the cloud ingestion pipeline, the plugin’s SQL transformations generate dashboard-ready insights from the raw telemetry, and the associated dashboard components become available without requiring modifications to the platform itself.

Why

Extensibility: new analyzers can be introduced without modifying the core platform.
Modularity: every plugin encapsulates its own service, telemetry schema, SQL transformations, dashboard definitions, and configuration.
Separation of responsibilities: plugin developers implement analyzer-specific logic, transformations, and dashboard definitions, while the platform provides installation, orchestration, telemetry transport, lifecycle management, and dashboard infrastructure.
Developer experience: creating a new analyzer does not require understanding or modifying the platform internals.
Scalability: the platform grows by adding independent plugins rather than continuously increasing the complexity of the core system.
Consistency: every analyzer integrates through the same platform contract regardless of its internal implementation.
Marketplace support: analyzers become independently installable capabilities that can later be distributed through a plugin marketplace.

Trade-offs

Plugin discovery, lifecycle management, and manifest validation introduce additional infrastructure complexity.
Every plugin must conform to the platform contract and manifest specification.
Compatibility between plugins and platform versions must be maintained over time.
Designing stable extension points requires more upfront architectural work than directly implementing features inside the core application.

Result

HealthCam evolves from a fixed health application into an extensible platform for computer vision capabilities. Adding a new analyzer becomes primarily a plugin development task rather than a platform development task.

Engineering decision

Raw telemetry storage with pre-aggregated dashboard data

Raw telemetry is preserved as the source of truth, while pre-computed insight tables keep dashboards fast and scalable.

Context

HealthCam continuously generates telemetry from multiple analyzers, including heart rate measurements, application usage, eye strain metrics, and future computer vision outputs. Dashboards need fast access to historical trends, while future analyzers may need the original measurements to compute new metrics or validate algorithms.

Decision

I separated the storage layer into two complementary datasets. Raw telemetry is preserved as the source of truth, while dashboard-specific insight tables store pre-computed statistics optimized for visualization.

The dashboard reads from aggregated insight tables, while raw telemetry remains available for future processing, validation, and new analytics pipelines. Aggregation runs asynchronously through scheduled GitHub Actions: a frequent job updates insight tables approximately every minute, while a daily reconciliation job recomputes the previous two days of telemetry to recover from delayed ingestion or missed runs.

Why

Single source of truth: original telemetry is preserved and never lost through aggregation.
Dashboard performance: charts and statistics can be served without repeatedly scanning large telemetry datasets.
Near real-time user experience: frequent aggregation keeps dashboards responsive while avoiding expensive queries.
Future flexibility: historical raw telemetry can be reprocessed to generate new metrics without requiring users to collect new data.
Separation of concerns: telemetry ingestion and dashboard visualization evolve independently.
Reliability: the daily reconciliation job helps recover from missed scheduled executions, delayed uploads, or temporary processing failures.
Scalability: expensive aggregation work is performed once during ingestion instead of every time a dashboard is opened.

Trade-offs

Raw telemetry and aggregated insights intentionally duplicate part of the same information.
The ingestion pipeline becomes responsible for maintaining synchronization between datasets.
Scheduled aggregation introduces eventual consistency rather than immediate updates.

Result

HealthCam can support growing amounts of telemetry without increasing dashboard query complexity or cloud compute costs proportionally. Dashboards remain responsive, while raw telemetry keeps the platform open to future analytics and AI features.

Engineering decision

Microservices instead of a monolithic architecture

HealthCam is split into independent services for capture, tracking, inference, notifications, orchestration, and cloud synchronization.

Context

HealthCam combines components with very different responsibilities, including webcam capture, application tracking, AI inference, notifications, orchestration, and cloud synchronization. These components require different technologies, release cycles, dependencies, and runtime requirements.

Decision

I separated the platform into independent microservices communicating through lightweight HTTP APIs. Each service owns a single responsibility and can be developed, started, stopped, updated, monitored, and replaced independently.

Why

Separation of concerns: each service focuses on a single responsibility.
Independent evolution: services can be modified and deployed without affecting unrelated components.
Extensibility: new analyzers and platform services can be introduced without restructuring the existing architecture.
Fault isolation: failures remain contained within individual services instead of affecting the entire platform.
Technology flexibility: each service can adopt the tools and runtime most appropriate for its purpose.
Scalability: computationally intensive services can evolve independently from lightweight platform components.

Trade-offs

The architecture is more complex than a monolithic application.
Services communicate over the network rather than through direct function calls.
Additional infrastructure is required for lifecycle management, orchestration, monitoring, and discovery.
More moving parts make debugging and deployment more operationally demanding.

Result

The platform evolved from a single application into a modular ecosystem of services. This architecture became the foundation for HealthCam’s plugin system, service orchestration, and future extensibility.

Engineering decision

Hybrid native and containerized execution

Platform services run natively when they need OS and hardware access, while computation-heavy analyzers can run in Docker for isolation and reproducibility.

Context

HealthCam combines two very different categories of software. Some services need direct access to the operating system and hardware, such as webcam capture, application tracking, desktop notifications, and system integrations. Other components are research-grade computer vision analyzers with complex Python dependencies, GPU requirements, and changing machine learning environments.

Decision

I adopted a hybrid execution model. Platform services execute natively on the host machine, while computation-heavy analyzers are containerized with Docker whenever appropriate.

Why

Dependency isolation: AI analyzers bundle their own Python environments without affecting the rest of the platform.
Reproducibility: containerized analyzers execute consistently across different development and deployment environments.
Native hardware integration: platform services retain direct access to webcams, desktop APIs, notifications, and operating-system features.
Easier maintenance: dependency conflicts and “works on my machine” issues are significantly reduced.
Future analyzers can be added without introducing additional dependency conflicts into the host system.

Trade-offs

Docker becomes a platform dependency for containerized services.
The platform manages two execution environments instead of one.
Deployment and troubleshooting become more operationally complex.
Different execution models require additional lifecycle management and deployment logic.

Result

HealthCam combines the advantages of native operating-system integration with reproducible containerized AI execution. Infrastructure services stay lightweight, while AI analyzers can evolve independently with their own runtime environments.

What this demonstrates

From prototype to platform architecture.

These decisions show how HealthCam evolved from a thesis prototype into a modular platform architecture. The project required product thinking, backend architecture, local service orchestration, data pipeline design, AI integration, and a developer-facing extension model.

Designed local-first AI processing for sensitive webcam data.

Built a microservice architecture around independent local components.

Created the foundation for plugin-based analyzer installation.

Separated raw telemetry from dashboard-optimized insight tables.

Used scheduled aggregation to balance performance and flexibility.

Combined native execution with Dockerized AI services.