case study 01 / 09Atlas Labs · Monitoring · 2026

Hetzner Fleet Monitor

Dozens of cloud servers across many projects, and no single place to see their load or be told when one is struggling.

Zero-dependency monitoring stack: an API poller, a SQLite store, an alert engine and a single-file HTML dashboard.

THRESHOLDALERTALERTALERT
servers watched
68servers watched
projects in one view
12projects in one view
poll cycle, in a 120 s window
33 spoll cycle, in a 120 s window
agents or pip packages
0agents or pip packages

01The situation

The company's servers run on Hetzner Cloud, split across twelve projects. The Hetzner console shows one project at a time, so answering "is anything overloaded right now?" meant clicking through all of them.

I wanted one view of the whole fleet and an alert when something stays hot, without installing anything on the servers themselves.

02What it had to do

  • Nothing installed on the servers: read everything from the Hetzner API.
  • Python standard library only, so it runs anywhere without pip.
  • Alerts that don't flap: a short spike shouldn't page anyone.
  • One poll cycle must finish well inside the two-minute interval.

03How it works

  1. Poll

    Every 120 seconds a poller lists the servers in all twelve projects and fetches their CPU, disk I/O and network metrics, eight requests in parallel.

  2. Store

    Readings go into a local SQLite database in WAL mode: servers, metrics, a poll log and alert state. Old rows are pruned on a retention window, with incremental vacuum so the file doesn't grow forever.

  3. Decide

    An alert engine with hysteresis. A threshold has to hold for a sustained period before it fires, it can escalate from warning to critical and back, it resolves on its own and re-notifies on an interval. A failed API call for a whole project is its own alert, not mistaken for a server problem.

  4. Notify

    Desktop notifications and an optional webhook. Notification is fail-soft: if the webhook is down, polling carries on.

  5. Show

    A single self-contained HTML dashboard that refreshes itself, with sparklines per server. No web server to run or secure.

  6. Test without waiting

    A self-test seeds synthetic history and forces each alert path, so the sustain, escalate and resolve logic can be checked in seconds instead of waiting for real load.

04What I found along the way

Serial fetching was too slow

Fetching metrics one server at a time didn't fit the interval. Moving to a pool of eight workers brought a steady-state cycle down to 33 seconds.

The dashboard could be broken by data

Embedding JSON in a script tag allowed a </script> breakout, and some client rendering used unescaped innerHTML. Both were fixed during review, before anything went live.

05What changed

  • One page shows load across the whole fleet instead of twelve console tabs.
  • Sustained problems raise an alert; brief spikes don't.
  • I later generalised it into zeroagent, a standalone version that also supports Linode and DigitalOcean.

06Takeaways

  • The Hetzner API exposes CPU, disk I/O and network, but not memory or free disk. Those need an exporter on each host, which is the planned next layer.
  • A self-test with synthetic timestamps was worth building early: it made the alert logic testable at all.