08 — High Availability, Statelessness & Health

Your app always runs as 2 or more identical copies. Kubernetes can stop and restart any copy at any time — during a deploy, on out-of-memory, or for node maintenance. Any copy must be able to handle any request.

What "stateless" means in practice

No in-memory sessions — a session stored in the memory of copy A is invisible to copy B. Store sessions in PostgreSQL or an external session store (ask DevOps for options). Signed stateless tokens (JWT cookies) also work.
No local file storage — files written to the container's disk disappear on restart, and an upload to copy A won't exist on copy B. Use an external file store (ask DevOps — typically S3).
No background jobs tied to a single instance — a naive in-process cron would run twice (once per replica). Use a proper job queue, a Kubernetes CronJob (ask DevOps), or a PostgreSQL advisory lock so only one replica runs the job.
No in-memory caches you can't afford to lose — per-pod caches are fine for pure performance, wrong for correctness.

Graceful shutdown (SIGTERM)

Kubernetes stops a pod by sending SIGTERM, waiting (30s by default), then killing it. Your app must:

On SIGTERM: stop accepting new connections
Finish in-flight requests
Close DB pools and exit

Node.js example:

const server = app.listen(PORT);

process.on('SIGTERM', () => {
  console.log('SIGTERM received — shutting down gracefully');
  server.close(async () => {
    await db.end();       // close the PostgreSQL pool
    process.exit(0);
  });
  // Safety net: force-exit before Kubernetes' 30s SIGKILL
  setTimeout(() => process.exit(1), 25_000).unref();
});

Health endpoints

Expose two endpoints (both must respond within 2 seconds):

Endpoint	Used by	Semantics
`GET /healthz`	livenessProbe	"The process is alive." Returns 200 whenever the app is running. Must NOT check the database — if it did, a DB outage would make Kubernetes restart-loop perfectly healthy pods.
`GET /readyz`	readinessProbe	"I can serve traffic." Returns 200 when the app is initialised and (if it uses a DB) PostgreSQL is reachable; 503 otherwise. Kubernetes stops routing traffic to a not-ready pod but does not restart it.

Node.js/Express example:

app.get('/healthz', (req, res) => {
  res.status(200).json({ status: 'ok' });
});

app.get('/readyz', async (req, res) => {
  try {
    await db.query('SELECT 1');   // quick DB connectivity check
    res.status(200).json({ status: 'ok' });
  } catch (err) {
    res.status(503).json({ status: 'error', message: err.message });
  }
});

Logging

Log to stdout/stderr only (never to files) — the cluster collects container output. Prefer one JSON object per line, respect LOG_LEVEL, and never log secrets or full request bodies.

Prompt tip

"The app must be fully stateless: sessions in the database, files in external storage, no instance-bound jobs. It must listen for SIGTERM and shut down gracefully (stop accepting requests, drain in-flight ones, close the DB pool, exit). Add GET /healthz returning 200 whenever the process is alive (no DB check) and GET /readyz returning 200 only when the app can serve traffic including a SELECT 1 DB check. Log JSON to stdout, honouring LOG_LEVEL."