Designing an incident pipeline that scales to zero

Road incidents do not arrive on a schedule. For most of the day there is nothing to process, and then a stretch of highway goes down and a burst of reports lands at once. A server that is always on to handle that is expensive and mostly idle. SafeRoute was built around that shape: an event-driven pipeline on AWS that costs close to nothing when nothing is happening and absorbs a burst when it is.

The intake path

Reports enter through API Gateway and land in a Lambda function that validates them and writes an incident record. The important decision here is not the compute, it is the data model. DynamoDB is a poor fit for arbitrary queries, so the tables are designed around the access patterns that actually exist: current incident by region, incident by responder, and incident history by corridor. Getting that mapping right up front is what keeps the read path fast later.

Orchestration instead of a worker

Dispatch has steps that have to happen in order: assign a responder, notify them, start a timer, escalate if nothing is acknowledged. That is a state machine, not a script. Step Functions models it directly, so the escalation rules live in the workflow rather than in a long-running process that has to stay warm.

report -> validate -> persist
         -> assign responder
         -> notify (SNS)
         -> wait 90s
              -> acknowledged? done
              -> no response? escalate + notify supervisor

Location as a first-class field

An incident without a location is not actionable. AWS Location Service handles geocoding and the proximity queries that answer the only question that matters in the moment: which responder is closest to this point. Keeping geography in a managed service means the pipeline does not carry its own spatial index.

Deploys that carry no keys

The whole stack is described in Terraform, and GitHub Actions deploys it by authenticating through OpenID Connect. There is no long-lived AWS access key sitting in a repository secret. The workflow assumes a role for the duration of the deploy and then the credential is gone. For infrastructure that touches civic data, that is not a nice-to-have.

What I would do differently

• Version every incident record from the start. State changes are the interesting data, and reconstructing them later is painful.
• Cap the burst explicitly. Concurrency limits on Lambda keep one noisy corridor from starving the rest of the network.
• Write the access patterns down before the table definitions. Every index that got added later was one I did not think through.