Skip to content
Chukwudum Chukwuedo

Case study

National surveillance and surge readiness

Architecture and engineering standards for a national public-health surveillance platform designed to absorb a tenfold surge without losing trust in the data.

Evidence status: confirmedDisclosure: Anonymised

Role
Senior Systems Architect & Technical Lead
Timeframe
Multi-year programme
Sector
National public health
Capabilities
Enterprise architecture · Technical leadership · Data engineering · DevOps · AI engineering
Evidence
confirmed
Disclosure
anonymised

System view

Architecture and relationships

Diagram not yet published

A publishable system diagram is not yet available. The case study describes the architecture and organisational relationships without exposing client detail.

Decision records

The choices that shaped the work

Decision 01

Design for surge, not steady state

Alternatives
Tune the existing steady-state pipeline; add capacity only after a surge; or design the ingestion and processing path for a defined surge envelope.
Chosen
Design the pipeline for a 2–10m records/day surge envelope while keeping the current 250k records/day path operable.
Why
Surge capacity is an architectural property. Retrofitting it during an incident creates the highest-risk change at the worst possible time.

What would change my mind

A validated demand model showing surge volumes materially below the design envelope, or a constraint that makes the surge design uneconomic without reducing public-health readiness.

Context

Public-health surveillance is a trust exercise performed at volume. The programme receives data from multiple sources, validates it, and makes it available to analysts and downstream services. The system has to remain legible to the people accountable for it, not just operational for the people running it.

That distinction matters in a national setting. A late feed, a rejected record or a schema drift can change what a public-health team believes about the population it serves. The architecture has to make those conditions visible.

Challenge

Current throughput was around 250,000 records per day. The design target for surge conditions was 2–10 million records per day. That is not a problem solved by adding replicas. It changes the behaviour of ingestion, validation, storage and downstream processing at exactly the moment when the programme has the least room for error.

The programme also carried a serious control environment. Caldicott principles, data transfer agreements, a DPIA, CAB change control and multi-environment testing all shaped what could be changed, by whom, and with what evidence. The data path included scheduled ingestion, JSON and XML feeds, SQL Server and SSIS components, Azure services, SFTP transfers, and AWS SQS and Lambda for event-driven work.

The ambiguity was organisational as much as technical. Several teams owned parts of the path. The programme needed standards that could survive hand-offs between architecture, DevOps, frontend, backend and AI engineering without turning into a central bottleneck.

My role

I was Senior Systems Architect and Technical Lead. I worked across business analysis, enterprise architecture and technical leadership. I set and enforced programme-wide engineering standards covering architecture, DevOps, frontend, backend and AI engineering, and I was hands-on when a team needed a decision or a blocker removed.

My accountability was technical. I did not own the programme budget, financials or line management. The value I added was making the technical system coherent enough for the delivery system to move without losing control.

How it fits together

The platform is best understood as a set of controlled movements: source ingestion, validation, processing, storage and consumption. Some feeds arrive on a schedule; others arrive through managed file transfer. The pipeline has to preserve provenance and handle malformed records without silently dropping them.

The architecture spans Azure and AWS services. SQL Server and SSIS carry parts of the data path; SQS and Lambda support event-driven work. Scheduled ingestion and file transfer sit alongside service-based processing. The important design question is not which product sits in which box. It is where the system can absorb delay, reject bad data, replay a feed and still give an operator a truthful answer.

I have deliberately not published the detailed architecture. The system view above is a placeholder until a diagram can be redrawn without exposing client or control detail.

The trade-offs

The decision record accompanying this case study is the one that shaped the architecture: design for surge, not steady state. It changed how we thought about queueing, back-pressure, replay, environment parity and test data.

A steady-state design would have passed the current-volume test and failed the moment the programme needed it. Designing for the surge envelope forced the team to name the operating conditions in advance: what happens when a feed arrives late, when a validation rule rejects a batch, when a downstream consumer is unavailable, and when the volume is several times the normal day.

That decision also changed the standards conversation. A standard that only described the happy path was not a standard. It was a preference.

Trust and failure

Caldicott, DTNs, the DPIA, CAB and multi-environment testing were not paperwork bolted onto delivery. They defined what counted as a safe change. The hardest failure mode was not a crashed service. It was a pipeline that continued to look healthy while a feed was late, a schema had drifted, or a validation rule had quietly rejected records.

We treated observability and reconciliation as part of the architecture, not as a monitoring afterthought. A pipeline that cannot explain what it did with a record is not trustworthy, even if every dashboard is green.

Delivery

Delivery was through technical leadership rather than line management: architecture reviews, standards, design sessions and unblocking. The aim was to make the right thing the easy thing, so teams did not have to rediscover the same decisions at each boundary.

That meant working with delivery leads, engineers and control functions in the same conversations. It also meant being willing to make a call when the architecture had drifted into a local optimum.

Evidence

  • Current throughput of around 250,000 records per day.
  • A surge design target of 2–10 million records per day.
  • Cross-environment pipelines and multi-environment testing.
  • Caldicott, DTN, DPIA and CAB controls embedded in the change process.

These are programme facts, not a claim that the surge target was exercised at full volume in production. The distinction matters, and it is part of the evidence.

Reflection

The architecture was strongest when it made failure visible. It was weakest when the standards were treated as a document rather than an operating practice.

What did not work

Two things did not work well enough. Standards that were published without a named owner became documentation drift. And surge capacity that was described but not exercised was an assumption rather than a capability.

I would define the evidence for surge readiness earlier, including the tests and operating drills that would prove it. I would also spend more time with the teams who had to live with the standards, because adoption is a design constraint, not a communications task.

Evidence plate

Evidence visual not yet published

Evidence in this case study is drawn from approved programme records. Figures are attributed to their source and are not recombined into new claims.