gazopsA field guide to complex systems

Systems. Stories. Second thoughts.

Engineering,
unfolded.

An independent publication by Gareth Hunt (Gaz)

Complex systems.
Human consequences.
Let’s open them up.

Read the first story
A small world. A real question.FIELD EXPERIMENT / 001
A release unfolds across a miniature paper worldCustomers send requests through a traffic switch to a stable release or a new release. Increasing exposure routes more requests to the new release. An introduced fault affects only requests routed there.THE CUSTOMERSTRAFFIC SWITCHSTABLENEW RELEASEDeployed ✓What could possibly go wrong?0102
FIG. 01 A deployment can be green while the customer experience turns red.
What this little world leaves out

This model calculates expected failures rather than simulating individual requests. There are 1,000 requests and none fail on the stable release. When you introduce a fault, 20% of requests sent to the new release fail. Failed requests = 1,000 × share of traffic sent to the new release × 20%. For 10% exposure, that share is 0.1. The twenty figures show how much of the audience reaches each release. They don’t count failed requests.

Real traffic varies. A second attempt at a failed request, a dependency used by both releases or a small group of unusually busy customers could change the result. A small test can also miss rare failures. Before sending more traffic to a release, you need evidence that reflects how customers use it and time to observe problems. This experiment doesn’t tell you when it’s safe to proceed.

The stories

Every system has a story.

Start with the decisions behind a release.
One complete story. More on the workbench.

02PLANNED

Application lifecycle

What Really Happens When Kubernetes Terminates Your Pod?

What happens to customers and unfinished work when infrastructure decides an application should stop?

03PLANNED

Health checks

When Your Health Checks Become the Problem

When is restarting an application genuinely the correct response to a failure?

04PLANNED

Autoscaling

Scaling Isn’t About CPU. It’s About Demand.

What are we actually trying to protect or improve when we scale a system?

05PLANNED

Cloud resilience

One Region. One Failure. One Expensive Decision.

How much resilience does a business need and what should it pay for?

The first story is ready to explore. The remaining stories are on the workbench.

The person behind the pages

Hello, I’m Gaz.
Let’s work through it.

I’m Gareth Hunt (Gaz), a platform engineer. I spend my time thinking about how systems behave, how they break and what our decisions mean for the people using them.

GazOps is where I explore those questions through small worlds you can interact with. There’s room for the awkward details too.

Let’s understand the behaviour
before choosing the tools. Gaz