Brendan Gregg's Blog

It's a made-up computer word that my word processor decorates with a wiggly red you-can't-spell line. At least it did until I clicked "Add to Dictionary" (it got too annoying as I was writing a book on computer observability). Some people abbreviate it as o11y.

Observability: The ability to observe.

Observe-ability. Observability.

In computer engineering we use it to describe the tools, data sources, and methods for understanding (observing!) how a technology is operating. We don't use the real word "observable" since that implies the wrong thing. Imagine "observable metrics": Are there metrics that aren't observable?

Using observability in sentences:

• What observability tools are installed? (Means: What tools exist that only read state?)
• What observability does that database have? (Means: What metrics and logs does it have?)
• How do you do observability? (Means: What products do you use for metrics, tracing, etc.?)
• Let me try some observability first. (Means: Let me look at the system without changing it.)

Wait, aren't all performance tools observability tools? No. Experimental tools change the state of the system to understand it. For example, benchmarks.

As an analogy, a car's dashboard is a collection of observability tools that let you understand how the car is operating (speed, rpm, temperature). A car's 0-60 mph time is an experiment.

When I was a performance consultant I'd show up to random companies who wanted me to fix their computer performance issues. If they trusted me with a login to their production servers, I could help them a lot quicker. To get that trust I knew which tools looked but didn't touch: Which were observability tools and which were experimental tools. "I'll start with observability tools only" is something I'd say at the start of every engagement.

Note that observability tools aren't completely harmless: Their execution consumes resources, usually negligible, but in some cases it's enough to perturb the target of study. This is the "observer effect."

Another use of the term observability is as a reminder to switch between tool types, and not to get stuck on one. A colleague (Roch Bourbonnais from memory) once told me:

"You have two hands. Observability and experimentation."

It stuck with me as it also makes the point that when you're only using one type to solve a performance problem you're working one-handed.