Skip to main content

Command Palette

Search for a command to run...

You Cannot Review Your Way to Quality πŸ”

Published
β€’8 min read
You Cannot Review Your Way to Quality πŸ”
R
Entrepreneur Β· Embedded & AI Systems Architect Β· 20+ yrs shipping IoT Β· Founder @ Ritzy Lab Β· Building EmbedIQ β€” AI-first firmware framework

What this is.
Experience + Opinion + Fact (50% experience Β· 35% opinion Β· 15% fact)
Written in collaboration with AI β€” I discuss, I do not outsource.


Chapter 1. Hired to Hold the Quality Gate 🏭

A Fortune 500 hardware company. An external vendor doing all the firmware and embedded software. A gap between "we'll handle it" and "this ships to consumers."

My job: sit between the two. Review every pull request. Catch every defect before it reached the product. Be the gate.

It was a clear mandate. I took it seriously. I built the gate, I sat at the gate, and I started reviewing.

The vendor engineers were competent professionals. Experienced. Working hard under real deadline pressure. I want to be clear about that, because it matters for everything that follows β€” the problem I eventually found had nothing to do with their skill.

β–Έ First principle. When you are hired to enforce quality, the first move always looks obvious: review more, catch more. The second move is harder to see.


Chapter 2. The Obvious Solution: Review Everything πŸ“‹

The first few months: a grind of pull requests, inline comments, and back-and-forth threads.

A missing null check that could corrupt a state machine on edge input. Caught it. Written up. Fixed. Merged. Move on.

An implicit assumption in a sensor driver β€” the kind that works fine in the lab and surfaces at 2 a.m. on a factory floor. Caught it. Fixed. Merged. Move on.

I added structure: review guidelines, a shared doc cataloguing common patterns. Checklist items. A process. All reasonable things. All things I believed would work.

For a while, the rate of defects in review slowed. The gate felt like it was holding.

β–Έ First principle. More thorough review catches more defects. That is true. But "catching more" and "solving the problem" are not the same thing.


Chapter 3. Same Class. Different Module. Every Sprint. πŸ”§

This is the moment I kept running into β€” and it took longer than it should have to name it clearly.

It was never the same bug. It was always the same class of bug, appearing in a new module, written by a different engineer, in a different subsystem, every sprint.

A null check missing here. An ownership boundary left implicit there. A state machine embedded in conditionals instead of modelled explicitly. The surface kept changing. The underlying shape never did.

I wrote longer guidelines. I added more detail to the review checklist. The cycle slowed a little. It kept going.

Each module was a fresh canvas. Each sprint was a fresh opportunity for the same structural hole to open somewhere new. I could not write guidelines fast enough to describe every location where the same gap might appear next.

The frustration I felt during that period was not with the vendor engineers β€” they were doing their best inside the structure they were given. The frustration was with the approach. With the loop itself. Review β†’ catch β†’ fix β†’ same class appears somewhere else. Review β†’ catch β†’ fix β†’ repeat.

Something was wrong with the model, not the people.

β–Έ First principle. A recurring class of defect is a structural signal. It is the architecture telling you something β€” not the engineers.


Chapter 4. Review Is a Lagging Indicator πŸ’‘

At some point I stopped asking "how do I catch this faster?" and started asking "why does this keep being possible?"

Here is what I eventually saw:

REVIEW-GATE MODEL (lagging): Code written β†’ Bug introduced β†’ PR submitted β†’ Reviewer catches it β†’ Fix β†’ Repeat ARCHITECTURE-ENFORCED MODEL (leading): Structure defined β†’ Wrong pattern structurally constrained β†’ Violation surfaces at compile/boot/CI β†’ No reviewer required

By the time a bug arrives at a reviewer, the architectural decision that allowed that bug was already made. The module boundary was drawn loosely β€” that happened weeks earlier, when someone was designing the module, under a deadline, without a structural constraint telling them to draw it tightly. The ownership contract was left implicit β€” that happened when there was no typed interface requiring it to be explicit.

Review sees the result. Review cannot see the moment of cause. That moment is upstream. That moment is in the architecture.

β–Έ First principle. Review is a lagging indicator β€” by the time the bug is visible, the structural decision that made it possible has already shipped into the codebase.


Chapter 5. Quality Has to Be Designed In πŸ“

The shift I am describing is not cosmetic. It is a genuine change in where quality lives.

In a review-gate model, quality is a person. It lives in the reviewer's judgment, applied after the fact. It scales as well as that person scales. Which is: not very far, and not very fast.

In an architecture-enforced model, quality is a property of the structure. Typed interfaces between layers make the ownership contract explicit β€” not as prose in a review doc, but as a constraint the compiler or runtime validates. Explicit state machines make behavior visible before deployment, not after. Layer boundaries are enforced, not hoped for. Violations surface at compile time, at system boot, in CI β€” before runtime, before the reviewer ever sees the code.

This is not a new idea in software engineering broadly. But it is a deeply underbuilt idea in embedded firmware. Most firmware projects run as flat procedural code β€” threads, callbacks, globals, and flags β€” with quality enforced through review and tribal knowledge. When you outsource that development, the tribal knowledge walks out the door with the team that held it. What remains is review. And review alone cannot carry the weight.

β–Έ First principle. Structure enforces what guidelines only describe. The right architecture makes the correct pattern the default path and the wrong pattern difficult to reach.


Chapter 6. If You Are Outsourcing Firmware Today πŸ€”

This pattern is not rare. I have seen it across many engagements, in various forms.

A hardware company β€” startup or established β€” hands firmware development to a vendor. The vendor delivers code. Someone internal reviews it. The same class of issue surfaces in different modules. Review guidelines get longer. The cycle continues.

The root cause is rarely vendor skill. The vendor engineers are often good. The root cause is the absence of an architectural baseline the vendor must conform to. Without a framework that enforces structure β€” explicit layer boundaries, typed contracts between modules, no uncontrolled cross-layer coupling, observable runtime state β€” every module becomes a judgment call. And judgment calls, made by different engineers under different deadline pressures, diverge.

The point of intervention is earlier than the first line of vendor code. Define the structure before work begins. Make the right pattern the only available path. A vendor working inside a well-defined architectural framework produces different output than the same vendor working from a blank file and a document of guidelines.

The question worth asking: if your vendor can write code in any pattern, and the only gate is review β€” what are you actually enforcing? βœ…

β–Έ First principle. The architecture you hand a vendor is the quality floor you will get back. No review process substitutes for a well-defined structure handed over before work begins.


Chapter 7. What I Do About This at Ritzy Lab 🀝

That year of reviewing pull requests, watching the same class of defect appear in a new module every sprint, changed how I think about firmware quality. The lesson compounded across subsequent projects. Same pattern, different clients, different chips.

At Ritzy Lab, this is the lens we bring to every firmware engagement. Before code gets written, the architectural contracts get defined. Layer boundaries are drawn explicitly. Ownership is typed, not assumed. The structure that makes violations visible before runtime β€” that comes first, not as an afterthought after review has already caught the same thing three times.

The details of what that looks like in practice β€” what the structural enforcement actually means across different RTOS environments and hardware targets β€” is a longer story.

Next: the specific architecture patterns that make violations visible before runtime β€” and what it takes to enforce them across a distributed team.

β–Έ First principle. The commitment is to put quality in the architecture, not in the review queue. When the structure enforces the contract, the reviewer becomes the last line of defence β€” not the only one.


Labeled: Experience + Opinion + Fact
*(50% experience Β· 35% opinion Β· 15% fact)*

Sources: (none required for this piece β€” all experience-based)

(Written in collaboration with AI β€” I discuss, I do not outsource.)

New to this labeling? Read the framework β†’ https://www.linkedin.com/pulse/20-years-ideas-articulation-craft-heres-how-i-write-now-ritesh-anand-2vwqc

β€” Ritesh | ritzylab.com

#EmbeddedSystems #FirmwareDevelopment #EmbeddedEngineering #IoT #FirstPrinciples

3 views