80/20 for deciding when it's good enough and where to focus
This is a sample section from High Output Software Engineering.
---
We, as engineers, love to optimize stuff. We love to find the best possible solution to a given problem. But this tendency can backfire if we forget to pay attention to the cost of our optimization in comparison to its benefit. It can become over-optimization.
80/20, also known as the Pareto Principle, states that roughly 80% of effects (positive or negative) in a given system come from 20% of causes, meaning that usually a small portion of inputs generates most of the results.
Applying it to our day-to-day job enables us to see a couple of patterns and its associated learnings:
80% of the value the company creates for customers comes from 20% of the code
That’s the code you need to do your best job, have good test coverage, spend time thinking about the best architecture, choosing the most appropriate patterns, adding top-notch monitoring and alerts, etc. With the other 80% of the code that generates 20% of the value you should aim to:
- outsource if possible
- default to action by leveraging existing patterns in your organization
- keep it simple so maintenance cost is minimal
- don’t touch it if it’s working, even if you see some bad practices in the code
For an e-commerce, for example, the checkout page is the most important user flow so that part of the code has to receive the most love from you. You should prevent technical debt from creeping in here at all costs.
At Orelo, a company where I was responsible for setting up the codebase from scratch, we paid a lot of attention to its main user journeys. Orelo was a podcast player meets Patreon, where users could support their favorite creators and get access to exclusive content, so we paid a lot of attention to the player experience and the payments system.
20% of your efforts to architect a given solution get you 80% of the way to the most optimal solution
Software architecture is an educated guessing game. You make educated assumptions and go about validating them by gathering more data (researching online, talking to stakeholders/peers, doing a proof of concept, etc.). Once you have new data you create a new (and now even more educated) assumption that is closer to the optimal solution and repeat the validation cycle.
But this educated guessing game follows a steep diminishing returns pattern. To get from 80% of the optimal solution to 90%, you might spend double the time you already spent to get to 80%.
Sometimes, in cases where you are architecting medical devices or rocket ships where lives are at risk, this cost is necessary. But for the reality of most of us (yes, I’m talking to you, software engineer of an email marketing startup with zero users 😆), it’s definitely not worth it.
Planning, when taken to the limit, is like execution. In order to have a perfect plan, you’d need to think of every line of code you will add/change/remove, and this defeats the purpose of the planning phase, which is to estimate time and remove the main project risks ahead of spending the bulk of the project total time. In a bit I’ll share the most important areas you should focus on during planning to reduce its risks (the unknown-unknowns and known-unknowns).
20% of the software engineers in a company create 80% of the value created by the engineering team
In this book, you are learning the mental models that enable you to be in this 20% group, which is a really good position to be in for reasons already discussed in the introduction (more opportunities and more negotiation leverage).
80% of the unnecessary complexity in a system comes from 20% of the company’s software engineers
The Pareto Principle also works when you invert it.
This is a counterexample that is important to understand so that you do some self-analysis and self-correction if you realize you may be currently falling into this 20% category.
Some engineers optimize for technical KPIs without product and business context. Because they are usually really smart and can articulate well the ins and outs of a given technology, they end up gaining decision power from managers and directors that are not living in the day-to-day of the codebase. They make decisions that are locally optimal from the standpoint of a given technical KPI but that are globally terrible from the standpoint of maintainability and complexity.
Some real-life examples I’ve seen:
- writing a new service in Go because Go was a “superior language” while the whole company was Node/TypeScript based. Result: higher context switching cost and steeper learning curve for most of the engineers moving to that team. The service ended up being refactored back to Node/TypeScript.
- adding Kafka without a real business need to manage some new queues while the whole company was AWS SNS/SQS based. Result: two messaging systems to maintain, outages in production due to unknown-unknowns about using this new technology.
---
This is a sample section from High Output Software Engineering. If you liked this one, you might enjoy the other samples available:
- Why read this book?, What it is?, Who is it for?, and How it works?
- Defining value as a software engineer
- The hidden costs of additions to a system (and why you should fight them)
- When in doubt, overcommunicate to prevent possible misalignments
If you're a software engineer working on product teams you might enjoy my book High Output Software Engineering — it's all about understanding value creation, making value-driven trade-offs, communicating with excellence and knowing how to navigate organization dynamics.
Buy it now for $0.99 (PDF & EPUB instant access)