I am currently looking for a new knowledge base "home", and then I discovered anytype. Which is what triggered this thought.
I like opinionated software. It attracts the right people, repels the wrong ones, and lets you (as an engineer) focus on the group of people that actually matters. When a product has a clear perspective, you don't have to wonder what it's trying to be. You just need to figure out if your opinion matches.
A/B testing is the opposite of that. And it rubs me in the wrong ways.
It is great, in theory...
A/B testing sounds rigorous. Data-driven! Scientific! You're not guessing, you're testing hypotheses. Which some times it is true.
But something about it has always bothered me, or at least how I've seen it implemented and abused. It shifts the question from "what should we build?" to "let's see what performs better." Those aren't the same thing.
The first requires deep thought, conviction, a perspective, real insights on what's right for your users. The second just requires two variants and enough traffic.
The trap
When A/B testing becomes the default, the incentive shifts. Instead of deeply considering a feature before shipping, you ship two half-baked versions and see which one "wins".
Thinking is slow. A/B tests are fast. And stakeholders ain't have time you know, time is money...
The result is a product shaped by whatever happened to perform better in a two-week experiment, not by a coherent vision of what it should be.
(And btw, some portion of your users got the shitty experience along the way. Where's the care in that?)
What the data doesn't tell you
A/B tests optimize for what you can measure. Clicks, conversions, time-on-page.
What they can't measure: trust, craft, whether this feature should exist at all. You end up optimizing for numbers, not for quality. The numbers tell you what people clicked, not what people respected.
But look... I'm not saying never experiment. Trying stuff is good. Learning from users is good.
But there's a difference between "we believe X is the right approach, let's validate it" and "we have two ideas, let's see which gets more clicks."
The first is thoughtful, the second is outsourcing your decisions.
Pick A. Or pick B. But pick something because you thought about it, not because you're hoping the data will think for you.
Opinionated software comes from opinionated people making actual decisions. That's the kind of product I want to use, and the kind I want to build.