Sancho PM· Philosophy · 3 min read
The Enchanted Helmet
When Data Blinds Instead of Guides
Don Quixote saw a magical helmet. It was a barber's basin. PMs see insights in dashboards that aren't there. Sancho knew the difference.
The Golden Helmet
At one point in his adventures, Don Quixote spots a man on a donkey wearing what the knight insists is the legendary Helmet of Mambrino — a magical golden helmet that makes its wearer invincible.
Sancho squints. “That’s… a barber’s basin, sir.”
Quixote doesn’t care. He attacks, claims the “helmet,” and wears it proudly for chapters.
It was still a basin.
The PM Version
We do this with data all the time.
The Vanity Metric Helmet. “Look at our page views! We’re growing!” Meanwhile, conversion is flat, retention is dropping, and nobody’s actually buying. But the number goes up, so it must be good.
The Correlation Crown. “Users who complete onboarding have 50% higher retention! Let’s force everyone through onboarding!” Never mind that engaged users complete onboarding because they’re engaged — not the other way around.
The Dashboard Mirage. We stare at dashboards, convinced they’re telling us something profound. Usually they’re just showing us numbers we already knew, packaged in prettier charts.
The A/B Test Illusion. “The test won with 51% confidence! Ship it!” Statistical significance ignored for the thrill of a decision.
We see magical helmets everywhere. They’re mostly basins.
The Sancho Approach
Sancho didn’t see magical armor. He saw a bowl. Here’s his approach to data:
1. Call the Basin a Basin
Before celebrating a metric, Sancho would ask: What is this number actually measuring?
The test: If this metric doubled overnight, would you actually be winning? If MAU doubled but DAU stayed flat, what did you really gain?
2. Seek the Metric That Hurts
Quixote avoided information that challenged his fantasy. Sancho looked for truth.
For PMs: Find the metric your team doesn’t want to talk about. That’s probably the one that matters.
3. Data Informs, Doesn’t Decide
Sancho used his eyes to see reality, not to construct fantasies. Data is evidence, not a verdict.
The rule: No dashboard should make a decision. Data shows what happened. Judgment says what to do about it.
4. Sometimes You Just Don’t Know
Quixote was certain of everything. Sancho was comfortable with uncertainty.
The PM equivalent: “We don’t have enough data to know” is a valid answer. Better than pretending a basin is a helmet.
Dangerous Data Patterns
Watch for these signs you’re wearing a basin:
- Cherry-picking time ranges to show improvement
- Celebrating inputs (signups) instead of outcomes (retained users)
- Ignoring segments that don’t fit the narrative
- Confusing correlation with causation
- Waiting for perfect data while reality happens
Each is a basin you’re calling a helmet.
The Monday Morning Test
When someone shows you a dashboard or metric, ask:
- What is this actually measuring?
- What does this NOT tell us?
- If this changed, what decision would we make differently?
- What’s the ugliest truth this dashboard is hiding?
If you can’t answer these, you might be staring at a barber’s basin.
Pragmatic, Not Anti-Data
This isn’t about ignoring data. It’s about seeing clearly.
Data is useful when you know what it is. Dangerous when you pretend it’s something magical.
Sancho would look at the basin and use it — for washing, not for battle. Know your basins. Use them appropriately.
Put this into practice
See through the data illusions. The Success Metrics template helps you track what actually matters.
