Missing the Flight: On Product Priorities
Your P0 is overflowing, your P1 is a catch-all, and your P2 is at the gate watching the doors close. On planning, expected value, and why none of this is bad luck.
I remember waiting in the priority security line at a major airport a while ago. Across the rope barriers, I could see the substantially longer queue for the others - the low-priority customers. Since many of us, by virtue of being at the airport at this specific time, would depart on similar if not the same flights, this priority bucketing already makes little sense. The TSA officer on the lower-priority line was also working faster than ours, based on the same insight: people are not happier if they get through security a few seconds faster, they’re mad if they miss their flight. That’s the win (or lose) condition here.
Nevertheless, our queue was marked mostly by people complaining the priority line wasn’t priority enough - surely they could show up to the airport with 5 minutes of buffer before the gate closes and security should be instant. I reckon the only priority level high enough for them would be a single, personalized security checkpoint.
I’m reminded of this situation every time I go through planning, no matter what type of planning. Strict, nested OKRs; flexible YOLO mode; planning poker; as an IC or as a PM manager. P0 buckets grow because each item individually is critical. P1 becomes a catch-all. Anything beyond doesn’t have the slightest chance of ever making it - P2 features miss their flight every time.
So why do it? Why did the airport do it? Well, a few reasons apply to both:
Expected value - the only poker concept actually relevant to planning. EV is the expected profit (or loss) for a given action, averaged over the long term. Positive EV doesn’t guarantee a specific individual outcome, but reflects that an action is strategically profitable. Priority passengers will miss their flight individually, sometimes. But enough passengers get through security faster, a sufficient percentage of the time, to balance the cost of a separate lane with the profit made from ticket upgrades.
Second-order effects on efficiency. The choice of priority bucketing, even if arbitrary, often forces items with similar properties to self-sort into those buckets. These repeatable, statistical properties can then be exploited to build processes around. A passenger jumping a rope will be tackled by security. A reasonable well-aligned org will similarly tackle ad-hoc requests by forcing them into the sort order: a last minute P0 is either not happening at all, or has to get in line, behind the other P0s.
Both of these concepts are often misunderstood and misapplied, but not intentionally. EV is a concept that runs counter to human nature: counter short-term gains, vulnerable to strong individual anecdotes (massive trap for leaders here!), subject to process rot that erases important records of historical performance.
Borrowing another concept from Poker: ignoring EV and making decisions based on immediate outcomes is knows as being results-oriented. This is a nice euphemism for a (financially) catastrophic short-sightedness. I won this round so I must have been playing well. I lost this round, but that’s just bad luck. We delivered a lot this quarter ahead of plan, it must have been our prioritization working! Ah, this quarter we didn’t, let’s do a retro and redo our planning process.
The second order effects are even more insidious, because they run counter to the human desire to clearly label items to avoid complexity, and then die on that hill. This symptom is worst in organizations where the end-to-end value creation is obfuscated from individual participants. This sounds abstract, but let’s work through a toy example - warning math ahead! But stick with it, it’s useful for planning (and winning at poker).
Company C’s value creation is the delivery of software features. Financial planning and forecasting happens roughly every quarter, and features created by the Product Engineering team P are marketed by marketing team M.
P is in planning right now, and is considering two features, with a high-level understanding that only one of them can actually be P0. Feature F(M) is a deep integration that M wants to headline at their annual user conference. F(L) is the feature the whole industry is suddenly hyping. L has become an “emergency” in planning, three competitors have shipped or announced something similar, so L (Leadership) is twitchy about it.
L’s P0 is obviously F(L). But a feeling of emergency is not a strategic decision, so much as it is being results-oriented. The conference lands in week 10 of this 13 week quarter, so the upside is nearly fully owned by M. M will put F(M) in front a room of high-intent buyers. If we force M to reconsider, we pay the cost: last-minute changes to materials, meetings that waste further time and money, etc.
If we ship F(M) we’re looking at $2M additional pipeline generation, and we’re nearly there. Confidence to ship is at 70% or higher. If we don’t - no matter what we actually do, we’re just looking at the inverse - we’re paying the aforementioned cost, say $400k. Our EV is
For F(L), the upside is unclear and thus smaller. Maybe the hype cycle buys us new trials or logos on the order of $0.8M. Whether we make it is a coinflip. We’re considering it at all, so it can’t be less than 50%, such a long shot at this point would be blatantly insane even for the most results-oriented person. So we’ll take the worst case 50%. On the downside, we’ll likely need to go through the same scramble and meeting battery as before, so the cost is similar. Our EV is
The decision is clear: ship F(M), not F(L). Again, a critical concept to understand is that it’s technically possible for this decision, if forced right now, to be profitable. But in the long run, if we do the same next quarter and beyond, we will provably lose money (and some sanity). The worst possible outcome, and I’m sure you’ve experienced this before at some point, is exactly that: L forces F(L) through and realizes profit, which biases their approach going forward, souring the collaboration between P, M and others. This doesn’t even have to be intentional, nor does the realized equity (actual profit vs. probable profit) have to be higher than the alternative - it just has to be enough not to be an obvious fail. This starts a vicious cycle based on a single lucky accident, which people are then scrambling to try to repeat.
The second important concept to understand is why to involve M at all. Surely P’s planning should determine what M will announce? This is the second-order effect coming into play. Because in the real-world, EV decisions aren’t made in isolation. Multiple actors constantly make both results-oriented and EV-based decisions concurrently. The plane has a departure time, scheduled based on a complex system of operational efficency. The plane, air traffic control, and other M-type actors in that example don’t care about the perks dished out by the airline’s security upgrades. But both need to work together to create overall positive EV for air travel. In the same sense, M’s scheduling, messaging and customer outreach around the event is a deadline nearly independent of the near-term planning considerations. It’s been kicked off seconds after the last event took place.
So EV informs priority, but EV itself is framed by the external constraints. If the event isn’t part of this quarter, EV flips. If event pipeline generation is $2.0M, then off-cycle pipeline growth is organic, maybe closer to $0.5M. Changing direction incurs the usual process churn, which is a less dramatic $100k. F(M) EV is now at
Still positive, but more of a coin flip. Now L and P have time to dig deeper into the probabilities and the up/downsides of F(L).
The looming flagship event is a conveniently clear example, but I’ve seen this play out in reality. A similar clarity comes from customer churn - suddenly there’s a specific downside and usually a specific feature gap cited (whether they’re actually related is a different topic). But the key insight is that planning benefits from shifting prioritization from short term results to long-term EV.
The second key insight is that unilateral planning is a financially unsound model. P can’t just ask M for their schedule, because M’s upside depends on P’s value creation. We’ve worked through a single-sided EV example, but in institutional economics, this is a multi-way equity problem. Before you start to add EV calculations and Monte-Carlo outcome simulation to your planning cycle, let me stress what you should take away from this: priority isn’t solely dependent on what you can do, and what you want to do, but needs to start by taking stock of all the high-equity parties and externalities that matter. There’s not enough time in real life to run the numbers (and not enough computing power in the universe), but you need to build this muscle early and exercise it often. Spot results-oriented behavior before it becomes the process.
Complexity is the core limit on applied game theory, something you may have noticed by now is underlying large parts of institutional economics. Game theory’s core concept, the Nash Equilibrium, is a good mental test to apply once priorities are settled, with one caveat. Players have reached an equilibrium when no single player can unilaterally change strategy to improve their outcome. That’s a statement about stability, but omits judgement about whether the outcome is any good. A famous example is the the Prisoner’s Dilemma: it sits at equilibrium and both players lose. So don’t read the equilibrium as the right answer. Read it as the question it forces: can any party here improve by moving on their own?
Again Poker leads us to the answer: we have to find the Edge. In theory terms, we’re trying to exploit a leak, a deviation from the game-theory-optimal (GTO) equilibrium. In the game, this is where we transition from the math to considering the real behavior and constraints of the system. This is how we win against competitors even with extremely similar offerings, we find the leak in their approach to the market, and they try to do the same. That’s what institutional planning is. Nobody runs the full solve. We find the least-worst option given how every party actually behaves, and we rank it by probability and the long run. The result you happened to draw this quarter doesn’t get a vote. Something will feel off to every party involved. Track the outcomes over enough cycles and you can prove it was the right call anyway.
OK, last but not least, what should the actual priority levels be? How many buckets are useful? Well that depends on how granular you can get everyone’s equity thinking to be. Over the past decade, the median planning cycle for my work was always roughly a quarter. And the lowest common amount of buckets that I found useful are three:
P0 or “critical” means we’ve identified specific milestones that have a clearly superior upside of other work considered, and there are enough externalities to make this work a required commitment. We will shift and lift capacity to make this happen. Other motions have already kicked off: GTM teams are primed, customers have been given promises, there’s an external deadline with upside coming up, and our upside probability is >75%. There may be multiple P0 as parts of our team are fungible, and other may not be, so some work can happen concurrently and the different P0 are ordered appropriately.
P1 or “committed” means we know exactly how much financal and human capacity we are going to commit to this work. We may not hit a major milestone, or EV might be less well defined (upside >50%), but there is a clear outcome, and our available capacity goes towards the least worst options, in order of realizable equity. Least worst means we literally can’t think of anything else, and nothing has come up in the feedback loop with stakeholders that would majorly influence the order here. If you’re methodically inclined, you can fix the lowest and highest estimate for upside probability and rank items within P1 by distributing the delta using quick Paired Comparison Matrix exercise (though consider your own social EV before suggesting that perhaps).
P2 or “aspirational” means we’re not likely to get around to this, but we expect to have some rounding error of capacity, and we’re sending a message where to allocate this if it becomes available. As mentioned before, that rarely materializes, but I’ve seen it happen a few times. First and foremost P2 is a great social communication tool: it allows you to send a clear message to stakeholders whose input ranked lower than they would expect, and contextualizes the tradeoff better than a simple “No”.
Any further levels convey a wrong sense of precision in my experience. Worse, they allow stakeholders to claim their own special spot. Imagine each priority passenger would get their own, personal priority number and the consequences that would bring.
Finally, keep a record. My spicy take is that planning retros are not records, as they exacerbate the worst biases and encourage pure results-oriented thinking. Track decisions long-term, and point back to clearly positive patterns in that record. If a new decision is pending, and there’s no comparable record, the EV (calculated or intuited) must be even more positive.
Remember this
You won this quarter, but that doesn’t mean you played it well. Only the record across many cycles tells you whether it was EV or luck, and results-oriented thinking will lie to you about which.
Priority doesn’t start with what you can do or what you want to do. It starts with every party whose equity is on the line, and the deadlines they set before you got in the room.
Good planning feels a little wrong to everyone at the table, that’s the least-worst option doing its job. Keep the record so you can prove it was, once the results are in.
Three. The answer is three levels. No, adding another won’t help.