Case study · Perfect Price · 2018 – 2019
The point where a customer lets the AI set the price
Perfect Price learned willingness to pay across hundreds of thousands of microsegments and recommended a price for each one. The recommendations were good. Customers used thirty percent of them. Moving that number was the entire job.
- Role
- Director of Product Management
- Dates
- January 2018 to July 2019
- Product
- AI dynamic pricing for large consumer-facing enterprises. Car rental, fuel retail, airline ancillaries, ride hailing
- Team
- Hired and mentored junior product managers, built a team of pricing strategists
What the product did
Pricing at scale is a problem of combinations. A fuel retailer does not have one price, it has a price per grade per station per hour, moving against a competitor across the street, the weather, a payday weekend, and whether the store has a decent coffee machine. No pricing team can hold that in their heads, so they set rules, follow the competition, and add a margin to cost.
Perfect Price learned the price elasticity of each microsegment from the customer's own transaction history plus external data, found where the elasticities disagreed with the current price, and recommended a new one. The recommendations flowed back into the systems the business already ran on, with a simulator and a rules layer on top so a pricing team could see what the model was proposing before it went live.
The problem was not accuracy
I arrived to a product with a churn problem and a good model. Those two facts sat next to each other for a reason. A recommendation engine only pays when somebody acts on the recommendation, and customers were acting on about thirty percent of ours. The other seventy percent were overridden by a human who had priced this station, or this location, or this route for years and did not need a model to tell them what it was worth.
Below roughly half, the product behaved like an analytics dashboard. It produced opinions. Humans priced anyway. Nothing showed up in the customer's revenue that anyone could point at during a renewal conversation, which is precisely why the renewal conversations were hard.
Above half, something different happened. Customers stopped treating each price as a decision and switched on automatic pricing. That is the threshold the roadmap was aimed at, and it reframed the work. The question stopped being how accurate the model was and became what would make a pricing analyst comfortable letting it act.
Share of AI-recommended prices customers actually used
The dashed line is the threshold. Past about half, customers stopped reviewing prices one at a time and turned on automatic pricing.
What moved it
- Segmentation the customer recognized. A microsegment the model invents is a number. A microsegment a revenue manager already has a name for is an argument they can check. Rebuilding segmentation around how customers described their own business is what made the recommendations legible enough to accept or reject on the merits.
- A feedback loop with somewhere to go. Overrides were the most valuable signal in the product and were being thrown away. Capturing why an analyst rejected a price, and closing the loop with them, turned the disagreement into the roadmap.
- Continuous deployment. Time to launch went from months to days. A feedback loop is worthless at a quarterly release cadence, because the analyst who told you the model was wrong about airport locations has stopped caring by the time you answer.
- SOC 2. I started the audit. It is not a product feature and it decided deals. Public company buyers could not sign without it, and the pipeline had been quietly capped by that for a while.
When the data disagreed with the customer
A nationwide mobile windshield replacement company came to us with a pattern it had been pricing on for years. Customers were less price sensitive on Sundays and Mondays, so prices went up on Sundays and Mondays, and the expectation was that the model would find more patterns like it.
I ran the backtest myself, a Markov chain Monte Carlo simulation over their transaction history, and the headline was good. There was room to raise prices in most of the country without losing demand. The Sunday pattern was the problem. It held at the national level and in most states. At the zip code level it dissolved. Individual markets did not share the sensitivity, so a uniform Sunday uplift was quietly overpricing some stores and underpricing others.
The easy read was that the model was wrong, because the customer's number had made money. Instead of asking the data science team to explain it, I went into the data and found the aggregation doing the work. Averages across markets had manufactured a pattern that no single market showed. The customer signed, and the engineers and data scientists started bringing me the odd results before the sales team did. That was the moment the product function had a seat at the modeling table.
Selling something a customer cannot inspect
The commercial funnel had the same problem as the product. Prospects were run through a paid back test, four to six weeks of simulating our prices against their history, and the exercise was close to blind. Nobody had established what this particular buyer would find compelling before spending six weeks proving something to them.
I rebuilt the process around value engineering, which is an old procurement idea and a useful one. Value is worth over cost, where worth is the benefit the customer receives and cost is everything it takes to get there, including their people's time. Increase one, reduce the other, at every step of the funnel rather than only in the pricing of the contract.
In practice that meant splitting a crowded discovery call into a business conversation and a technical one, giving the prospect a framework and an agenda before either, and being honest that a company without the pieces of a working data team was not a customer we should chase. It also meant handing the business case to the prospect. We supplied the measured impact. They wrote the case, in their own language, to their own leadership, which is the only version of it their CFO was ever going to believe.
Three people had to say yes, and they were not persuaded by the same thing.
The evangelist
Pricing and revenue
Wants proof that pricing by hand is leaving money on the table every day, and evidence that their own users will accept a machine price. The natural ally and the person who should own the business case.
The CFO
Finance and procurement
Wants worth comfortably above cost in net profit, and a reason it makes no sense to build this internally. Give at least two commercial models, because an annual charge up front is its own objection.
The CTO
Data science, engineering, product
Wants an execution plan that names their concerns out loud. This group is proud, and a vendor can read as a verdict on their team. The work is convincing them the vendor is not taking anything over.
What the model was worth when it ran
Two results from customer back tests, as the company presented them at the time.
Global car rental firm
Just over $1M a year, from one airport location
Annual profit at a single major airport station moved from $7.9M to $9.0M, a lift of 13.4%, with prices adjusting automatically as the pickup date approached rather than in response to a nervous human watching bookings slow.
Global fuel retailer
3.15% more retail profit at the same volume
Roughly $17M a year, from moving off cost-plus and competitor-following pricing to pricing on measured willingness to pay. Volume did not change, which is the part that matters. This was margin the retailer was giving away, not demand pulled forward.
What I would do differently
The back test carried too much. It was asked to prove the model, discover the business, build trust with the pricing team, and justify the price of the software, all in four to six weeks that a prospect experienced as a long quiet gap. It should have proved one thing and moved on.
More to the point, it proved the wrong thing. A back test tells a buyer the model would have made money last year. It says nothing about whether the pricing team will live with the recommendations on a Tuesday morning, and that is where adoption is actually won and lost. Knowing what I know now, the trial I would run is a small slice of the business priced automatically for a month, with the override button in plain sight and every override read.