Good Morning Vroomers,

Today’s Daily Vroom is going to be very different, and I should probably warn you that it’s going to be a bit of a rant.

There’s something that’s been bothering me for a long time, and after spending much of the past year looking into how the collector car industry determines what cars are supposedly worth, I’ve come to a conclusion.

This whole side of the market is broken, and frankly, I’m amazed at how much of what’s out there people are expected to trust.

That’s quite a statement, I know, but hear me out.

Over the past year I’ve spent an enormous amount of time digging into this subject. I’ve looked at pretty much every established valuation product I could find, studied how they arrive at their numbers, what information they’re using, how frequently they update it and, most importantly, whether those numbers bear much resemblance to what cars are actually selling for.

Then, after I wrote about the growing number of garage apps a while back, my social feeds started filling up with advertisements for all these new AI powered valuation tools. Naturally, I started trying them as well, and boy, there’s some trash out there!

What’s particularly frustrating is how impressive some of them look. Beautiful websites, clever interfaces, all sorts of slick charts that make you feel like you’re dealing with something incredibly sophisticated. You type in a car, a number appears, and apparently that’s what your car is worth.

Except that quite often, when you look at how they’ve arrived at that number, it’s complete nonsense.

And I’m not talking about someone being 5% too high or 10% too low. Nobody gets every car right, and I wouldn’t expect them to. I’m talking about fundamental problems with the information being used, how cars are being compared, and in some cases whether the transactions behind these supposed valuations even happened.

The worrying part is that most people using these sites/apps aren’t going to spend hours investigating the numbers. They’re going to assume there’s something meaningful behind them, particularly when the website looks so professional.

And that’s a problem, because people are making decisions involving tens, hundreds of thousands and sometimes millions of dollars.

To be clear, this isn’t just about the new arrivals. Some of the established names have been doing valuable work for years, but they’re also getting things wrong, albeit in different ways.

The more I’ve looked into this, the more convinced I’ve become that the problem isn’t a shortage of information. It’s that far too much of that information is being turned into answers that simply aren’t justified.

Let’s start with something that drives me mad.

You type your car into a website, press a button, and almost immediately you’re told it’s worth $86,450.

Really? How on earth did you arrive at that?

I’m not suggesting a website shouldn’t be able to give you an answer quickly. Of course it should. But anyone giving you one definitive number without showing how they arrived at it isn’t doing the job properly.

Before I even begin to trust that figure, I want to know what cars went into the calculation. How many actually sold? Were they comparable? Were they the same generation, body style and transmission? What about mileage, specification and when they sold?

And were they even real completed sales?

Because all of those things matter, sometimes enormously.

Take the E46 BMW M3. Anyone who follows these cars knows there’s a difference between the six speed manual and the SMG equipped cars. Their price ranges certainly overlap, but they can sell in quite different parts of the market and often attract different buyers.

Yet I’ve seen tools happily lump the two together and produce one range for a 2004 M3 without separating the two.

How can you possibly give someone a meaningful answer if you haven’t established something as basic as that?

I’ve also seen a figure for a Porsche 911 based on around 1,200 transactions, with the middle half of the results running from $60,000 to $168,000. Now, that’s an impressive number of transactions, but what exactly is the owner supposed to learn from such an enormous range?

We already know Porsche 911s sell for vastly different amounts. What we need to know is what the relevant ones are actually bringing.

That’s the distinction so many of these tools seem to miss. They appear to think that having a large amount of information automatically makes the answer better, when quite often it’s the exact opposite.

I’d much rather see three genuinely comparable sales than a beautifully presented average of 200 cars that nobody in their right mind would compare against each other.

And if there aren’t three? Fine. Show me two, or one, or simply tell me there isn’t enough evidence.

There’s nothing wrong with not knowing. There’s an awful lot wrong with pretending you do.

Here’s another problem that I keep coming across, and I genuinely struggle to understand why it’s still happening.

Since when did an asking price become a selling price?

A dealer advertises a car for $150,000. Does that mean the market is paying $150,000 for that car? Of course not! It means someone would like to receive $150,000 for it, which is a very different thing.

That car might sell tomorrow for $145,000, it might sit there for nine months and eventually sell for $125,000, or the owner might decide not to sell it at all. We simply don’t know.

Yet there are tools scraping dealer listings, feeding those asking prices into calculations and presenting the result as though it represents actual market transactions.

I came across one example involving a 1989 Porsche 911 where the headline figure was an incredibly precise $226,697, despite the underlying information relying heavily on advertised prices.

Why $226,697? What makes that extra $697 remotely meaningful?

I saw another example where a tool produced a figure for a relatively recent 911 based on just three sales, while around thirty dealer listings shown on the same page were asking less. Now, dealer asking prices aren’t completed sales either, but surely such a glaring difference should at least prompt some questions before the website confidently announces its answer.

And it gets worse when unsuccessful auction results are thrown into the same calculations.

Take a Mercedes-Benz 280SE Cabriolet that was bid to $260,000 in July without selling, before finally selling for $240,000 in September.

What did the car sell for? $240,000. That’s the transaction.

The earlier $260,000 bid is interesting information, particularly if you’re trying to understand what happened during the car’s previous auction, but it wasn’t a sale. The seller didn’t accept it, nobody bought the car for that amount, and it certainly shouldn’t be treated as another completed transaction.

Yet I’ve seen tools putting asking prices, rejected bids and completed sales into the same chart as though they’re all interchangeable.

They’re not, and anyone who understands the auction market should know that.

Then we’ve got buyer’s premiums, which are another source of inconsistency. A car hammers at $18,750 but gets reported as $20,063 including the buyer’s fee. Elsewhere, another platform reports only the hammer price. Unless you account for those differences, you can end up comparing two entirely different definitions of a selling price.

And these aren’t obscure technicalities that only someone like me, who spends far too much time looking at auction results, should care about!

They’re the absolute basics.

If you’re building a product whose entire purpose is to tell people what cars have sold for, surely the first requirement is knowing what actually constitutes a sale.

Now, I do want to give the established names some credit here, because quite a few of them have done an enormous amount for this industry over the years.

They’ve spent decades collecting information, documenting cars, publishing results and helping make what was once a fairly opaque market much more transparent. That’s genuinely valuable work, and I have a lot of respect for the people behind it.

But that’s also why I find some of their shortcomings particularly frustrating. These aren’t companies that have thrown a website together over a weekend. They have the experience, the knowledge and, in many cases, the resources to do considerably better.

So when I see outdated figures, averages that tell us almost nothing and comparisons that don’t seem to make sense, I find myself wondering why this is still happening.

Here’s one example I came across.

A guide showed a 1994 Porsche 911 Cabriolet in good condition at approximately $50,000 to $64,000. On the very same page were two recent sales at $95,025 and $95,550.

Now, I’m not suggesting those two sales automatically mean the guide is wrong. There could be significant differences in mileage, condition, originality or specification. But that’s precisely what needs to be explained.

Instead, you’re left staring at a guide figure and two actual sales that appear to tell a very different story.

To make matters more confusing, the twelve month summary on that page pooled a much wider selection of 911s from 1990 to 1994 and showed an average of $246,400.

I’m sorry, but what on earth is someone who owns a 1994 Cabriolet supposed to do with that information?

And that’s before we get into how frequently some of these guides are updated.

I’ve been looking at figures published in July while sitting here in October, knowing full well that plenty of relevant cars have changed hands in the meantime.

A market guide updated periodically can still be useful for longer term reference, but let’s not pretend a number produced three months ago tells us where a particular car sits today.

Cars are selling every single day, and the market doesn’t politely wait for the next quarterly update.

I also find some of the analysis on offer rather extraordinary. One example showed a benchmark of $71,833.33 based on three cars.

Three transactions, and apparently we’ve managed to determine the answer to the nearest cent!

If there are three relevant sales, just show me the three cars and what they achieved. Let me see whether one had significantly fewer miles, whether another had an unusual specification, or whether the most recent result was substantially different.

That’s far more valuable than pretending the calculation has somehow discovered a precise figure.

I have a particular problem with averages in a market like ours, especially when they’re presented as the answer to what a particular car is worth.

I came across one example involving the Porsche 356 Carrera where the average was shown as $433,200, based on twelve transactions. Sounds reasonable enough until you look at the actual sales and discover that one car brought $857,500, pulling the average upwards, while most of the other cars sold for nowhere near that figure.

What exactly is that average telling the owner of a 356 Carrera? Not very much, as far as I’m concerned.

Now, a median is certainly better in some circumstances, but it’s hardly a magic solution either. If you’ve got three sales, the median is simply whichever car happened to sell for the middle price. Giving that number an official sounding label doesn’t suddenly make it representative of the entire market.

And then there was another example showing a headline figure of almost $3.8 million, alongside a chart ranging from approximately $1.56 million to $6.7 million. One of those figures wasn’t even a completed sale, but a bid that nobody accepted.

This is what I mean when I say that the calculation can be perfectly tidy while the information behind it is completely misleading.

I don’t want the average of a group of cars that happen to share a model name. I want to know where the relevant cars have actually been selling, how many sales support that range, and why those particular cars were chosen.

Surely that’s a lot more useful than being handed an average or median and expected to trust it.

The same goes for market trends. I came across a chart claiming a particular market had jumped 156% in a month. But was that actually the market moving, or did a small number of much more expensive cars happen to sell during that period?

Those are two very different things, and unless you’re accounting for the mix of cars being sold, a huge percentage increase can tell a completely misleading story.

I love data, as anyone who’s been reading TDV for a while will know. But one of my biggest annoyances is being shown twenty charts, moving averages and confidence scores when the information underneath them is at best misleading and at worst simply wrong.

More data on a screen doesn’t automatically mean more useful information.

In fact, the better you understand the market, the more you realize how much of that information needs to be filtered out before you can begin to give someone a sensible answer.

Now we get to the newer AI products, because this is where things are starting to become particularly ridiculous.

I should say that I’m a huge believer in AI. I use it extensively, and I think it’s going to transform plenty of industries, including ours.

But there’s an enormous difference between using AI to make a complicated subject easier to understand and expecting AI to magically understand a market just because you’ve given it access to some information about cars.

And I suspect that’s where a lot of these newer products are going wrong.

It’s now possible to put together a genuinely impressive looking website in remarkably little time. Add an AI conversation, some automotive information and a few attractive graphics, and suddenly you’ve got something that looks like it knows the answer to everything.

Until you actually start testing it.

I’ve had conversations with these tools where one minute they’re explaining how difficult it is to establish what a particular car is worth, and the next minute they’re confidently announcing a figure of $150,201.

Where did the $201 come from? What suddenly made the system so certain?

It can be almost comical, except that someone relying on the answer might not realize how questionable it is.

The thing is, AI is extremely good at communicating information. But that’s not the same as being able to determine whether the information is correct.

It doesn’t automatically know which auction results represent completed sales, whether two cars are genuinely comparable, how to deal with different generations and specifications, or whether the information available is sufficient to support a conclusion.

You need proper rules, algorithms, market knowledge and a huge amount of testing behind the scenes. And even then, there are going to be examples where the answer isn’t straightforward.

That’s something I’ve come to appreciate enormously over the past year. The deeper you get into this, the more complicated it becomes.

Which is why I find it so frustrating when someone puts a nice interface around questionable information and effectively says, “There you go, that’s what your car is worth.”

Some of these sites also seem far more interested in getting you to upgrade than helping you understand the original answer. You get a headline number, but want to see what transactions support it? Upgrade. Want more relevant comparisons? Upgrade again.

I’m all for charging for a genuinely useful service, but surely the evidence supporting an answer shouldn’t be treated as some optional luxury.

And it isn’t only the newcomers. I went through one established product that wanted me to create an account and fill out more than twenty fields before it would tell me anything useful. It had “modified” switched on by default, counted asking prices and cars that never sold alongside completed sales, and didn’t even show the individual prices behind its calculations.

After all that, it handed me a range so wide it was barely an answer. And if I wanted anything more useful? That was, naturally, in the paid version.

Think about that for a moment. You’ve made someone jump through all those hoops, mixed together different types of transactions, hidden the evidence and produced a range that doesn’t really help them. Then you ask them to pay for more.

Surely we’ve got this completely backwards.

It feels as though we’ve reached a point where it’s easier than ever to build something that looks authoritative, while the actual work required to make it authoritative is being overlooked.

And that, to me, is a much bigger problem than whether a website has a particularly clever AI feature.

There’s one more thing I want to mention because it says a lot about how I think this whole subject should be approached.

Most of these tools will happily show you prices for other cars of the same make and model, but many won’t immediately tell you whether the exact car you’re looking at has sold before.

Surely that’s something you’d want to know?

Take a 2010 Lamborghini Murciélago that sold for $865,000 and then changed hands again six months later for $700,000.

That’s $165,000 less for the exact same car in just six months.

Now, I wouldn’t automatically interpret that as the Murciélago market dropping by that amount. There could be all sorts of reasons for the difference, including condition, presentation, timing, the auction platform and who happened to be bidding.

But the car’s own selling history is enormously relevant. I’d certainly want to know about it before being shown some generic number based on other Murciélagos that might have very different histories and specifications.

And this gets to something I think the whole industry needs to understand.

When someone asks what their car is worth, what they’re really trying to understand is what the market is telling them.

Not what someone is asking for a similar car. Not what a seller turned down six months ago. And not what an average of loosely related cars happens to produce.

They want to know what cars genuinely like theirs have actually sold for, how recently those transactions happened and which ones are particularly relevant.

Of course, condition, history, color, mileage and originality still matter. No two cars are necessarily identical, even when they left the factory with the same specification. But that’s all the more reason to show the evidence rather than hiding it behind a single number.

And if the information is limited, say so. I would have far more respect for a tool that shows one relevant sale and explains why it matters than one that produces an apparently precise figure from a collection of questionable comparisons.

After spending so much time looking at this market, what strikes me most is that the actual questions people want answered aren’t particularly complicated.

What have cars like mine really sold for? How does mine compare? If I’m looking to buy another one, where can I find it? And if I’m thinking of selling, where should I sell it, should I use a reserve and would I be better off having someone experienced handle the auction?

These are all connected questions, yet we expect people to jump between different websites, guides, search engines and auction platforms to piece everything together themselves.

And often, after all that work, they’re still left wondering which information they can actually trust.

I don’t believe the answer is to give people even more charts or another dashboard containing hundreds of figures. It’s to do the difficult work behind the scenes and make the information that really matters incredibly easy to understand.

Show someone the cars that actually sold. Explain why they’re relevant, what makes one a closer comparison than another, and what the evidence does and doesn’t tell them.

That sounds simple, but believe me, it isn’t. There’s a huge amount of work involved in getting it right, and the further I’ve got into the subject, the more respect I have for just how difficult some of these problems are.

I also know there are some very knowledgeable people doing genuinely good work in this industry, and I don’t want to suggest that everyone has got it wrong or that mistakes should never happen.

They will. You can have the best information in the world and still come across a car that sells for a price nobody expected. Sometimes there’s simply not enough evidence, and sometimes two apparently similar cars produce completely different results for reasons that aren’t immediately obvious.

That’s the nature of the collector car market, and it’s part of what makes following it so interesting.

But there’s a world of difference between acknowledging that uncertainty and hiding it behind a number that looks convincing.

And that’s really what has been bothering me for so long.

It’s far too easy to produce an impressive looking answer, and far too difficult for the average person to know whether it’s actually correct.

When people are making decisions involving serious amounts of money, surely we should be demanding better.

I know this has been a longer and rather more opinionated Daily Vroom than usual, but I think it’s an important conversation to have, and one that’s been overdue for quite some time.

And I haven’t spent the past year looking at all of this just so I could have a rant in today’s newsletter. We’ve been working on something that I believe takes a very different approach, and tomorrow I’ll show you exactly what it is.

After everything I’ve said today, you can judge for yourself whether we’ve got it right.