If They Know How AI Sees Your Law Firm, Ask Them to Show Their Work

I watched another one of these videos this week.

Legal marketing vendor. New Proprietary AI tool.

How exciting!

Theirs alone, of course.

It shows you how ChatGPT, Google, and the rest “see” your law firm’s website.

Then it fixes whatever’s keeping you out of the answers.

The pitch opened with a fair point: most lawyers have no idea whether AI recommends them when a potential client asks.

True.

But I have no idea whether the tool does what they say it does.

And neither do you.

This isn’t about one vendor. We hear some version of this pitch every week now. Different logos. Same pitch.

So I’m not going to pick on anybody.

I’m just going to ask them to show their work.

What are they actually selling?

ChatGPT, Gemini, Perplexity, Claude, and Google’s AI Overviews don’t tell anyone how they decide which lawyers to recommend.

There’s no “How AI Sees Smith & Jones, PLLC” report.

So when someone says they can show you how AI “views” your site, in my experience they’re doing one of two things.

First, they run your pages through an AI model and show you what comes out.

This is usually what “semantic mapping” means. Each page gets turned into a long string of numbers that represents what it’s about. Pages about similar things get similar numbers. Then the tool groups them into topics, compares them to your competitors, and draws you a pretty map with the gaps circled.

Whose AI model?

Probably not theirs.

Odds are good the “proprietary AI” is OpenAI, Google, Anthropic, or an open source model, maybe fine-tuned a bit, with their prompts and a dashboard on top.

That’s fine. Plenty of good tools work that way.

But “proprietary” is doing a lot of work in that sentence. 🙄

Usually it means the prompts and the charts.

And whose model it is doesn’t matter much anyway. Even if they built one from scratch, it would tell you how their model reads your site. ChatGPT is still going to decide who to recommend on its own.

And sending your pages to OpenAI’s API isn’t the same thing as seeing how ChatGPT decides who to recommend. The ChatGPT your potential clients use is searching the web, using their location, remembering past chats, and running on versions and settings nobody outside OpenAI can see.

And a lot of who gets recommended probably has nothing to do with your website. Reviews. Directories. “Best lawyers in Chicago” lists. News. Reddit. A map of your site can’t see any of that.

Then there’s the score.

These tools love a number. “Your semantic relevance went from 0.72 to 0.81!” 📈

Relevant to what? According to which model?

And the easiest way to move that number is to cram more of the right words onto more pages. Maybe a page for every suburb…

We’ve seen how that ends. More on that in a minute.

Can the map still be useful? Sure. If it shows that your site barely mentions half your practice areas, that’s a solid content audit.

It’s just not a peek inside ChatGPT.

Second, they ask AI tools a bunch of questions and count how often your firm shows up.

Closer to useful. Still very easy to do badly.

Nobody outside OpenAI knows what people actually ask ChatGPT. So the vendor picks the questions. Usually short, Google-style ones. Real people type paragraphs about their car accident.

Ask ChatGPT the same question 10 times. You might get 10 different lists of lawyers. Change the wording, the city, the account, the day…

A screenshot of one answer with your firm in it tells you almost nothing.

The most useful part usually isn’t the score. It’s seeing which websites the AI keeps citing in your market, and catching it when it gets something wrong about your firm.

Neither of these is a scam.

But both get sold as something they’re not.

We’ve seen this movie before

“Proprietary” has a long history in SEO.

It’s mostly not a good one. 🫣

If you’ve been practicing for a while, you probably got the emails.

Proprietary link building system.

Proprietary ranking technology.

Proprietary content network.

Guaranteed page one rankings.

Under the hood, a lot of that “proprietary” stuff was the same handful of tricks. Buying links. Networks of junk blogs that existed only to link to clients (private blog networks, in SEO-speak). Article spinners that rewrote the same post a hundred ways. Thousands of directory submissions. A keyword-stuffed page for every suburb within 50 miles.

It “worked.”

Until it didn’t.

In early 2012, Google started knocking entire paid link networks out of its index. In April 2012, it rolled out Penguin, an update aimed squarely at manipulative links. In 2014, it sent out manual penalties to sites using private blog networks.

Plenty of businesses, law firms included, watched their rankings fall off a cliff. Then they paid someone else to clean up the mess.

The “proprietary system” was the liability.

Google has been warning about this for years. Its own guide to hiring an SEO still says:

“Beware of SEOs that claim to guarantee rankings, allege a ‘special relationship’ with Google, or advertise a ‘priority submit’ to Google. There is no priority submit for Google.”

Swap “Google” for “ChatGPT.”

Sound familiar?

Don’t get me wrong. Not every proprietary SEO tool was snake oil. Some agencies built genuinely useful software.

But in my experience, the louder someone says “proprietary,” the less they can tell you about how it works.

Even the people who built it don’t know

I don’t know exactly how ChatGPT decides which personal injury lawyers to recommend in Phoenix.

Get this: neither do the people who built it.

Sundar Pichai, Google’s CEO, on 60 Minutes back in 2023:

“There is an aspect of this which we call… all of us in the field call it as a ‘black box.’ You know, you don’t fully understand. And you can’t quite tell why it said this, or why it got wrong.”

Sam Altman, OpenAI’s CEO, at the AI for Good Global Summit in 2024:

“We certainly have not solved interpretability.”

Dario Amodei, Anthropic’s CEO, in his 2025 essay, The Urgency of Interpretability:

“People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work.”

He also says that when one of these models does something, “we have no idea, at a specific or precise level, why it makes the choices it does — why it chooses certain words over others.”

Why it chooses certain words over others.

Like your firm’s name instead of the firm down the street.

The CEOs of Google, OpenAI, and Anthropic can’t tell you exactly why their models say what they say.

But a vendor renting those same models by the API call can?

Seems problematic.

I’ll grant this much: they’re talking about what’s happening inside the model. Researchers call that mechanistic interpretability, and it’s nowhere near solved.

You can still watch what the models do from the outside. Ask lots of real questions, lots of times, and write down the answers. Some of us have run a lot of these tests and have good hunches.

But that’s what they are. Hunches.

It’s measuring behavior, not reading minds. And it’s only as good as the method.

Which is why you should ask to see the method.

Ask them to show their work

When you can’t check the results yourself, the burden of proof is on the seller.

Not on you.

So before you buy the tool, sign the “AI visibility” retainer, or celebrate your new “AI score,” ask them to answer these in writing:

What exactly are you measuring? Citations? Mentions? In which AI tools? Which versions?

Did you build this AI, or is it OpenAI, Google, or Anthropic under the hood? If it’s a wrapper, what’s actually proprietary?

Is this how ChatGPT sees my site, or how your tool sees it?

What questions are you asking the AI, and who picked them? Can I see the list?

How many times do you ask each one? From where?

If you ran it twice and changed nothing on my site, how different would the results be?

What did you change, and what happened after? Same questions, same method, over weeks, not an afternoon.

Did firms that didn’t use your tool improve too?

Did any of this produce more calls, consultations, or signed cases? How do you know? (good luck)

Can I see the raw data?

If they answer clearly, great. Some will. Hire them.

If the answer is “it’s proprietary,” that’s an answer too.

Proprietary describes how you do it. It doesn’t excuse not showing that it works.

Don’t day trade your marketing budget

Let’s put this together.

Nobody can fully explain why these models recommend who they recommend.

The answers change from one run to the next. The models change constantly.

And how people find lawyers keeps changing. Some potential clients ask ChatGPT. Some Google it. Some ask their brother-in-law. Plenty do all three before they call anyone.

Betting your firm’s growth mostly on AI search starts to look a lot like day trading crypto.

Volatile. Hard to tell what’s moving it. Lots of people selling the system that beats it.

Some people get rich. Lots of people get wrecked. Few can tell you whether it was skill or luck.

That screenshot of your firm in a ChatGPT answer? It’s a screenshot of one good trading day. 📈

No good financial advisor tells you to put your retirement into one coin.

They tell you to diversify.

Same with your marketing.

I’m not saying ignore AI search. Your potential clients are using it. Pay attention to it.

Just don’t make it the whole plan.

Invest in your reputation and your referral sources. In our experience, they still send a lot of firms their best cases.

Invest in reviews.

Invest in organic and local search. They aren’t dead. Not even close.

Invest in paid media you can measure. Google Ads and LSAs have their problems, but you can turn them up and down and see what happened.

Invest in your brand. TV, radio, billboards, podcasts, sponsorships, your community. Whatever makes sense in your market.

Invest in getting talked about. Press, speaking, bar associations, other lawyers.

Invest in an email list you own.

Invest in intake. None of this matters if the call goes to voicemail.

That’s what omni-channel means to me. Not “be everywhere.” Be where your potential clients are, so no one channel can sink you.

But here’s a thing…

A lot of what seems to get firms mentioned in AI answers comes from those other channels. Reviews. Mentions on other sites. People searching your name. A website that clearly says what you do and where.

That’s a hunch, not a guarantee.

But if I’m right, diversifying isn’t just a hedge against AI search.

It might be the best AI search strategy you’ve got.

I don’t know the right mix for your firm.

But you can figure it out. Track calls and forms by channel all the way to signed cases. Then move money based on clients, not someone’s “AI visibility score.”

Anyone confident in their method will be happy to show you.

Anyone who isn’t will tell you it’s proprietary.

🍿