How to Vet a Marketing Agency's Case Studies Before You Believe Them

Why Most Agency Case Studies Are Built to Survive a Skim, Not a Question

A case study on an agency's website has one job: get you to stop scrolling and believe the pitch. It doesn't have to survive cross-examination, because most contractors never ask it to. That's not an accusation against every agency in this business. It's just how the page gets written. Someone picks the best month from the best client and builds a graphic around it.

The tell is almost always missing context. A screenshot that says "213% increase in leads" with no dates, no starting number, and no mention of what else changed that month (a new service page, a paid ad campaign, a slow season ending) isn't lying, exactly. It's just incomplete in a way that flatters the agency. 213% of four leads is eight leads. 213% of forty leads is a business-changing number. The graphic looks the same either way.

Screenshots are the biggest offender. A cropped Google Analytics chart with the client name blacked out, no axis labels, no comparison period, proves almost nothing on its own. It could be real. It could be a seasonal bump. It could be a different client than the one being discussed in the surrounding paragraph. You have no way to check, and that's the point of a screenshot: it looks like data while functioning like an illustration.

There's also a timing problem specific to contracting. A roofer's lead volume triples after a hailstorm whether or not the marketing changed. An HVAC company's calls spike in the first heat wave of the year. A landscaping outfit gets a spring bump every single year, agency or no agency. A case study that lines up a marketing start date with a seasonal or storm-driven spike, without naming the season, is borrowing credit from the weather. This isn't hypothetical: it's one of the most common ways a real, honest agency accidentally overstates its own impact, because the calendar did some of the work and nobody subtracted it back out.

None of this means every agency running case studies is running a scam. Plenty of the underlying results are real. But real results get diluted into a marketing graphic the same way anecdotes get told at a bar: rounded up, stripped of the boring parts, optimized for the reaction. Your job before you sign a contract is to un-round it. Ask for the boring parts back: the client, the date range, the starting point, and what else was happening in the business at the same time. An agency with real results will have that on file. An agency with a good story usually won't.

The Five Questions That Separate a Real Result From a Good Story

You don't need a marketing background to vet a case study. You need five specific questions, asked in this order, and you need to notice how fast and how specifically the agency answers each one.

• What was the baseline? Rankings, calls, and leads before work started. "We got them to page one" means nothing if you don't know what page they started on.
• Over what timeframe? A result that took 18 months is a different product than one that took four. Both can be true and honest. Only one applies to your first-year expectations.
• What else changed at the same time? New trucks on the road, a referral partnership, a competitor going out of business, a seasonal swing (storm season for a roofer, a cold snap for HVAC). Marketing rarely moves alone.
• Can I see it live, right now? Not a screenshot. An actual ranking check, an actual site, an actual Google Business Profile you can pull up on your own phone.
• Can I talk to them? Not required for every case, and some agencies genuinely operate under NDA with certain clients. But if the answer is no for every single case study on the page, that's the pattern to notice, not any one no by itself.

Watch how the agency answers, not just what they say. "Let me pull that up for you" is a different answer than a pivot to a different case study, or a vague "it varies by client." It does vary by client. That's exactly why the specific numbers for the specific case in front of you should be available on request.

Pay attention to the order of operations too. An agency that answers baseline and timeframe before you even finish asking has clearly explained this case before, to other prospects, because the numbers are real and memorized. An agency that needs to "check with the team and get back to you" on a headline case study they chose to publish is a soft flag. It's their own best example. They should know it cold.

A reasonable agency expects these questions. If asking them changes the tone of the conversation, from confident to defensive, that's information too, and it's often more reliable than anything printed on the case study page itself.

The Difference Between a Verifiable Result and a Reasonable Range

Some things an agency tells you can be checked today. Some things are honest estimates based on how the work generally performs. Both are legitimate, but they need to be labeled correctly, and a lot of case study copy blurs the line on purpose.

Verifiable: current rankings for a specific keyword, a live Google Business Profile with review count and star rating, a site's load time, whether a case study client's site actually exists and looks like what's described. These are facts you can check yourself in five minutes with a phone and a browser.

Reasonable range, not a guarantee: how long competitive SEO terms typically take to move (in this industry, generally 4-9 months depending on the term and the market), how many pages a well-built local silo tends to run (94-plus cluster pages is typical for a build like ours, not a promise for every trade in every city), how a map pack ranking behaves once you're sitting in the top 3. These are patterns, built from doing the work repeatedly, not a specific promise about your specific business.

An agency that's straight with you will draw this line without being asked. "Here's what we can show you today, and here's the range you should realistically expect based on what we've seen" is a sentence that should make you trust them more, not less.

Red Flags: What a Fabricated or Padded Case Study Looks Like

You won't always be able to prove a case study is fake. But you can learn to recognize the shape of one that's padded, borrowed, or invented, because the tells repeat across the industry.

• No client name, no live link, ever. Some NDAs are real. All 12 case studies on a page being anonymous is not an NDA pattern, it's a pattern.
• Stock photography standing in for the client. A "roofing client" case study illustrated with a generic stock photo of a roofer who obviously isn't the business owner named in the text.
• Numbers that don't do math. "300% more calls" next to a call-tracking screenshot showing totals that don't support a 3x jump. Most people don't check the math. Check the math.
• The same result copy, reused for different trades. If the plumber case study and the electrician case study read like the same paragraph with the trade word swapped, the specifics were never really about either business.
• A results claim with zero timeframe anywhere on the page. Not in the headline, not in the fine print, not available on request. Time is the one variable that's cheapest to hide and most important to know.
• Reviews or testimonials that read like ad copy. "Their innovative, results-driven approach transformed our business" is not how contractors talk. Real client language sounds like a voicemail, not a press release.
• An agency that gets cagey when you ask to speak to the client directly. Not every client wants a reference call. But a flat refusal on every single case, combined with the flags above, is worth walking away from.

One or two of these on their own aren't damning. An agency can have a genuinely great case they're just not allowed to name. But when three or four of these stack up on the same page, you're not looking at proof. You're looking at a page designed to look like proof.

What to Actually Ask For Before You Sign

Vetting a case study isn't about catching an agency in a lie. Most of the time it's about getting specific enough that you know what you're actually buying. Bring this list to the sales call.

1. A live example in your general market or trade, if they have one. A site you can pull up, a ranking you can search, a Google Business Profile you can look at right now.
2. The starting point. Where the client was before the engagement, in their own words if possible, not just the agency's summary of it.
3. The timeline, broken into phases. What happened in month one versus month six versus month twelve. Real work has a shape over time. It doesn't jump from zero to done.
4. What the client was doing on their own at the same time. New trucks, new hires, a referral push, review requests, seasonal demand. Marketing shares credit with the rest of the business more often than case studies admit.
5. Whether the number is a peak or an average. "Best month" and "typical month" are both honest words to use, and both get used to describe the same chart depending on which one sounds better.
6. What didn't work. Ask directly: "What's a case where this didn't go the way you expected, and what did you do about it?" An agency with real field experience has an answer. An agency that's only ever seen success has a marketing page.

None of this is adversarial. A shop that's been doing the work long enough has these answers on hand because they've had to explain a slow month to a real client before. That's not a weakness to hide. It's the difference between an agency that's actually run campaigns and one that's only ever built the page describing them.

How This Applies Specifically to SEO, Local Search, and AI-Search Claims

Case study vetting gets harder as the service gets more technical, because the buyer has fewer intuitions to check the claim against. A before-and-after website redesign, you can eyeball. A ranking claim, a map pack claim, or an AI-search visibility claim requires knowing what you're looking at.

For SEO ranking claims, ask which specific keyword, on which specific date, in which specific city. "We got them ranking" without a keyword attached is not a claim, it's a mood. And once you have the keyword, search it yourself. Rankings are public. There's no reason an agency should be unable or unwilling to show you a live search result for a claim they're making about their own past work. Also ask how competitive the term actually is in that market. "Ranked #1 for [trade] in [small town]" and "ranked #1 for [trade] in a metro with forty competitors bidding on ads for the same term" are very different achievements wearing the same headline.

For local map pack claims, understand what "top 3" actually means before you evaluate the case study: it means showing up in the three-listing map block that appears above the regular organic results for a "near me" or local-intent search. That's a specific, checkable position, not a vague sense of visibility. Ask the agency to search the term with you, from a phone, in the client's service area, not from an office computer where personalized search history can skew what shows up.

For AI-search visibility (whether a business gets named when someone asks ChatGPT or a similar tool for a recommendation), this is newer ground, and it's exactly the kind of claim that's easiest to overstate because most contractors haven't developed an instinct for checking it yet. Ask the agency to run the actual prompt a customer might type, in front of you, and show you the answer. If they can't or won't do that live, the claim isn't ready to be a case study yet, it's a hope. Be skeptical of AI-search claims with no methodology attached at all: which tool, which prompt, run how recently. These answers shift over time and between tools, so a single old screenshot proves less here than it does for a normal ranking claim.

Across all three, the same rule from the sections above applies: a claim you can check yourself in the room is worth more than a claim you're asked to take on faith, no matter how good the agency's story is. If an agency built its pitch around AI-search visibility as the differentiator, that's exactly the claim to stress-test hardest, since it's the one you have the least built-in intuition for catching a soft version of.