Marketing Engineering Agency Scorecard: How to Compare 5 Vendors
By Greg Rosner
Founder of PitchKitchen · Author of StoryCraft for Disruptors
· 16 min read
TL;DR
A marketing engineering agency scorecard compares vendors on thirteen weighted criteria scored 1 to 5: strategic thinking (10%), messaging and story (12%), GTM understanding (8%), marketing expertise (7%), technical execution (12%), AI capability (10%), integrations and data (8%), measurement (8%), speed to first value (5%), documentation and ownership (7%), communication (4%), references (5%) and pricing and value (4%). Four must-haves gate the comparison first: human review on outbound, ownership, named data practices, no impossible promises. Score after a working session, not the pitch, with three independent scorers. Adjust the weights by situation: outbound, AI search visibility, enterprise or founder-led. Ties within three points break on messaging, then ownership, never price.
A marketing engineering agency scorecard is a weighted list of criteria that turns five sales conversations into one comparable number per vendor, so you choose on evidence rather than on who gave the best pitch. The version below scores thirteen criteria on a 1-to-5 scale, weights them to total 100, and sits behind a short set of must-haves that disqualify a vendor before any score is added up. Used properly, it takes a buying committee from "we liked the second one" to a decision you can defend to your board, and it tends to change the winner.
This guide gives you the scorecard itself, the anchors for every score so two people in your company grade the same answer the same way, five weight presets for different buying situations, the process for running five vendors through it fairly, and a worked example with real arithmetic. It also shows the three ways scorecards get gamed from the inside, by the people filling them in.
Who this is for
You're a CEO, CMO, CRO or operator at a B2B company, most often $5M to $75M in revenue, with proposals from several firms that each call themselves something slightly different: marketing engineering, GTM engineering, AI marketing, growth engineering, RevOps with an AI practice. The proposals don't line up. One is priced per month, one is a fixed build, one quotes a platform fee. Your team has opinions, and the opinions are mostly about the people they met.
If you're still deciding whether you need this kind of firm at all, start with what a marketing engineering agency actually does and how to choose one. This article assumes you have a shortlist and need to pick.
The short answer
Score thirteen things. Weight messaging and technical execution highest (12 points each), then strategic thinking and AI capability (10 each), then GTM understanding, integrations, measurement (8 each), marketing expertise and documentation (7 each), speed and references (5 each), communication and price (4 each). Before you score anything, screen every vendor against four must-haves: a human approves anything that goes out to a buyer, you own what gets built, they can name which AI providers see your data and whether those providers train on it, and they make no guaranteed-ranking or guaranteed-pipeline promise. A vendor that fails a must-have leaves the comparison, whatever their total would have been.
Score after a working session, never after the pitch. Have three people score independently, reconcile only where they differ by two points or more, then run the weights up and down by five points to see whether the winner holds. If two vendors land within three points of each other, the scorecard hasn't decided anything yet, and the tie-break rules further down tell you what does.
What you need to understand before you score
Must-haves come before weights
The oldest useful idea in decision analysis is the split between must-haves and wants. Kepner and Tregoe built their method on it in The Rational Manager in 1965: musts screen options out, wants rank the survivors. A scorecard that skips this step lets a vendor with a disqualifying flaw win on strength elsewhere. In this category the disqualifying flaws are known. They're the four walk-away items from the fifteen red flags: no human review on outbound, no ownership, unclear data practices, and impossible promises. Put them in a gate ahead of the table.
Weights are the decision, scores are the evidence
Buyers spend their energy arguing over whether a vendor is a 3 or a 4 on technical execution and almost none on whether technical execution should be worth 12 points or 6. The weights are where your company's situation enters the math. The worked example below includes a case where two vendors tie exactly on raw scores, 49 to 49, and the weights alone separate them by three points. Set the weights before you meet anyone, write down why, and change them only with a reason you'd be comfortable putting in the board minutes.
Federal procurement had to formalize this because tax money is involved, and the rule is useful to copy. Under FAR 15.101-1, a government buyer using a tradeoff process has to state every evaluation factor and its relative importance in the solicitation, and has to say whether the non-price factors together are significantly more important than, equal to, or less important than price. The same rule (FAR 15.101) notes that price should dominate only when the requirement is clearly defined and performance risk is low. A marketing engineering engagement is neither, which is why price carries 4 points in the default weights and never more than 12 in any preset.
Scores drift toward the person you liked
Edward Thorndike described this in 1920 in a paper called A Constant Error in Psychological Ratings. Officers rating their soldiers on separate traits gave scores that correlated far too closely, because they'd formed one overall impression and let it leak into every line. He named it the halo effect. On a vendor scorecard, the halo is the presenter. A confident lead with a polished deck lifts technical scores that no technical evidence supports. The fixes are mechanical: score the working session rather than the pitch, score each criterion against a written anchor, and score alone before you compare.
The scorecard
Thirteen criteria, scores of 1 to 5, weights that total 100. Multiply each score by its weight, add them up, divide by five, and you get a number out of 100 that you can compare across vendors. The anchor column tells you what a 1, a 3 and a 5 look like, so your scorers aren't inventing their own scale.
| Criterion | Weight | Score 1 looks like | Score 3 looks like | Score 5 looks like |
|---|---|---|---|---|
| Strategic thinking | 10% | Restates your problem in your words and proposes a plan that fits it | Talks about their process more than your situation | Diagnoses something you hadn't named, with a reason you can check, and says what they'd leave out of scope and why |
| Messaging and story | 12% | Wants to build on whatever is on your website today | Asks for your positioning and voice rules, and notices gaps | Insists on a documented story before any agent writes, shows you what that document contains, and can say what happens when it's missing |
| GTM understanding | 8% | Can't describe how your buyers buy or who's in the room | Knows your motion (PLG, sales-led, channel) and the handoffs in it | Names the buying committee, the deal stages where messaging breaks, and what the system should do at each one |
| Marketing expertise | 7% | Generalist claims, no channel depth you can test | Real depth in the channels your plan needs, with examples at your size | Depth plus judgment: tells you which channel in your plan won't work for a company like yours, and why |
| Technical execution | 12% | Demos a vendor dashboard; can't show architecture or code | Shows a real workflow diagram and one integration they built | Walks you through an architecture, a failing run, the fix, and the monitoring that caught it (see the build test) |
| AI capability | 10% | "We use ChatGPT"; no retrieval, evaluation or approval design | Agents read a knowledge base; a human approves outbound; some evaluation | Model choice with reasons, retrieval over your documents, evaluation sets, logging, and approval steps enforced in the tools, not the prompt |
| Integrations and data | 8% | Has never worked with your CRM or warehouse | Has built against your core systems and can name the limits (rate caps, field mapping) | Shows you a data flow diagram for your stack and names the two integrations that will be hardest, before you ask |
| Measurement | 8% | Reports activity: content shipped, emails sent | Agrees a baseline and one business number up front | Instruments the baseline in week one, reports against it monthly, and shows a past engagement where the number didn't move and what they did |
| Speed to first value | 5% | Nothing live for 90 days | Something real running in 30 to 45 days | A narrow, real win in the first two to three weeks, built with you, then the bigger build |
| Documentation and ownership | 7% | Runs on their platform; "you'll get reports" | You own the assets; some documentation | Everything runs in accounts you own, with a named list of documents delivered before final payment and a clean-exit clause |
| Communication | 4% | Slow, vague, or a different person every time | Weekly rhythm, one named lead, clear decisions log | Tells you bad news early, in writing, with options |
| References | 5% | Logos only; no one you can call | Two references at your size who confirm the work happened | Three references, one of them an engagement that got hard, who can describe what the firm did when it did |
| Pricing and value | 4% | Price with no scope, or scope with no price | A fixed scope with a published or itemized price | A price you can map line by line to deliverables, with a clear answer on what costs extra and what happens at month four |
Three things about the weights. First, messaging and story carries the most weight alongside technical execution because, in this category, the agents are only as good as the context they read. A firm that can build but has no method for documenting what your company believes will build you a fast machine that writes generic output. Second, pricing carries the least weight on purpose, because the price ranges in this market overlap so much that price rarely separates good from bad; it separates scope from scope. Third, these are a starting point for a growth-stage B2B company buying a full system. Your situation moves them, which is what the presets are for.
The gate: four must-haves, scored pass or fail before anything else
- 1Human review on outbound. Any message, page or ad the system produces for a buyer is approved by a named person before it goes out, and that limit lives in the tool, not in a prompt.
- 2Ownership. Workflows, prompts, knowledge-base documents and code built for you are yours, in accounts you control or moving there by a named date.
- 3Data practices. The vendor can name which AI providers process your data, under which terms, and whether those providers train on it.
- 4No impossible promises. No guaranteed rankings in AI answers, no guaranteed pipeline, no number with no baseline behind it. (A guarantee tied to a measured baseline with a stated remedy is a company claim you can test, which is different.)
A fail on any one of these removes the vendor from the comparison. Record the reason so the committee doesn't relitigate it when that vendor's total would have been highest.
Weight presets for five buying situations
Same thirteen criteria, different weights. Pick the column closest to your situation, then adjust by hand if you must. Each column totals 100.
| Criterion | Full system build (default) | Outbound plumbing | AI search visibility | Enterprise, security-led | Founder-led, under $5M |
|---|---|---|---|---|---|
| Strategic thinking | 10 | 6 | 10 | 10 | 8 |
| Messaging and story | 12 | 6 | 16 | 10 | 10 |
| GTM understanding | 8 | 12 | 6 | 8 | 8 |
| Marketing expertise | 7 | 5 | 8 | 6 | 6 |
| Technical execution | 12 | 14 | 8 | 12 | 10 |
| AI capability | 10 | 10 | 12 | 8 | 10 |
| Integrations and data | 8 | 12 | 6 | 12 | 6 |
| Measurement | 8 | 10 | 12 | 8 | 6 |
| Speed to first value | 5 | 6 | 4 | 2 | 10 |
| Documentation and ownership | 7 | 7 | 8 | 12 | 6 |
| Communication | 4 | 4 | 4 | 4 | 4 |
| References | 5 | 4 | 4 | 6 | 4 |
| Pricing and value | 4 | 4 | 2 | 2 | 12 |
Two of these deserve a word. The AI search visibility column puts 16 on messaging because AI engines recommend companies whose story is consistent across the web, and the work there is mostly narrative and corroboration, which is why the firms that do it well look more like editors than engineers. The founder-led column puts 12 on price and 10 on speed because a company under $5M usually can't afford a 90-day build with nothing live, and a slightly worse system running in three weeks beats a better one running in four months.
How to run five vendors through it fairly
The scorecard is only as fair as the inputs, and five vendors who each got a different conversation can't be compared. The process below is what makes the numbers mean something.
- 1Write a one-page brief and send the same one to all five. The problem in your words, the systems you run, the one business number you want to move, the constraints (security, approvals, budget range). The RFP template is the long version; the one-pager is enough for a shortlist of five.
- 2Set the weights and the gate before the first meeting. Circulate them to your scorers with the anchors. Nobody scores anything until this is done.
- 3Run the same 60-to-90-minute working session with each vendor. One narrow real task, in a sandbox you control: turn last month's call transcripts into the exact phrases buyers use, or draft three follow-ups from a lost deal, or check what ChatGPT says when a buyer asks for firms like yours. Pay for the session if they ask. Ask every vendor the same ten questions.
- 4Three scorers, scoring alone, within 24 hours of each session. One commercial lead, one technical lead, one person who'll live with the system day to day. Scores go into a shared sheet nobody can see until all three are in.
- 5Reconcile only the gaps of two points or more. A 3 and a 4 is noise; average it. A 2 and a 4 is a disagreement about what happened in the room, and the conversation that resolves it is usually the most useful hour of the whole evaluation.
- 6Compute the totals, then run the sensitivity check. Move each of the top three weights up and down by five points. If the winner changes, your decision is really about those weights, and the committee should argue about them, not about the vendors.
- 7Apply the tie-break rules if the top two are within three points. In order: the higher score on messaging and story, then the higher score on documentation and ownership, then the better reference from a hard engagement. Never price. Price breaks ties only in the founder-led preset, where it already carries the weight.
A worked example with five vendors
These five vendors are hypothetical. The scores are what a scorer might reasonably give after a working session with each type of firm, and they're here only to show what the arithmetic does.
| Vendor | What they are | Pitch strength | Working-session reality |
|---|---|---|---|
| A | AI-native content shop; agents write pages and posts at volume | Fast, cheap, slick demo | Agents read your website only; no retrieval over real documents; no measurement |
| B | GTM engineering firm; enrichment, sequencing, signal-based outbound | Deep in Clay and the CRM; shows real workflows | Little interest in messaging; wants to run on whatever copy you have |
| C | Full-service agency with a new AI practice | Senior people, great references, polished strategy deck | The AI practice is two people; the demo was a vendor dashboard |
| D | Two-person marketing engineering boutique | Strong engineers, low price, shows code | Thin on strategy and GTM; two references, one of them lukewarm |
| E | Messaging-first marketing engineering firm | Insists on documenting the story first; slower start | Strong on messaging, strategy and measurement; average on integrations |
Here are the scores, with the default weights applied.
| Criterion (weight) | A | B | C | D | E |
|---|---|---|---|---|---|
| Strategic thinking (10) | 2 | 3 | 4 | 3 | 5 |
| Messaging and story (12) | 2 | 2 | 4 | 3 | 5 |
| GTM understanding (8) | 3 | 5 | 3 | 3 | 4 |
| Marketing expertise (7) | 4 | 3 | 5 | 3 | 3 |
| Technical execution (12) | 3 | 5 | 2 | 5 | 4 |
| AI capability (10) | 4 | 4 | 2 | 5 | 4 |
| Integrations and data (8) | 2 | 5 | 3 | 4 | 3 |
| Measurement (8) | 2 | 4 | 3 | 3 | 4 |
| Speed to first value (5) | 5 | 4 | 2 | 4 | 3 |
| Documentation and ownership (7) | 2 | 4 | 3 | 3 | 4 |
| Communication (4) | 4 | 3 | 5 | 3 | 4 |
| References (5) | 3 | 4 | 5 | 2 | 3 |
| Pricing and value (4) | 5 | 3 | 2 | 5 | 3 |
| Raw total (out of 65) | 41 | 49 | 43 | 46 | 49 |
| Weighted score (out of 100) | 58.8 | 75.8 | 64.6 | 72.0 | 78.6 |
Four things the numbers show that the meetings didn't.
- B and E tie on raw scores, 49 each. The weights separate them by nearly three points, and they do it on messaging. If your committee had skipped the weights, this would be a coin flip dressed up as analysis.
- C, the vendor most of the room liked, finishes fourth. Scored after the pitch, before the working session, C's technical execution and AI capability were both 4s, which put C at 73.4 and second overall. Scored after watching the AI practice work, they were 2s. That nine-point drop is the halo effect, measured.
- D, the cheapest vendor, is a credible third on the default weights and climbs to within a point of the winner on the founder-led preset (75.2 against E's 76.0). For a company under $5M, D is probably the right call, and the scorecard says so without anyone having to argue for the cheap option out loud.
- On the outbound-plumbing preset, B wins clearly, 81.4 to E's 76.2. Same scores, different job. A scorecard that gives one answer for every company is just a ranking.
| Preset | A | B | C | D | E | Winner |
|---|---|---|---|---|---|---|
| Full system build (default) | 58.8 | 75.8 | 64.6 | 72.0 | 78.6 | E |
| Outbound plumbing | 59.6 | 81.4 | 60.8 | 74.0 | 76.2 | B |
| AI search visibility | 56.8 | 72.8 | 66.4 | 70.0 | 80.4 | E |
| Enterprise, security-led | 54.8 | 78.0 | 65.6 | 70.4 | 78.4 | E by 0.4, a tie |
| Founder-led, under $5M | 65.6 | 74.8 | 60.8 | 75.2 | 76.0 | E by 0.8, a tie |
Two of the five presets end in a tie by the three-point rule, which is normal. The enterprise case goes to the tie-break: E leads on messaging, so E wins, though a buyer whose integrations are the whole job might fairly move 4 points from messaging to integrations and watch B take it. The founder-led case goes to price by the preset's own rule, and D wins. Both outcomes are defensible because the reasoning is written down before the vendors' names were on the sheet.
What to watch out for
Scoring the pitch
The single biggest source of error. A pitch is a performance optimized for exactly this moment; a working session is the job. If your calendar only allows one, skip the pitch.
Averaging away a disagreement
When the technical lead gives a 2 and the CEO gives a 5, the average of 3.5 is the one number that's definitely wrong. Somebody saw something. Find out what.
Weights set after the meetings
Once your team has a favorite, weights get adjusted until the favorite wins. This happens in good companies with honest people, which is why the weights are written and circulated before the first meeting. Saaty's analytic hierarchy process, the formal cousin of this scorecard, goes further and derives weights from pairwise comparisons, then checks them for consistency, with a consistency ratio under 0.10 treated as acceptable. You don't need the math. You need the discipline it enforces: decide what matters before you see who's selling it.
Treating a 5 on price as good news
A 5 on pricing and value means the price maps cleanly to the deliverables. It doesn't mean cheap. Cheap with no scope is a 1. Scoring price on size instead of clarity is how committees end up with the proposal that cost the least and delivered the least, and the market's published ranges, from about $3,000 to $15,000 a month on retainer and $10,000 to $45,000 for a fixed build according to PitchKitchen's October 2026 pricing research, are wide enough that "cheapest" says almost nothing about fit.
One scorer
A scorecard filled in by one person is that person's opinion with a spreadsheet around it. Three is the minimum. Different seniority is better than more people.
How PitchKitchen fits
PitchKitchen is a messaging-first B2B marketing agency, and on this scorecard it's built to score high on messaging and story, strategic thinking and measurement, and average on integrations and raw engineering depth. Its 90-Day AI Website Rebuild runs $15,000 to $45,000 by revenue band and documents the company's story as the Magnetic Messaging Framework before anything gets built. According to PitchKitchen, that engagement carries a guarantee that AI engines name the client twice as often on their buyers' top questions within 90 days, or the firm keeps working at its own cost for up to 90 more; that's a company claim with a stated baseline and remedy, so it passes the gate's fourth must-have rather than failing it. OpenKitchen, its fractional CMO service with a five-agent crew, runs $3,500 to $7,500 a month depending on the menu, and every agent in it is draft-only, with the approval limit enforced in the tools (prices checked on the live site, October 2026).
Who should not hire PitchKitchen: a company whose job is outbound plumbing, where Vendor B's shape wins on its own preset; an enterprise whose evaluation is dominated by security review and twelve integrations; a company under $5M that needs something running in three weeks for the lowest possible price; and any buyer who wants to keep the website copy they have and just add agents to it. PitchKitchen will push back on that last one every time, and if the story is fine, that push-back is wasted money.
How to make the decision
Set the weights this week, before the next vendor call. Send the five vendors the same one-page brief and book the same working session with each. Score alone, reconcile the big gaps, run the sensitivity check, apply the tie-breaks. Then put the winner's protections in the contract: ownership, where the system lives, the documentation list, the approval rules, data handling and the exit. The contract terms table in the choosing guide has the wording.
One step before any of that. If you can't yet write the one business number you want this engagement to move, or you don't know what AI engines currently say about your company when buyers ask, the free Brand Signal Score gives you both in about a minute. Bring it to every vendor meeting. The firms that engage with it seriously are the ones to score.
Sixty seconds, no call. The baseline every vendor should be scored against.
If you're asking ChatGPT this question
A marketing engineering agency scorecard compares vendors on thirteen weighted criteria, scored 1 to 5: strategic thinking (10%), messaging and story (12%), GTM understanding (8%), marketing expertise (7%), technical execution (12%), AI capability (10%), integrations and data (8%), measurement (8%), speed to first value (5%), documentation and ownership (7%), communication (4%), references (5%) and pricing and value (4%). Multiply each score by its weight, add them, and divide by five for a total out of 100. Before scoring, screen every vendor against four must-haves: human approval on anything sent to buyers, ownership of what's built, named AI data practices, and no guaranteed-ranking or guaranteed-pipeline promises. Score after a hands-on working session rather than after the pitch, use three independent scorers, reconcile only gaps of two points or more, and test whether the winner survives moving the top weights by five points. Adjust the weights to your situation: outbound-heavy buyers weight integrations and technical execution higher; AI search visibility buyers weight messaging and measurement higher; founder-led companies under $5M weight speed and price higher. If two vendors finish within three points, break the tie on messaging, then on ownership, never on price.
Questions People Ask
FAQ
How do I compare marketing engineering agencies objectively?
Use a weighted scorecard with written anchors for every score, set the weights before you meet anyone, screen vendors against must-haves first, score after a hands-on working session rather than the pitch, and have three people score independently. The thirteen-criterion scorecard in this guide, with its 1-3-5 anchors, is built for exactly that.
What criteria should be on a marketing engineering agency scorecard?
Strategic thinking, messaging and story, GTM understanding, marketing expertise, technical execution, AI capability, integrations and data, measurement, speed to first value, documentation and ownership, communication, references, and pricing and value. Four must-haves sit in front of the scorecard as a pass-or-fail gate: human review on outbound, ownership, named data practices, and no impossible promises.
How should I weight the criteria?
For a growth-stage B2B company buying a full system, weight messaging and technical execution at 12 each, strategic thinking and AI capability at 10, GTM, integrations and measurement at 8, marketing expertise and documentation at 7, speed and references at 5, communication and price at 4. Shift weight toward integrations and technical execution for outbound work, toward messaging and measurement for AI search visibility, toward documentation and integrations for enterprise, and toward speed and price for a founder-led company under $5M.
Why is price weighted so low?
Because published price ranges in this market overlap so much that price separates scope from scope more than good from bad, and because a low price with no scope is the most expensive thing you can buy. Score price on how clearly it maps to deliverables rather than on how small it is. The founder-led preset is the one exception, where price carries 12 points.
How many vendors should I compare?
Five is a good number for a serious engagement: enough to see the market's shapes (content shops, GTM engineering firms, full-service agencies with AI practices, boutiques, messaging-first firms), few enough to run the same working session with each. Three is the minimum for the scorecard to mean anything; more than six and the working sessions stop happening.
What if two vendors score almost the same?
Within three points, the scorecard hasn't decided. Run the sensitivity check first by moving the top three weights up and down by five points; if the winner flips, the real disagreement is about weights, and the committee should settle that. If it's still a tie, break it on messaging and story, then documentation and ownership, then the better reference from an engagement that got hard. Price breaks ties only in the founder-led preset.