# Sam Bloomberg-Rissman — Full Text Bundle This file concatenates the prose content of sambr.com so an agent can retrieve everything in one request. --- title: Sam Bloomberg-Rissman — sambr.com canonical: https://sambr.com/ author: Sam Bloomberg-Rissman role: Sr. Director, Product Management — Event Technology employer: Salesforce location: Berkeley, California, USA email: website@sambr.com instagram: https://www.instagram.com/sambr/ linkedin: https://www.linkedin.com/in/sambloombergrissman/ language: en last-updated: 2026-08-18 license: All prose © Sam Bloomberg-Rissman. Quotation with attribution is welcome; please link back to the source URL listed on each section. Photographs are not licensed for reuse without permission. sources: - https://sambr.com/ # home + gallery - https://sambr.com/#about # about / bio - https://sambr.com/#product # product history - https://sambr.com/notes # writing index - https://sambr.com/writing/too-small-to-tell # essay - https://sambr.com/writing/your-agent-works-prove-it # essay - https://sambr.com/writing/show-dont-tell # essay - https://sambr.com/writing/nobody-gave-it-the-address # essay - https://sambr.com/writing/claude-built-my-mcp-server # essay - https://sambr.com/writing/dont-build-the-agent # essay - https://sambr.com/llms.txt # agent-facing map --- ================================================================================ ABOUT Source: https://sambr.com/#about ================================================================================ Pullquote: "Both practices are the same: decide what to keep in the frame, and trust the rest." I'm Sam. I live in Berkeley with my kid and a music setup that was definitely worth every penny (ask me again in a year). I spent most of my twenties traveling the world making photographs for a living — 20-something countries, a few Lonely Planet covers, and a frankly unreasonable amount of time in airports. Now I shoot because I can't stop, which turns out to be a better reason anyway. I'm drawn to the blue hour, books, and conversations that go longer than planned. I have strong opinions about music, photography, and the right way to spend a day off — most of which involve walking somewhere without a particular destination, which the dog finds more charming than anyone else does. I believe perfection is a useful compass but a terrible destination. The best photographs and the best products have this in common. Pull up a chair. Stats: 15+ years in product · Camera of choice: Leica M11 · Current role: Salesforce. ================================================================================ PRODUCT — OVER FIFTEEN YEARS OF SHIPPING Source: https://sambr.com/#product ================================================================================ Years of work in product management — mostly at Salesforce, Royal Caribbean and Disney, all digital. Everything below shipped to real customers. 2025— Sr. Director, PM — Event Technology @ Salesforce Leading product strategy for AI-native event experiences, deploying the Agentforce platform across the full attendee journey for Dreamforce, World Tour, and beyond. Building a global PM team to unify event tech strategy. 2023—2025 Sr. Director, PM — Marketing Localization @ Salesforce Scaled localization across Salesforce using generative AI and machine translation. Stood up a new localization ops function with tooling and process that sped delivery and widened global reach. 2020—2023 Director of Product Management @ Salesforce Led a team of six PMs building the tools behind salesforce.com — content management, site search, chatbots, and the broader marketing-web ecosystem. 2018—2020 Director, Digital Experience & PM @ Royal Caribbean Ran a large-scale e-commerce overhaul with a dozen scrum teams. Managed 20+ PMs and 200+ developers shipping the call-center tool, guest-facing web app, and inventory/pricing platform. Rebuilt the terminal experience with facial recognition — got guests on vacation faster. 2017—2018 Director, Ecommerce @ Royal Caribbean Led product, design, and engineering for a tours-and-activities startup inside the group. Ran the SAP Hybris platform, the consumer site, and the back-office tooling; built APIs that made the product sellable on third-party platforms. 2016 Digital Guest Experience Director @ Disney Parks Led the overhaul of the FastPass+ system at Walt Disney World — guest satisfaction rose 10%, complaints dropped 50%. Managed 4–6 PMs shipping new digital experiences across the global portfolio. 2013—2016 Manager, Digital Guest Experience & PM @ Disney Parks Moved Disney PhotoPass from desktop-only to a mobile-app business — engagement up 64% YoY. Led design of a self-guided vacation setup tool, and owned homepage + global nav performance for each destination site. 2013 Optimization Lead @ Disney Parks Built the eCommerce digital optimization team from 2 to 6. In its first year, delivered $50M+ in incremental revenue. Institutionalized A/B testing as a practice across the org. 2011—2013 Information Architect @ Disney Parks On the UX/IA team. Played a key role in the design overhaul of disneyworld.com — still the basis for Disney Parks digital properties today. Helped design the planning tools for the MyMagic+ program. 2001—2011 Photographer & Owner @ SBR Photography A lifestyle and travel stock photography company with work from over 20 countries, represented by Getty Images. Photos appeared in major publications and on Lonely Planet Guidebook covers. ================================================================================ ESSAY · TOO SMALL TO TELL. SAY SO. Source: https://sambr.com/writing/too-small-to-tell ================================================================================ slug: too-small-to-tell source: https://sambr.com/writing/too-small-to-tell published: 2026-08 author: Sam Bloomberg-Rissman tags: product, experimentation, ab-testing, measurement, ai-agents estimated-read: 7 minutes (~1600 words) Deck: I built an agentic system that runs marketing experiments end to end. Its most valuable output was a refusal to answer. One version of my landing page got a 7.1% signup rate. The other two got zero. In most companies that variant ships on Monday. It shouldn't. The 7.1% is one person. The zeros are nine and eleven people who didn't sign up. Put those numbers on a slide with a bar chart and you have a winner; put them through a posterior and you have three versions of a page about which you have learned essentially nothing. I know which one I'd have believed a year ago. I spent a few weeks building a system to make that mistake structurally hard. It generates landing pages, splits traffic behind a single URL, waits for conversions to mature, and reports a verdict against a contract written before any of it ran. I wrote almost none of the code — Claude Code did, while I decided what was worth building and what was worth deleting. Then I pointed it at myself, ran one real experiment on real strangers, and it told me nothing. That was the good outcome. Here's why. § 01 · Every easy number is a proxy The hard part was never generating pages. Generation is the part that demos well. The hard part was stopping the system from measuring the wrong thing, because in marketing almost everything easy to measure is a proxy for the thing you actually want. Clicks. Click-through rate. Sessions. Form starts. Scroll depth. Every one of them correlates with revenue, and not one of them is revenue. The danger isn't that they're useless — it's that they're available. They arrive instantly, they move satisfyingly, and the thing you care about arrives days later in a different system, if it arrives at all. So in this system the exclusion isn't advice. It's a list in code that the reward function refuses to accept as a primary metric. Not a guideline in a document nobody reads at 11pm with a dashboard open, and not a code review someone can wave through. A test fails. I learned this properly by nearly getting it wrong. I had planned to point the factory at Amazon affiliate products — build pages, send traffic, earn commission. The purchase happens on Amazon, where I can instrument nothing. But the outbound click is right there: instant, precise, mine. It's the perfect metric except for the part where optimising it optimises for people who leave and don't buy. A page can win the click by overpromising and earn nothing at all. That idea died on arithmetic before I designed a single page. Two cents of revenue per visitor against clicks costing thirty. Detecting a 20% improvement would have needed fifty thousand sessions of an experiment that loses money on every one. Run the unit economics before the design work. Software people know to profile before optimising; it's the same discipline pointed somewhere less familiar. § 02 · Decide what counts as evidence before you look Before I posted anything, I wrote down what the experiment would need in order to say something, and committed it. Three versions of the page, identical above the signup box, differing only in how the box asked. Primary metric: a stored email address. A winner would require the probability of beating control to exceed 0.95 and that version to have at least two hundred matured sessions. Stop at six hundred sessions or fourteen days, whichever came first. One look at the data, at the stopping point. The uncomfortable part was in there too. At a 10% baseline, separating these asks by 50% needs roughly 685 sessions per version. One LinkedIn post from a personal account plausibly produces thirty to a hundred and fifty in total. So the pre-registration said, in writing, before launch: this run is expected to be inconclusive, and inconclusive is the expected outcome rather than a failure. > Writing that down is the whole trick. It costs nothing in advance and it is almost impossible afterwards. Once you've seen 7.1% against 0.0%, no amount of integrity reliably reconstructs what you would have accepted as evidence beforehand. You don't decide to fool yourself. You just look at the data first and then decide what the threshold should be, and those two things happen in an order you never notice. § 03 · What the run actually said Fourteen days, then the time cap. Thirty-four sessions, one signup. Eleven sessions on control, no signups. Fourteen on the second version, one signup. Nine on the third, none. Verdict: inconclusive — no version separated from control at the pre-registered threshold, against a planned six hundred sessions. The most interesting number in the whole run is one I'd never have looked for. The version holding the only signup scored 0.448 — worse than a coin flip against a control with zero. That isn't a bug; I checked, twice, because it looked like one. The signup was five days old and the maturity window is seven. A conversion that hasn't had time to happen can't be evidence, and neither can one that has only just happened, so it isn't counted yet. With zero matured conversions everywhere, those probabilities reflect nothing but sample size — the third version scored highest precisely because it had the fewest sessions and therefore the widest uncertainty. > That is what no data actually looks like from the inside. Not an empty screen. A set of confident-looking decimals that mean nothing, attached to a bar chart that would have read as a clear winner. And the honest coda: at the 2.9% signup rate I actually observed, separating those asks at 50% would take about 2,600 sessions per version. Around 7,800 in total. I had thirty-four. § 04 · The bugs were never in the interesting part None of the things that broke were the bandit, the assignment maths, or the agents. They were all in the plumbing, and every one was invisible until something real ran through it. The first email send would have failed for every recipient, and been unrecoverable. The provider sits behind Cloudflare, which rejected the default Python user agent with a 403 before the request reached the API at all. Worse: every failure was being written to the database as sent, so the update could never have gone out again without hand-written SQL. Two bugs that compose into total, silent, permanent loss — and both were invisible until I stopped trusting the configuration and actually sent something to myself. Fixing one class of bug introduced another. I moved to a URL-based deploy specifically to eliminate hand-copied errors. It worked, and it silently discarded every secret in the request, so the database came up with no password and sat there dead. The fix removed one failure mode and quietly installed a different one in the same commit. And the one I nearly shipped. Filtering link-preview bots out of an experiment is obvious — LinkedIn fetches your link to build its card, and so does every platform it gets reshared to, and none of them can convert. Less obvious: LinkedIn's in-app browser sends "LinkedInApp" and its preview fetcher sends "LinkedInBot". Match the platform instead of the bot and you delete most of your real readers from the experiment while believing you've just cleaned your data. I wrote that regex, wrote a test that stepped around the problem rather than asserting the right thing, and caught it only on a second look. There's a pattern in all three. > Each one produced a plausible result rather than an obvious error. Nothing crashed. That's the failure mode worth building against, because it's the one your tests won't tell you about unless you go looking for it specifically. § 05 · One disclosure Twenty-two minutes into the live run I changed the page. It was showing "0 tests" and "10 commits" for a repository with several hundred of each, because the build counted them by running tools that don't exist in the deploy environment. A page whose entire argument is that its numbers are measured rather than typed was publishing two false ones to every reader arriving from the post. My own rule said no mid-run content changes. I made the change anyway, and logged it as a deviation rather than editing the rule to match what I'd done. It can't have biased the comparison — that block is identical across all three versions, so every version changed at the same instant — but a pre-registration you quietly revise afterwards isn't one, and the reasoning belongs in the record where someone can disagree with it. § 06 · The best thing it produced was a refusal Two people got the readout. One of them was a stranger who'd signed up five days earlier, having been told they'd get the result whichever way it went. The email says the sample was too small to tell, shows the numbers that would have made a nice chart, and explains why they don't support the chart. Everything underneath worked. Assignment, propensity logging, bot filtering, the conversion beacon, reward maturation, the send path — all of it ran unattended for two weeks and produced a correct answer to a question the data could not support. The machinery is validated. The marketing question isn't, and can't be, on thirty-four sessions. I set out to learn whether an agentic system could run a marketing loop end to end. It can. What I didn't expect was that the most valuable thing it did all fortnight was decline to tell me what I wanted to hear. Any system can produce a number. The useful ones tell you when the number means nothing. ================================================================================ ESSAY · YOUR AGENT WORKS. PROVE IT. Source: https://sambr.com/writing/your-agent-works-prove-it ================================================================================ slug: your-agent-works-prove-it source: https://sambr.com/writing/your-agent-works-prove-it published: 2026-07 author: Sam Bloomberg-Rissman tags: product, ai-agents, evaluation, benchmarks, reliability estimated-read: 5 minutes (~1300 words) Deck: A single run isn't a measurement. Agents are nondeterministic, most teams score them as if they aren't, and the number that ends up on the slide is often quietly dishonest. § 01 · The number on the slide Somebody runs the new agent through a test set, it gets 92% right, and that number goes on a slide. From that moment it's real. It anchors the roadmap, the launch decision, the exec update. Nobody in the room asks the three questions that actually matter: which run, out of how many, and what did "right" count as. I went deep on how you actually tell whether an agent works — the studies, the benchmarks, the eval tooling, the practitioner writing buried underneath the vendor noise. I came out convinced that most of the numbers we quote about agents are softer than we think, and that the softness is knowable. We just don't look. This is the fourth piece in a series about the unglamorous layer beneath agents. The first three were about access, data, and maintenance. This one is about measurement, and it turns out to have the same shape as the others: the interesting problem isn't the model, it's everything we assumed we could skip. § 02 · A single run is not a measurement Here's the fact that reorganized how I think about this. Run the same agent on the same task twice and you often get two different runs — not two phrasings of the same solution, but different strategies, different tool calls, different outcomes. One study put 60,000 agent runs against a standard coding benchmark and found that single-run success estimates moved by two to six percentage points depending on which run you happened to report. The standard deviation stayed above a point and a half even at temperature zero, the setting that is supposed to make the model deterministic.[1] The mechanism is simple and a little unsettling. Trajectories diverge inside the first few percent of tokens, and small early differences cascade into entirely different solution paths. Another study found that how consistently an agent behaves on a task predicts how well it does. The same agent on identical inputs produced 2.3 to 4.2 distinct action sequences per ten runs; the tasks it approached consistently landed at 82 to 87% accuracy, the inconsistent ones at 41 to 65%.[2] > A single run tells you the agent can succeed. It doesn't tell you it will. So a green checkmark from one run isn't a measurement. It's one sample from a distribution you never characterized. When a two or three point improvement shows up between two versions, you genuinely cannot tell, from single runs, whether you improved the agent or just resampled the noise. § 03 · The metric that lies without meaning to Once you accept that each run is a sample, the next question is what you do with a handful of them. This is where the most common quiet dishonesty in the field lives. Two metrics look almost identical and mean opposite things. pass@k is the probability the agent succeeds at least once in k tries. It goes up as k grows. It's the right metric when you can retry cheaply and a human picks the winner — code generation where you review three candidates and keep the good one. pass^k is the probability the agent succeeds on every one of k tries. It goes down as k grows. It's the right metric when your user lives every attempt, which is every customer-facing agent in production. The arithmetic is unforgiving. An agent that succeeds 75% of the time per attempt is at 42% by pass^3. τ-bench, the benchmark that introduced pass^k, found that even a strong function-calling agent solved under half its tasks and fell below 25% by pass^8 in the retail domain.[3] Same agent, same tasks, different survival function. > Reporting pass@k for a customer-facing agent isn't lying. It's publishing a number that is technically true and practically false. Your user does not get to retry. They don't see the one run in five that worked; they see the run they got. If your metric quietly assumes a human is standing by to pick the best of k, and no human is, you've shipped a number that describes a product you didn't build. § 04 · The path is part of the answer There's a second thing a final-output score cannot see. An agent can reach the right answer through a route that will not generalize — a lucky guess, a shortcut that happens to work on your test set. And it can do the opposite: produce a fluent, confident report about work it never actually did. One study of coding agents measured exactly that second failure — models that submitted a patch on all five runs and resolved the task on none of them, at rates from 16% to 80% depending on the model.[4] Both failures look fine if you only grade the destination. This is the event agent that didn't know its own address, moved up a level. There, the failure was invisible until you looked at what users actually asked. Here, the failure is invisible until you look at what the agent actually did — the trajectory, not just the result. You have to read the path. The output will not tell on itself. § 05 · There are three places to measure, and you're using one Most teams measure an agent once, offline, before launch, and treat that as the answer. Offline evaluation runs a fixed dataset with known answers in a controlled environment. It's where regression testing lives, and it's necessary. It's also the only place with clean ground truth, which is why everyone stops there. Online evaluation scores real production traffic, where you rarely have ground truth and have to lean on proxy signals and sampled human review. Monitoring watches the live system for drift, because the agent that measured well in March is grounded on information that has changed by October. If that last line sounds familiar, it's the argument from the previous essay wearing different clothes. Measurement isn't a gate you pass through once. It's a loop you keep running, for the same reason the data work is a loop: the world underneath the agent doesn't hold still. § 06 · The discipline None of this is exotic. Sample more than once and report the spread, not a point. Pick the metric that matches how your user actually experiences the product. Read trajectories, not just outputs. Measure again after you ship. It isn't sophisticated. It's just work, and it's the work most teams skip, because a single good demo is so much more persuasive than an honest distribution. I keep landing on the same place across all four of these essays. The gap is never where the demo points. It's in the layer underneath — the access, the data, the maintenance, and now the measurement — that nobody owns until something breaks in public. > A demo is a single run you got to choose. Production is every run, chosen by someone else. Reliability is the difference between the two. It's the difference between an agent that impressed a room once and an agent people can depend on. And you cannot claim it from one green run. You have to prove it. References [1] Bjarni Haukur Bjarnason, André Silva, Martin Monperrus. "On Randomness in Agentic Evals." arXiv:2602.07150. https://arxiv.org/abs/2602.07150 [2] Aman Mehta. "When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents." arXiv:2602.11619. https://arxiv.org/abs/2602.11619 [3] Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan. "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains." arXiv:2406.12045. https://arxiv.org/abs/2406.12045 [4] Aman Mehta. "Confident and Wrong: Silent Semantic Failures in Coding Agents." Snowflake AI Research. arXiv:2603.25764. https://arxiv.org/abs/2603.25764 ================================================================================ ESSAY · SHOW, DON'T TELL. Source: https://sambr.com/writing/show-dont-tell ================================================================================ slug: show-dont-tell source: https://sambr.com/writing/show-dont-tell published: 2026-06 author: Sam Bloomberg-Rissman tags: product, product-management, prototyping, ai-agents, craft estimated-read: 3 minutes (~800 words) Deck: No, AI isn't making product management obsolete. It handed me what the job always needed and never had: a cheap way to show instead of tell. § 01 · The question everyone asks Everyone keeps asking me some version of the same question. Is AI making the product management job obsolete? > No. I'll give the longer answer, because the short one annoys people. The job is not obsolete because the job was never the part AI is good at. AI writes the deck. It does not decide the deck was the wrong thing to build. Somebody still has to find the problem worth solving, figure out a way to solve it, and get a room full of people who disagree to actually work together on it. That has always been the job. It still is. § 02 · What the job actually is Kai McConnell put it best. The whole job of a product manager, she told me once, is to create shared understanding among the people trying to solve a problem together. Not the spec. Not the roadmap. Not the standup. Just the picture in everyone's head, and whether it's actually the same one. I've repeated that line to more people than I can count, usually as if it were my own. (Sorry, Kai.) Strip the job down and you get three things: finding the problems and opportunities worth chasing, working out how to solve them, and getting everyone working on them to share an understanding of what "done" looks like. None of that is producing artifacts. It's all judgment, and judgment is the part the tools don't have. AI does not find the problem. It does not know which opportunity is real. It cannot tell you that the room only thinks it agrees. > What AI changed is not the job. It's how I do the third part. § 03 · The way I used to do it For years my answer to shared understanding was telling. The way to do it was to create a deck, write a brief, and tell people in meetings. I was good at all three. Crisp PRDs, tidy acceptance criteria, the occasional diagram I was far too pleased with. The documents were excellent. The understanding was not. Here's the problem with words. When I describe a flow, I've finished the job in my own head. The reader hasn't started. They take my sentences and rebuild a version of the thing inside their own assumptions, and no two people rebuild the same one. Engineering pictures one product. Design pictures another. The exec pictures a third, usually with a chart in it. Everyone nods. Everyone leaves the room confident. Everyone is confident about something different. > Telling optimizes for the speaker. Showing optimizes for everyone else. § 04 · Why I kept telling anyway Every PM knows this. The reason we kept telling anyway is that showing was expensive. To show, you needed a working thing, and a working thing needed engineers and a sprint and a build, so you couldn't spend it on a maybe. Telling was free. So we told, and we held our breath, and we found out at the demo whether the picture in the room had ever matched the one in our heads. That math is gone now. The thing AI tools actually changed for me is not speed of writing. It's that the cost of showing collapsed to roughly the cost of telling. I can stand up a clickable prototype in an afternoon. I can build a rough working version of an agent and put it in front of people, who then stop arguing about the abstract noun "agent" and start arguing about the actual thing doing the actual thing, which is the only argument worth having. So I stopped writing the paragraph about how the onboarding should feel and built the onboarding. I stopped pitching the feature in a meeting and put a working version in front of people instead. The artifact becomes the center of the table. Nobody has to reconstruct anything, because it's right there, behaving. Disagreement gets specific immediately. "I don't like it" turns into "this step is wrong," which is progress you cannot buy with a deck. The deliverable was never the document. It was never the prototype either. The deliverable has always been the shared picture, and everything else, the spec, the deck, the working demo, is only the means to it. I spent years mistaking the means for the work. § 05 · So, obsolete? The opposite. AI took the cheapest, worst medium I had for the hardest part of the job and swapped in the best one. It did not take the job. It removed my excuse for doing it badly. If the room leaves with three different pictures now, that's on me, not the tools. The job was always shared understanding. The medium was a compromise I made because the better one was out of reach. It isn't out of reach anymore. So I show. I don't tell. Mostly. ================================================================================ ESSAY · NOBODY GAVE IT THE ADDRESS. Source: https://sambr.com/writing/nobody-gave-it-the-address ================================================================================ slug: nobody-gave-it-the-address source: https://sambr.com/writing/nobody-gave-it-the-address published: 2026-05 author: Sam Bloomberg-Rissman tags: product, ai-agents, data-quality, knowledge-management, maintenance estimated-read: 5 minutes (~1100 words) Deck: Your agent logic is probably fine. The information underneath it isn't. That's the job now. § 01 · The embarrassing failure Someone posted on LinkedIn that an event agent couldn't tell them where the event was located. Not the schedule. Not the speakers. Not the session abstracts. The physical address. The team had done everything right. They'd thought carefully about the conversational experience. They'd connected the right systems. They'd written good instructions. And the agent confidently didn't know where the event was, because nobody had thought to give it that information. This is the failure mode nobody talks about when they demo agents. Not hallucination. Not broken logic. Not a model limitation. Just: the thing the user needed wasn't there. It's embarrassing in its simplicity. And it's far more common than anyone admits. § 02 · The assumption buried in every build When you build an agent, you think about capabilities. What can it do? What can it access? What questions should it be able to answer? You design for the use cases you can imagine, instrument for the behaviors you expect, and ship when it works well enough in testing. What you don't do, almost by default, is audit for completeness. Nobody stops during the build and asks: what would this agent not be able to answer? What information exists in the world that a reasonable user might expect it to have, that we simply haven't given it? The question feels obvious in retrospect. In practice, it never gets asked — because you're focused on what you're building, not what you're omitting. The address wasn't missing because of a technical failure. It was missing because everyone involved assumed someone else had handled it. The agent team assumed the data team had it. The data team assumed it was in the system. The system had it in some form, somewhere, for some purposes — just not in a form the agent could use. > The data exists in the organization. The agent just can't reach it, or reach it cleanly, or reach it in a way that surfaces a useful answer. § 03 · The wrong question we keep asking Most teams evaluate their agents by asking: is it working? It's the wrong question. Working means the logic executes. Working means the conversation completes. Working means no errors in the logs. An agent can be working perfectly and still be failing the people using it — giving confidently incomplete answers, staying silent when it should help, or deflecting questions it has no reason to deflect. The right question is different: what is my agent actually being asked, and is it answering well? These sound similar. They're not. The first question is about the system. The second is about the interaction. The first tells you the agent is running. The second tells you whether it's useful. When you look at what people actually ask, patterns emerge fast. There are categories of questions the agent handles well. There are categories where it hedges or fails or goes quiet. And when you trace those failures back, the cause is almost never the agent logic. It's the information the agent is grounded on — what's there, what's missing, what's out of date, what's structured wrong for the kind of retrieval the agent needs. The agent didn't fail to answer "where is the event?" because the reasoning was bad. It failed because the answer wasn't in the ground truth it was working from. Fixing that isn't an agent problem. It's a data problem. § 04 · The loop nobody told you was the job Once you accept that agent quality is mostly a data quality problem, the work changes. It becomes a loop. You look at what users are asking. You identify where the agent is struggling — not just hard failures, but weak answers, hedged answers, answers that are technically accurate but not actually useful. You trace each one back to its root: is the information missing entirely? Is it present but unstructured? Is it present and structured but stale? Each of those has a different fix, and none of those fixes happen in the agent layer. Then you fix the data. You add what's missing. You structure what's scattered. You establish who owns keeping it current. And then you run the loop again — because new questions will surface new gaps, because content changes, because what users need shifts over time. This is maintenance. Not maintenance as an afterthought — maintenance as the primary ongoing discipline of running an agent in production. It's not glamorous work. There's no demo for "we added the event address." Nobody posts about the incremental data cleanup that made the agent meaningfully better for the next hundred conversations. But that's where most of the improvement actually lives. The organizations whose agents get better over time aren't the ones with the most sophisticated architectures. They're the ones that have made a practice of closing the loop — watching what the agent can't answer, and making sure it can answer it next time. § 05 · The groundskeeper problem In the two previous pieces in this series, I argued that the future of content is agentic access, not better destinations — and then that building the access layer is mostly a data model problem, not an engineering problem. This is the third part of that argument, and in some ways the most unglamorous: it's never done. You don't build an agent, verify it works, and move on. You build an agent, put it in front of people, watch what breaks, and fix the underlying data. Then you do it again. The agent logic might be stable for months. The data underneath it won't be. Events move. Speakers change. Information that was accurate in March isn't accurate in October. And every change is a potential gap between what a user asks and what the agent can usefully answer. There's a job at every venue called the groundskeeper. Their work is mostly invisible. When it's done well, you don't notice it. You only notice when it hasn't been done — when the lights are out, when the field is uneven, when something that should have been maintained wasn't. > Agents need groundskeepers. People whose job is to watch what's being asked, close the gaps, and keep the underlying information current and complete. That role doesn't have a clean title yet. It's distributed across data teams and content teams and product teams, and nobody feels fully responsible for it. That's the gap. Not the model, not the protocol, not the interface. The gap is the assumption that once the agent is live, the information work is finished. It isn't. It's just starting. ================================================================================ ESSAY · I ASKED CLAUDE TO BUILD MY MCP SERVER. HERE'S WHAT I ACTUALLY GOT. Source: https://sambr.com/writing/claude-built-my-mcp-server ================================================================================ slug: claude-built-my-mcp-server source: https://sambr.com/writing/claude-built-my-mcp-server published: 2026-05 author: Sam Bloomberg-Rissman tags: product, ai-agents, mcp, claude-code, data-modeling estimated-read: 6 minutes (~1500 words) Deck: Standing up an MCP server took a day. Understanding what I built took longer. The interesting part wasn't the engineering. § 01 · The obvious next step Last week I published an essay arguing that the future of content isn't better destinations. It's enabling agentic access, exposing your content and capabilities so they work in any agent context, not just the surfaces you control. MCP servers, structured knowledge, agent-readable interfaces. The whole argument. Then I went and implemented it on my own site. Hard to argue for something you haven't tried. I used Claude Code to build it. I described what I wanted (a public, read-only MCP server for sambr.com that would let any MCP-aware client query my photo gallery, my writing, my bio, and my contact info without scraping), and Claude Code wrote the implementation. It took about a day of iteration. It works. What I want to do here is open the hood. Not because the engineering is complicated (it isn't), but because the interesting decisions weren't technical. They were about data, and what you learn when you try to make your own content queryable by machines. § 02 · What Claude Code actually built The server runs as a Vercel serverless function at https://sambr.com/api/mcp. Each request is stateless: no sessions, no memory between calls. A request comes in, gets handled, and the function terminates. Vercel's runtime is short-lived by design, so stateless is the right fit. The protocol is JSON-RPC 2.0, which is what MCP runs on. A client sends a method name and parameters, the server returns a result. The outer layer of the implementation is essentially a router. It receives a method like tools/call or resources/read, dispatches to the right handler, and returns a structured response. That routing logic is about 120 lines of clean, unsurprising code. The server exposes three things: Tools, callable functions: find_photos, get_photo, find_hero, list_writing, get_essay, about_me, contact. These are what an agent actually invokes when it wants to do something. Resources, named content at URIs like sambr://gallery and sambr://writing/dont-build-the-agent. Resources are readable blobs rather than callable functions, closer to a document than an API endpoint. Prompts, suggested templates that tell a client how to use the server well. summarize_photographer, recommend_photos, cite_essay. These are hints, not commands. An agent can follow them or ignore them. Underneath the tools, two data stores. Photo data lives in Vercel KV, a Redis-based key-value store, under two keys: gallery (the full photo array) and hero (the homepage selection). Essays, bio, and contact info are hardcoded in a data.js file, static text that doesn't need a database. That's the whole thing. It's genuinely not complicated. The protocol handles the rest. § 03 · Where it's different from a standard API The obvious question when you first look at an MCP server is: why not just a REST API? The tools look like endpoints. The JSON-RPC envelope looks like overhead. What's actually different? The difference isn't in the transport. It's in who the client is. A REST API is designed to be called by code that a developer wrote, code that knows exactly which endpoint to hit, what parameters to send, and how to parse the response. The developer read the docs, wrote the integration, and everything is predetermined. An MCP server is designed to be called by an agent that has never seen your server before. The agent reads the tool descriptions at runtime, decides which tool to call based on what it's trying to accomplish, constructs the parameters from context, and interprets the result. There's no developer in the loop. The tool description is the documentation, and the agent reads it dynamically every time. > With a REST API, you write for a developer who will read your docs once. > With MCP, you write for an agent that will read your tool descriptions on > every single call. This changes how you write the descriptions. Look at the find_photos tool definition and you'll find a line that reads: "Place / location metadata is NOT indexed. There is no way to filter by city, country, or region." That sentence exists because without it, an agent asked to find photos from California would try a location filter, get nothing, and either fail silently or hallucinate. The description is behavioral guardrails, not just documentation. The other structural difference is discoverability. A client connecting to the MCP server for the first time calls tools/list and gets back everything it needs to understand what's available. No docs site, no SDK, no prior knowledge required. The server describes itself, and the agent figures out what to do with it. That's the protocol doing real work. It's not just a convention; it's what makes the whole thing composable across tools and clients that have never met before. § 04 · The decision that taught me the most When I first tested the server, I asked it to find photos taken in California. It returned nothing. Not a hallucination, just an honest empty result, because location isn't indexed. That wasn't a bug. It was a decision I'd made deliberately, and then forgotten I'd made. The photo metadata doesn't include location data. My Leica M11 doesn't embed GPS coordinates in the EXIF. I could add location manually (go back through several years of photographs, tag each one with a place), but I chose not to. The effort wasn't worth it for a personal site, and I didn't want to introduce inconsistent or half-complete data into a queryable system. Better to scope it out cleanly than to have an agent return partial results and present them as complete. So the tool description says explicitly that location isn't indexed. An agent that reads it won't try. An agent that ignores it will fail gracefully rather than confidently wrong. > The quality of your MCP server is directly constrained by the quality of > your underlying data. You can't query what you didn't capture. This is where the engineering stops being the interesting part. The server is fine. The protocol is fine. The constraint is the data model: what exists, what's structured, what's queryable, and what decisions were made years ago when nobody was thinking about agents at all. Most organizations are in a version of this situation, at much larger scale. The content exists. The data exists. But it was captured for human consumption, not machine retrieval: embedded in page layouts, split across CMS fields that made sense for publishing workflows, tagged inconsistently, or simply never tagged at all. Standing up an MCP server on top of that doesn't fix it. It exposes it. § 05 · What this means for the argument I made last week The essay I published last week argued that the shift from destination-based content to agentic access is a distribution problem, not a content problem. I still believe that. But building this server added a layer I undersold. It's also a data model problem. And the data model problem is harder, because it requires going back (back through existing content, existing metadata, existing publishing workflows) and making decisions about structure that nobody thought to make at the time. That's not a technical project. It's an organizational one. Someone has to own it, resource it, and make the call that it's worth doing. For my photography site, the scope was manageable and the tradeoffs were clear. For an enterprise content library with fifteen years of accumulated pages, PDFs, and CMS records, the same audit is a different kind of undertaking entirely. But the direction is the same. Figure out what your content actually is, not as pages, but as entities with attributes and relationships. Decide what's worth making queryable and what isn't. Be explicit about the gaps. Then build the interface that exposes it cleanly. Claude Code can build you the server in a day. The data model is still your problem. Which is, I think, exactly right. The engineering is the easy part. It always was. ================================================================================ ESSAY · DON'T BUILD THE AGENT. ENABLE THE ACCESS. Source: https://sambr.com/writing/dont-build-the-agent ================================================================================ slug: dont-build-the-agent source: https://sambr.com/writing/dont-build-the-agent published: 2026-04 author: Sam Bloomberg-Rissman tags: product, strategy, ai-agents, mcp, llms-txt, knowledge-management estimated-read: 7 minutes (~1700 words) Deck: We've been asking the wrong question. The goal was never a better app. It's making your capabilities available wherever your users already are. § 01 · The channel history nobody talks about For the past twenty years, there were three ways users reached your content. HTTP — the website. SMTP — email. And eventually, native apps. Every information architecture decision, every engagement strategy, every analytics stack was built around those three assumptions. You owned a surface, drove people to it, and measured what happened when they arrived. That model shaped how we think about products at a level so fundamental it's become invisible. When we say "build an experience," we mean: design a surface. When we say "meet users where they are," we mean: optimize for mobile, or send a better email. The destination is always implicit. It never occurs to us to question it, because for twenty years it was the only model that existed. A fourth channel has arrived. It doesn't have a homepage. It doesn't arrive at your front door. And it's growing faster than any of the three that came before it. § 02 · The channel you're not designing for When an attendee asks ChatGPT what AI sessions they should attend at a conference, they're not opening the event app. When they tell Claude to help plan their day, they're not on the website. When a Slack copilot surfaces a speaker recommendation mid-conversation, your content either shows up — correctly, usefully, in context — or it doesn't show up at all. These aren't edge cases. They're becoming primary surfaces for how people research, plan, and act. And for most organizations, the answer those agents return is a hallucination, a poorly parsed scrape of a web page, or an honest "I don't have that information." > We're invisible to the fastest-growing engagement channel in a generation — > and we built that invisibility into our architecture. This isn't a content problem. It's a distribution problem. And unlike prior distribution shifts, this one doesn't reward you for showing up late. § 03 · The mistake hiding in plain sight The instinct when AI arrived was to build an agent. A conversational experience, powered by a model, that helps users navigate content. It's the right instinct — that work matters and should continue. But it's too narrow. Building an agent means building another destination — a different interface, a smarter one, but still something the user has to find and open and stay inside. We replaced the app with a chatbot and called it transformation. The deeper shift isn't about building a better agent. It's about enabling agentic access — exposing your content and capabilities as skills that any agent can invoke, regardless of where the user happens to be. The question isn't "how do we make our agent better?" It's "how do we make our content work inside the agents our users are already using?" That reframe changes everything downstream: what you build, what you instrument, what you measure, and what counts as success. § 04 · What agentic access actually looks like Making your content agent-accessible isn't speculative. The infrastructure exists now and the patterns are becoming clear. Structured data as the foundation. Agents retrieve and synthesize; they don't browse. If your content isn't marked up semantically — Schema.org, JSON-LD, clean and queryable APIs — agents either can't use it or use it incorrectly. This is the unglamorous work that determines whether you exist in AI-mediated contexts at all. llms.txt as your agent-facing front door. An emerging convention that gives AI systems a curated, trustworthy map of your content and capabilities. Think of it as what robots.txt did for search — a deliberate signal about how you want to be found and represented. Companies that adopt it early are making an intentional choice. Companies that ignore it are making that choice too, just passively. MCP servers as the capability layer. The Model Context Protocol is becoming a standard interface for exposing tools and data to AI agents. An MCP server means any of them — Claude, ChatGPT, a Slack copilot, a custom enterprise agent — can query your content, surface personalized recommendations, and act on a user's behalf, without that user ever opening your app. You stop being a destination and start being a capability that travels. Together, these aren't features you ship in a release. They're infrastructure — the layer beneath the experience that determines whether your value can exist outside the surface you control. § 05 · The discipline we've been avoiding Here's the uncomfortable part. Most organizations can't do this yet — not because the technology isn't ready, but because the underlying content isn't. We've spent twenty years producing pages, not knowledge. A page is a human-readable container. It has a URL, a layout, navigation, a designed context. It was built to be seen, not queried. The information inside it — a session description, a speaker bio, a product detail, a pricing tier — is embedded in HTML, wrapped in design decisions, tangled with adjacent content. An agent trying to extract a clean fact from a typical web page is doing archaeology. Knowledge is different. It's structured, typed, and relationship-aware. A session isn't a block of text on a page — it's an entity with attributes: title, format, track, speakers, time, room, prerequisites, related content. Those attributes have relationships to other entities. A speaker belongs to a company, has a role, has spoken at prior events, has expertise tags that connect to sessions in adjacent tracks. When that structure exists, agents can do something genuinely useful with it. When it doesn't, they guess. > The shift from pages to knowledge isn't a content strategy refresh. It's a > different discipline entirely — one most product and marketing teams have > never had to develop. Knowledge management means treating your content as a data model, not a publishing workflow. It means asking: what are the entities that matter in our domain? What are their attributes? How do they relate to each other? Who owns the quality of each? How does it stay current? These are questions that information architects and data engineers think about. They're not questions that content teams or web producers have historically been asked to answer — and that gap is now a strategic liability. The organizations that will be most visible in an agentic world aren't the ones with the most content. They're the ones with the most legible content — structured, maintained, and queryable at the entity level. Getting there requires investing in knowledge infrastructure the same way we invested in web infrastructure in the early 2000s. It felt like plumbing then too. It turned out to be the foundation everything else ran on. § 06 · What this means for how we build I've spent most of my career designing destinations. The discipline was: understand the user's journey, design each step, instrument everything, iterate toward better outcomes on the surface you control. It's good work. It produced real value. It's the wrong frame for what's coming. In an agentic world, the product leader's job shifts from designing experiences to designing access. The questions change. Instead of "what does the user see when they land here?" you ask "can our content be found, understood, and acted on by a system we don't control?" Instead of "how do we increase engagement on our property?" you ask "how do we make our value portable enough to exist in any context?" > Build for the surface, and you're one channel shift away from irrelevance. > Build for the capability underneath, and the surface becomes secondary. This requires different instincts and, honestly, different metrics. Engagement on your property is still worth measuring. But if that's the only thing you're measuring, you're blind to the growing share of value you're delivering — or failing to deliver — through channels you never see. § 07 · The bet worth making The organizations I'm watching most carefully aren't the ones with the most sophisticated apps or the most polished conversational agents. They're the ones treating their content and capabilities as infrastructure that should work anywhere — in any agent, through any interface, on behalf of any user whose AI has decided this is worth their attention. That's a different kind of product work. Less visible, harder to demo, slower to show in a dashboard. But it's the work that determines whether you're present in the world that's arriving — or whether you're a well-designed destination that fewer and fewer people visit, because fewer and fewer people are browsing. We built twenty years of strategy on three channels. The fourth one is here. The question is whether we treat it like a feature or like the shift it actually is. ================================================================================ CONTACT Source: https://sambr.com/#contact ================================================================================ Email: website@sambr.com Instagram: https://www.instagram.com/sambr/ LinkedIn: https://www.linkedin.com/in/sambloombergrissman/ Site: https://sambr.com Prints: https://prints.sambr.com