I built a production web application, and I did not read even one line of its code. The code exists: a FastAPI application, a Postgres database, more than thirty migrations, a Slack app, a command-line tool, backups, monitors. I never opened a file, on purpose.

This is not what people call vibe coding. Vibe coding accepts whatever runs. I accepted only what I could verify from the outside, and I refused the rest. The AI agent (Claude Code) was the contractor. I was the client with a precise specification and a hard acceptance test. That is the same job we normally do with human contractors. The code quality seems to be good enough to pass all my tests, which is probably similar to what you get from human contractors. The difference is the speed of the loop.

The loop

Every change follows the same five steps.

First, I write the work item as prose in an issue: the problem, the rule I want, and how I will know that it works. The agent reads it and argues back. We go a few rounds until the plan converges, and the plan lives in the issue, not in a chat window. Often, especially for UI features, I get a few mockups as well for me to choose from. I make sure to use one of the better agents for this, e.g. Claude Opus, or, if it’s an important or large work item, Fable. (This will probably age badly. Human from the future: These were considered state of the art at the time of writing.)

Second, the agent builds on an isolated branch in a git worktree with its own copy of the database. Nothing it does can touch production, by design.

Third, it runs the test suite, which currently takes about five minutes, and then it automatically starts a rehearsal: It pulls a fresh copy of the live data, opens every page the change touches, and reads the server log.

Fourth, it opens a merge request with a description written for a human. I read the description, I click through the rehearsed application, and I approve the merge myself. That is my rule, and it is the one rule that I never relaxed. The agent handles potential merge conflicts, which can occur if multiple agents are working on features at the same time. One of the most useful ways to avoid problems is to adhere strictly to Alembic data schema versioning, so that features which touch the same data cannot be merged without a data migration, typically including a rebase of the later branches onto the ones that go before it. More information about that below.

Fifth, the change deploys and the monitors watch it.

This is the process in a nutshell. I think that many of the components can be swapped out, but that the overall structure is pretty solid.

What I look at instead of code

The schema page

Since my time in academia working on HPC code in Fortran, carefully planning memory layout of all my variables, avoiding memory leaks and out-of-memory problems, and manually designing optimized OpenMP and MPI parallelization strategies, I have come to believe that the data structure behind the scenes is the basis of everything. I think the same holds for web apps. It follows, then, that the UI is just a slightly more ephemeral shell on top of the data. It makes a lot of sense, therefore, to be very meticulous with the data schema, and a little bit less so for the UI: In other words, a UI bug doesn’t hurt as much as a badly designed data schema.

The engine, as I call my web app, visually renders its own schema in detail and in an aesthetically pleasing way: every table, its row count, its unreferenced rows, when it last changed, and whether the audit trail covers it. This is the page that I open after every change. It once showed a box that read “empty, 0 unreferenced, no trail entries” next to sixteen tables full of rows. That box was a table that nothing read any more. I had an agent investigate it, and delete it afterwards. Part 3 shows this schema in detail.

The rehearsal

A green test suite once hid a traceback on every page load. The suite tested the functions; the pages needed the functions in a different order. Since then, no change ships without a click-through on a copy of the live data, with the server log open. Every bug that still reached production came from a rehearsal that differed from production in exactly one detail. So the copy is exact now: same database engine, same data, same sign-in path.

The migrations

Every change to the schema is a numbered, reversible step with a written explanation of why. I read the explanation, never the SQL. Two checks keep the steps honest: One command compares the code’s picture of the schema with the real database and must find nothing to say. One test builds a database from the steps, builds another from the code, and compares them column by column.

The unified access to data

Every change to the data goes through the application. Nobody writes SQL directly: not me, not the agent, not a script. So every change carries who made it, when, through which “door” as I call it (web, CLI, migration), and which version of the code. A change without an author is unwritable, and any change can be undone from the trail. This is what the audit part of the schema looks like:

erDiagram
    CHANGE_SET ||--|{ EVENT : "groups"

    CHANGE_SET {
        text id PK
        timestamptz at
        text actor "a handle, no foreign key - history outlives rows"
        text source "web, CLI, migration"
        text code_version "which code made the change"
    }
    EVENT {
        int id PK "monotonic - the handle an undo takes"
        text entity_type "points at any table, no foreign key"
        text entity_id
        text kind "seven kinds, database-checked"
        text field "which column, for an update"
        json old_value
        json new_value
    }

The monitors

An open-source error tracker (GlitchTip) collects every exception with its stack trace. Each scheduled job reports to a heartbeat monitor when it runs. There is also an alarm that fires on silence designed to catch a job that never started, which is useful for the automatic backup strategy that the app has in place.

The stack

For the engineers, the checklist of what production means here:

• Hosting: one small VPS in Europe (Hetzner), one Linux box, a few euros each month.
• Sign-in: Google sign-in through oauth2-proxy, limited to the accounts of our own Workspace. Two roles, readers and editors, named in one line of configuration.
• Three doors: the browser (the proxy puts the signed-in address in a header), the command line for development (locked by SSH to the box), and an API token for CLI production usage.
• Database: Postgres. I started on SQLite in an early prototype and moved before the second user arrived.
• Migrations: Alembic, more than thirty numbered steps, each with its prose.
• Backups: scheduled, each run reporting to a heartbeat.
• Monitoring: GlitchTip for errors, heartbeats for the jobs.
• Application: FastAPI, server-rendered pages, one stylesheet.
• Beside it: a Slack app for the daily digests, and a command-line tool for bulk work and recovery.
• Source: a self-hosted GitLab, one merge request per change, merged by a human.

The skill

What did I actually do, if not programming? I wrote what I wanted, in sentences a colleague could check. I split big wishes into steps that each have a test. I said no, often: the most useful sentence I wrote was “this is too complicated, fix the cause one layer down”. And I refused to accept “it works” without seeing it work on real data.

Sales operations people already do all four things. They write requirements for vendors, cut projects into phases, push back on scope, and sign off on acceptance tests. The AI agent changes one thing only: the answer to a requirement arrives in minutes instead of in the next steering meeting. That speed is what makes the loop worth running for a tool with a handful of users.

The technical half of my experience helped with the vocabulary. I know what a foreign key, a migration and a heartbeat are, so I could name what I wanted checked. I did not need it to read the implementation, because I never did.

The limits

Where the acceptance test was weak, the bugs came through. Five bugs reached production, and each came from a rehearsal that was one detail short of the real thing. The fix each time was a more exact rehearsal, not a look at the code.

Three things I never skipped: SSO integration before the first real record, backups before the first week, and the audit trail before the first colleague.

And I would still bring in an engineer for two things: A security review before people outside the company get accounts, and anything that involves money.

Part 3

The schema is where all of this fit lives: Every rule above is a rule that the database keeps, and every number in a digest traces to a column with a name. Part 3 shows the schema itself, and what it makes possible: the digests, the generated messages, the slide decks.

Part 3 comes next. It shows the data schema and what it feeds.

In the gold rush, the sellers of shovels made safer profits than the gold miners. Sales kept the language of that time: The word prospecting comes from the search for gold. The shovel business also continues today: A full industry sells tools to people who sell. This industry has CRM licenses, implementation partners, admin certificates, and connector marketplaces. I have worked in such a role as technical expert for years, working with prospective and existive customers, serving as their technical point of contact and helping them achieve their goals.

However, I am also an inveterate tool builder. So earlier this year, I did the opposite of what a salesperson usually does. I refused to buy the shovel. I made my own shovel. I built a complete prospecting engine. The engine is a web application with sign-in, a production database, hosting, automatic backups, and monitoring. It currently connects directly to Slack and to my document tools.

I paid no license. I spoke with no vendor. I attended no configuration workshops. The costs are a small European VPS, a few euros each month, and an AI subscription (Claude) that I already pay.

Half of my experience is commercial: prospecting, pipeline, proposals, and sales operations. The other half is technical: computational physics and enterprise AI architecture. This series shows what occurs when the two halves want the same tool.

The demo that fits nobody

Each CRM demo shows a different company’s sales process with your logo on it. The demo is attractive. The workflow is almost yours. The word “almost” is the expensive part.

You buy the licenses. Then you pay an implementation partner to adjust the tool to your process. Then you appoint an admin to maintain those adjustments. Two years later, the team does its real work in spreadsheets and puts a summary in the CRM on Friday afternoons. If you have ever worked with a sales team, you know a version of this story.

The vendors do not cheat you. Their product must be an average of thousands of sales processes. Thus it can match your process only partly. Configuration makes the difference smaller. The difference never becomes zero, because “configurable” means that you can select only from their options.

The important number is 100%

I will not claim a savings multiplier. My claim is about fit. The fit is 100%, because the tool is exactly what I want.

When you buy, you select from options. When you build, you make decisions. Each field on each screen exists because I decided to put it there. A prospect in my engine is not “an Account with custom fields”. It has only the states that occur in my real process. Its grades are my grades.

Its deadlines send alerts to Slack, where I already look. Nothing else is on the screen. The tool contains my sales process and zero other ideas.

This table shows the exchange.

Note that the second column contains obligations, not only savings. Backups, monitoring, sign-in, and an audit trail become your tasks.

In part 2 shows how I do these tasks at production quality without a team. The short answer is that these tasks cost discipline, not extra people.

In part 3 I’ll show that the data schema is at the heart of everything, making this possible.

The condition

This works only if you know exactly what you want. The cause of the success is not the AI, and not a developer. The cause is domain knowledge. I specified this engine precisely because I participated in some sales process or another for years. I know what makes a prospect qualified. I know the purpose of each deadline.

I know what a salesperson must have before a talk with a decision maker. An expert with an unclear wish list gets an unclear tool. The quality of the AI does not change that. To know what you want is less common than to know how to write code.

You do this specification work also when you buy. Each buyer of a CRM produces requirement documents, score matrices, and workshop notes. When you build, the same text goes to an AI agent. The agent supplies the result in minutes. A partner only schedules the next meeting.

You own the data

The data is mine, in a plain Postgres database. One command exports all of it. The team can grow without per-seat costs. I have no lock-in and no renewal date. When my process changes, the tool changes in the same afternoon.

When to buy

A purchase is better in these conditions:

• You need hundreds of seats, have no time to set up single sign-on across departments, and want to work with approved vendors with a service-level agreement (SLA).
• Compliance rules say that you must have a certified third party.
• Your sales process is standard, so the difference is small and a license is the cheap option.
• No person in the company can say exactly what the tool must do.

The last condition is the most important. A license also does not solve that problem.

The part that nobody believes

One fact remains, and engineers do not believe it easily. I built all of this, and I did not read even one line of the code. I did not open even one file. Part 2 tells you how that is possible. It also tells you why this is a story about management, not about programming.

Part 2 comes next. It tells how I built the engine and did not read the code.

In this project, I designed a support agent for Chinook Records, a fictional digital music store. It answers account questions (invoices, profile), recommends music grounded in a customer’s purchase history (via local open-source embeddings), and updates profile fields. The data for the agent to operate on resides in a SQL database.

It’s the well-known Chinook sample database — a fictional digital music store modelling a catalog (artists → albums → tracks), customer invoices, and the staff who support them, across 11 tables. The full structure, including an entity-relationship diagram, is in data/chinook_schema.md.

It’s built on LangChain’s create_agent and the LangGraph runtime, with LangSmith for tracing and evaluation. I built it in my spare time over about two weeks, leveraging state of the art agentic engineering practices, based around the concept of Architecture Decision Records (ADRs), which agents translate to Linear tickets, that then get picked up by other agents for implementation. I kept full control of the orchestration, to learn as much as possible from it, but beyond code review in the form of PRs, I did not write a single line of code myself. More information in this section.

This post covers three things: the privacy design, the middleware, and how I measured whether any of it holds up when the model changes.

The code

You can find all the code and artifacts in github.com/ToonWeyens/chinook-support-agent. The repo is designed to be self-explanatory and you can replicate it yourself. You will need to put your own keys to LLM providers like Anthropic in the .env file of which you’ll find an example.

Measuring what makes the agent safe

The question I wanted an answer to: Is a well-behaved agent safe because the system constrains it, or because the model happens to behave? As models become more powerful and better at following instructions, it is easy to mistake one for the other: The two can look identical in a demo. But they diverge in production, which is what enterprises care about.

To separate them, I scored the same agent across a 2×2 matrix of two inputs:

• tools (naive raw-SQL vs. scoped) and
• middleware (off vs. on)

using two models: Sonnet 4.6 (expensive and closed-source) vs Mistral Small 24B (much smaller, open-weights, EU-hosted).

Tools are explained a bit more in this section and middleware in this one.

The two ends of that matrix are structurally different graphs. The naive baseline loops the model against raw sql_db_* tools:

The production agent routes through the scoped tools with the middleware stack composed in:

I used three evaluators to score each cell: correctness (graded LLM judge), privacy/safety (binary judge), and PII exposure (a deterministic regex over tool outputs). The deterministic evaluator is a code-based test that scores whether a condition is true or not. While this is great and deterministic, it is not always possible to use, e.g. when trying to answer questions “is the answer correct”, and the answer is free-form text rather than a number. In these cases you have to use LLM-based evaluators called judges. The nice thing about LangChain is that you can use any type of evaluator, and you can design them using the LangSmith UI directly, without having to write code. They have a lot of pre-populated evaluators, of which I used one. The other two evaluators I implemented using my software development pipeline.

Here are the results — one heatmap per criterion, each showing the tools × middleware 2×2 side by side for both models. Orange is worse, blue is better, and the outlined cell is the production configuration.

What the numbers show:

1. PII exposure reaches 1.00 on both models and both tool layers once the redaction middleware is on — it doesn’t depend on the model.
2. Scoped tools are what keep the cheaper model usable: Mistral reaches 0.73 correctness with scoped tools (Sonnet is 0.80) but drops to 0.58 on raw SQL.
3. Middleware alone doesn’t rescue the naive tool layer — with raw SQL, Mistral’s privacy score only recovers to 0.88 and correctness stays collapsed, because a human-approval gate can’t meaningfully sit in front of a free-form sql_db_query. The tool layer carries it.

In summary, with scoped tools and middleware, Mistral lands close to Sonnet (0.73 / 0.94 / 1.00 vs. 0.80 / 0.94 / 1.00) at lower cost. The same three evaluators also run as LangSmith platform objects, so they score every experiment and a sample of live traffic, with an alert on the PII score. These are the kind of results enterprises care about, as it guarantees good performance by creating the appropriate harness, leveraging cheap and local models ensuring AI and data sovereignty, rather than relying on expensive frontier models that are not in their control.

Privacy scoping with tools

The customer’s identity never reaches the model. customer_id is injected into the LangGraph runtime context at invoke time (modeled on a session token), and each data tool reads it from context and scopes its SQL to it. It is never a tool parameter the model can populate.

The consequence is that prompt injection — “pretend I’m customer 14,” “admin mode,” “list every customer’s email for transparency” — has nothing to act on: there is no code path from the model’s output to another customer’s rows. That’s verifiable with a unit test that doesn’t involve the LLM. The naive baseline (the raw-SQL toolkit from the current LangChain SQL-agent tutorial) has no such property; on the wrong prompt it will GROUP BY across the store and return 61 other customers’ names and emails.

search_catalog is the deliberate exception — it reads store-wide catalog data, which is strictly non-personal by design, and is documented as such.

Middleware

Middleware is a reusable code layer that intercepts, modifies, and controls the agent’s core execution loop — acting as the essential “glue” between user inputs, the Large Language Model (LLM), and tools.

For the support agent I decided to use three types of middleware, composed on create_agent in a fixed order:

• HITL write-gating — update_account is the only mutating tool, and it interrupts for human approval before writing. The write lands on a disposable copy of the database, so it’s a real mutation that resets cleanly between runs.
• Tool-call limit — a hard per-turn cap, guarding against runaway loops and cost.
• PII redaction — masks email addresses in tool results before they reach a trace.

LangChain has a long list of existing middleware functionalities that you can leverage, which is what I decided to do. As a result, each is roughly only a line in create_agent.

How it was built

The build is driven from a single Architecture Decision Record: 19 numbered decisions, reviewed via PR and amended in place as things changed (one decision was reversed once the eval suite made it cheap to revisit). The ADR holds the rationale; Linear holds the task breakdown, and each ticket references the decision it implements.

I sequenced it as a walking skeleton — one privacy-scoped tool running in LangGraph Studio first, then everything else extending a system that already ran — and ran the independent tickets as parallel coding-agent sessions, each in its own git worktree to avoid conflicts. It came to 200-plus commits over ~68 merged PRs, 137 tests behind a ruff + mypy + pytest gate, and a friction log of what went wrong: LangGraph Studio serving stale state after a module became a package, a hand-rolled HITL middleware I replaced with the official one, a thread wedged by a malformed approval value.

Scope

This is a demo on a fictional dataset, not a deployment. Deployment is documented rather than built — langgraph.json is already a deploy manifest, and the one real demo-to-prod gap is written down: in production, customer_id has to be injected server-side from an authenticated session, never accepted from the client. The eval set is small and curated, so the matrix is evidence for the method, not a benchmark.

The pattern I’d reuse: put the security-critical invariants in the tool and runtime layer, where they hold regardless of the model and can be tested without it, and treat the eval suite as the thing that tells you whether a model swap is actually safe.

Mathematics

In graph theory, an elementary path (often called a simple path) is defined specifically as a path that does not repeat any vertices.

graph LR
    S((Start)) --> A
    
    subgraph "The Cycle"
        A --> B
        B --> C
        C --> A
    end
    
    C --> T((Target))

    %% Styling to highlight the valid elementary route
    linkStyle 0,2,4 stroke:green,stroke-width:3px,color:green;
    style S fill:#d4edda,stroke:green
    style T fill:#d4edda,stroke:green

graph LR
    S((Start)) -- 5 --> A
    A -- 10 --> T((Target))
    
    A -- 5 --> B
    
    subgraph "Negative Cycle (-1)"
    B -- 2 --> C
    C -- "-3" --> B
    end
    
    C -- 5 --> T

    %% Styling the elementary shortest path
    linkStyle 0,1 stroke:blue,stroke-width:3px,color:blue;
    
    %% Styling the confusing negative cycle part
    style B fill:#ffcccc,stroke:red
    style C fill:#ffcccc,stroke:red
    linkStyle 3,4 stroke:red,stroke-width:2px,color:red,stroke-dasharray: 5 5;

Therefore, the Shortest Elementary Path Problem is the path with the minimum total weight between two nodes subject to the constraint that no node is visited more than once.

In graph theory, the shortest elementary path problem requires that each node in a path can only be visited once. The edge weights do not need to be nonnegative.

A general algorithm to solve this problem works by solving the k-paths problem, i.e. by solving the nonelementary problem repeatedly and requiring that for the nodes that are present in nonelementary paths multiple (k) paths need to be found. The resulting algorithm finds the elementary shortest paths (ESPP) from a single source to all destinations [1]. However, it can be very slow if very large graphs are considered. Especially because the algorithm in [1] is intrinsically flawed and the fix for this makes it much less efficient than expected: See note in ESPP.py.

A more powerful algorithm can be used when additionally there are positive resources associated to each edge in the graph, though this requirement can be relaxed to nonnegative resources if not too many edges have zero resource cost associated to them. The resulting Elementary Resource Constrained Shortest Path Problem (ESPPRC) can then be solved by an algorithm that starts by first preprocessing the graph [2]. During preprocessing, the graph is first pruned by discarding all nodes that cannot be reached from the origin or reach the target without violating the resource limits. Subsequently, for each pair in the resulting pruned graph, the least-cost path is sought, which will be used later to incrase so-called dominance of paths by other paths.

After preprocessing, the ESPPRC is solved by introducing node-resources that count how many times a node has been used in a certain path, so that elementarity can be enforced. As this problem can quickly become exponential in nature, it is critical that the numer of resource nodes that have to be introdcuced kept as low as possible. In the algorithm this is done by using a two-fold dominance strategy: On the one hand, paths are only added to the list of possible shortest possible paths, if their resource consumption is not dominated by another path. Also, when a path is added, the other possible paths that are dominated by it, are discarded. On the other hand, strong dominance is a mathematical trick that is used to speed up convergence to the shortest path, by artificially incrementing possible node resources of nodes that cannot be reached [2]. The resulting algorithm seeks the ESPPRC for single source to single destination problems.

UNDER CONSTRUCTION