<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
  <title>Soleur Blog</title>
  <subtitle>Insights on agentic engineering and company-as-a-service</subtitle>
  <link href="https://soleur.ai/blog/feed.xml" rel="self" />
  <link href="https://soleur.ai/" />
  <updated>2026-12-03T00:00:00Z</updated>
  <id>https://soleur.ai/</id>
  <author>
    <name>Soleur</name>
  </author>
  <entry>
    <title>Your Alarm Passed Every Rehearsal. In Real Life It Never Rings.</title>
    <link href="https://soleur.ai/blog/alarm-passed-every-rehearsal-never-rings/" />
    <updated>2026-12-03T00:00:00Z</updated>
    <id>https://soleur.ai/blog/alarm-passed-every-rehearsal-never-rings/</id>
    <content type="html">&lt;p&gt;Somewhere in your business there is an alarm you were told works. Maybe it pings you when a nightly job breaks. Maybe it is the notice that fires if a secret key leaks. Someone you trust, a contractor or an AI agent, said the words you wanted to hear: &amp;quot;we tested it.&amp;quot; And it passed.&lt;/p&gt;
&lt;p&gt;Here is the question nobody asks at that moment: could the test have failed?&lt;/p&gt;
&lt;p&gt;For parts of our own alarm system, the honest answer was no. They passed every rehearsal we ran, and in real life they would never have rung. This post is about how that happens, what it would have cost us, and the one question that tells you whether your alarms have the same flaw.&lt;/p&gt;
&lt;h2&gt;Why a rehearsal that cannot fail feels exactly like a rehearsal that passed&lt;/h2&gt;
&lt;p&gt;Picture a note passed down a line of workers. Each worker does a job, then hands the note to the next. If something goes wrong, the note says so, and the right person is supposed to react: stay quiet, retry, or sound the alarm.&lt;/p&gt;
&lt;p&gt;In real life, that hand-off goes through a coordinator that writes the note out again from scratch. The new note keeps the words of the complaint, the trail of what happened, and a short line about the cause. It drops the stamp: what kind of problem this was, and the numeric code that says why a request was refused. Think of a translator who keeps your sentences and strips the letterhead.&lt;/p&gt;
&lt;p&gt;Plenty of alarm logic reads that stamp. If the stamp says &amp;quot;a release is in progress&amp;quot;, stay quiet and try again later. If it says &amp;quot;a secret was detected&amp;quot;, sound the loud alarm. In real life the stamp is gone, so those instructions never match and the note falls through to the boring default.&lt;/p&gt;
&lt;p&gt;In a rehearsal, none of this happens. The practice stage hands the original note straight over, stamp intact. Every instruction matches. Every test goes green. The tests were never wrong about the instructions. They were testing a hand-off that does not exist in real life.&lt;/p&gt;
&lt;p&gt;That is the whole trap. A green result from a practice stage that skips the real hand-off proves nothing about the alarm, and it feels like proof because nothing looks broken.&lt;/p&gt;
&lt;h2&gt;What it would have cost us&lt;/h2&gt;
&lt;p&gt;We found this in three separate places. They are different pieces of the system, and each shows a different way the silence hurts. We did not have a whole fleet of dead alarms. We had three specific defects.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A false alarm, every time a release overlapped.&lt;/strong&gt; Nine scheduled jobs that run on their own are built to notice when a new version of the product is being rolled out and to wait politely instead of failing. That signal travels as exactly the sort of stamp that gets stripped. When a release outlasted the one retry the job allows, the job did not recognize it. It reported a failed setup and sent a failed &amp;quot;I&#39;m alive&amp;quot; signal, the very false alarm the waiting rule exists to prevent. This one was noisy rather than damaging, and no data was lost. But a founder who gets enough false alarms learns to stop reading alerts, which is the more expensive outcome.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A serious alert dressed as a boring one.&lt;/strong&gt; A separate guard watches for leaked credentials. When it found one, the instruction was to take the leak path: loud, urgent, human. But the leak signal was a stamp too. After the hand-off it arrived looking like an ordinary &amp;quot;the guard itself is broken&amp;quot; failure. The most serious message in the system showed up wearing the costume of the most routine one. Our rehearsal showed the guard doing exactly the right thing.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A refusal read as a slow reply.&lt;/strong&gt; A third piece of the product talks to an outside AI service. When that service refused a request outright, for example because the request was malformed and could never succeed, the reason for the refusal was a numeric code. The code was lost at the hand-off, so the program read every refusal as &amp;quot;it timed out&amp;quot;, and the response to a timeout is to try again. A request that could never succeed would be sent three more times. The rehearsal never saw it, because the practice stage kept the code.&lt;/p&gt;
&lt;p&gt;Three places, one shape. In each, the green checkmark was telling the truth about something other than what we cared about.&lt;/p&gt;
&lt;h2&gt;Why we counted instead of fixing them one by one&lt;/h2&gt;
&lt;p&gt;The obvious move was to fix the places we knew about and move on. We did fix them. But finding them one at a time is how the next one slips in. The first two were found in review, not by any alarm. A system that depends on someone happening to notice will eventually miss one.&lt;/p&gt;
&lt;p&gt;So we counted instead. We wrote a lint, which is an automatic checker that reads the code and stops a change from shipping when it finds a pattern we have banned. This one works as a census, a full count of every place in the code that catches an error where a hand-off is involved, and then reads something about it.&lt;/p&gt;
&lt;p&gt;The checker sorts every one of those places into one of two piles.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The error already crossed the hand-off.&lt;/strong&gt; The code is reading the stamp — the label — on a note that has been rewritten. This is always wrong, because the stamp is gone. The checker flags it every time. The only pass it gives is a bare &amp;quot;is this an error at all&amp;quot; check; beyond that, there is no way to excuse a place in this pile.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The error is still live inside the task.&lt;/strong&gt; Here the note has not been passed on yet, so the original is in hand, stamp and all. Reading the stamp is fine, and it is in fact the right place to do it: check the label while you still have it, then pass the verdict along as a plain result. But the checker only allows this in a short, written list of places. Adding a place to that list is a visible, deliberate act. The addition shows up in the change for a reviewer to see and challenge.&lt;/p&gt;
&lt;p&gt;It watches for three kinds of label reading: asking what kind of problem this is, asking what it is named, and asking for its numeric code. If any of those turns up on the wrong side of the hand-off, the change is stopped. The count comes back clean.&lt;/p&gt;
&lt;p&gt;The old way, a careful person had to notice. The new way, nobody has to. The bad pattern cannot come back silently, because coming back means failing a check in plain sight or adding a name to a list in plain sight.&lt;/p&gt;
&lt;h2&gt;The other half: a rehearsal that can finally fail&lt;/h2&gt;
&lt;p&gt;A counter alone is not enough. The rehearsal was the other half of the lie, so we rebuilt the practice stage too.&lt;/p&gt;
&lt;p&gt;The old rehearsal handed errors over directly. The new one hands them over the way the real system does, rebuilding the error with the real system&#39;s own rebuilding code instead of a hand-made imitation. The stamp is stripped in rehearsal the same way it is in life. Now an alarm that depends on a stamp that will not be there fails its test, in the practice room, where failing is cheap.&lt;/p&gt;
&lt;p&gt;The two halves do different jobs. The counter finds every place that reads a label. The honest rehearsal checks that an alarm behaves when the label is gone.&lt;/p&gt;
&lt;p&gt;This is one repair of seven in &lt;a href=&quot;https://soleur.ai/blog/first-we-made-green-mean-something/&quot;&gt;the story of making green mean something&lt;/a&gt;, zoomed in.&lt;/p&gt;
&lt;h2&gt;What this does not prove&lt;/h2&gt;
&lt;p&gt;We would rather tell you the edges than let you round them up.&lt;/p&gt;
&lt;p&gt;The counter knows one tool&#39;s hand-off by name. It only looks at the folder of jobs built on that tool, and it recognizes the hand-off by its literal call. It reads code by pattern, matching shapes of text rather than meaning. It checks three kinds of label reading and nothing else; an alarm that finds some fourth way to be wrong would sail past it.&lt;/p&gt;
&lt;p&gt;And what it proves is narrow. It proves that one particular lie cannot return silently. It does not prove that every alarm is correct. A clean count means that no code reads a stripped stamp. It says nothing about whether the response to a stamp is a good one.&lt;/p&gt;
&lt;p&gt;The trap itself is not unique to our tool. The same trap shows up in other tools in different forms. In our tool the label is dropped. In others, the original arrives intact but tucked underneath a wrapper, so the code that catches the error sees the wrapper first and has to know to look inside. In both, code that expects the original kind of problem can be wrong without any test noticing. We are not saying any tool is broken, or that details are always stripped. We are saying the question is the same wherever work changes hands: queues (lists of waiting work), workflow engines (software that runs a chain of tasks) and calls between services (one program asking another).&lt;/p&gt;
&lt;p&gt;The idea transfers, even if the implementation does not. List every place that catches an error. Sort each one into &amp;quot;the error already crossed a hand-off&amp;quot; or &amp;quot;it is still live&amp;quot;. Make the second pile a short list that people have to argue to join. Your builders&#39; version will look different from ours.&lt;/p&gt;
&lt;h2&gt;What you do with this&lt;/h2&gt;
&lt;p&gt;You do not need to read any code to use it. Ask whoever builds your systems one question:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&amp;quot;Was this alarm ever tested against the real hand-off, and has anyone counted every place an error can be swallowed?&amp;quot;&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;A good answer names the hand-off, says what survives it, and shows the practice stage rebuilt to match. A weak answer is &amp;quot;yes, it&#39;s tested&amp;quot; with nothing about how. An alarm only ever rehearsed on the practice stage is untested for the one case that matters.&lt;/p&gt;
&lt;p&gt;For the technical write-up, see &lt;a href=&quot;https://github.com/jikig-ai/soleur/pull/11278&quot;&gt;how we built the check and moved the tests onto an honest rehearsal&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;And if you are the one person running the whole company, with no time to audit every alarm yourself, that is the problem Soleur exists for. Your AI team checks its own work, writes down what it learns, and gets better at telling a real pass from a rehearsal. &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;Join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;Can an alert pass every test and still never fire?&lt;/h3&gt;
&lt;p&gt;Yes. If the test practices on a stage that skips the real hand-off between workers, the error keeps its label in the test and loses it in real life. The alert reads a label that is no longer there, so it stays silent or fires the wrong message. The green result was true about the rehearsal and false about the real thing.&lt;/p&gt;
&lt;h3&gt;What does it mean for an error to cross a hand-off?&lt;/h3&gt;
&lt;p&gt;Work in software is often split into small tasks passed from one worker to the next. When a task fails, the error is usually rewritten before the next worker sees it. The message survives, but details such as the kind of problem and its numeric code may not. An error that has crossed the hand-off is the rewritten one, and code that reads the original details from it is reading something that is gone.&lt;/p&gt;
&lt;h3&gt;Does this problem only affect one tool?&lt;/h3&gt;
&lt;p&gt;No, although it looks different in different tools. In our tool the label is dropped. In others, the original error arrives intact but wrapped, so the code that catches it sees the wrapper first. The common thread is that wherever work changes hands, the code that catches an error may not be looking at the original, and a test that skips the hand-off will not show it.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Can an alert pass every test and still never fire?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes. If the test practices on a stage that skips the real hand-off between workers, the error keeps its label in the test and loses it in real life. The alert reads a label that is no longer there, so it stays silent or fires the wrong message. The green result was true about the rehearsal and false about the real thing.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What does it mean for an error to cross a hand-off?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Work in software is often split into small tasks passed from one worker to the next. When a task fails, the error is usually rewritten before the next worker sees it. The message survives, but details such as the kind of problem and its numeric code may not. An error that has crossed the hand-off is the rewritten one, and code that reads the original details from it is reading something that is gone.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Does this problem only affect one tool?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No, although it looks different in different tools. In our tool the label is dropped. In others, the original error arrives intact but wrapped, so the code that catches it sees the wrapper first. The common thread is that wherever work changes hands, the code that catches an error may not be looking at the original, and a test that skips the hand-off will not show it.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>First We Made &#39;Green&#39; Mean Something. Then We Made It Fast.</title>
    <link href="https://soleur.ai/blog/first-we-made-green-mean-something/" />
    <updated>2026-11-19T00:00:00Z</updated>
    <id>https://soleur.ai/blog/first-we-made-green-mean-something/</id>
    <content type="html">&lt;p&gt;If you use AI tools to run parts of your business, a green check mark is the bargain you made. You stopped reading every line of what your tools produce because something promised to check it for you. Whatever that something is — a test suite, a review agent, a health monitor — its verdict is the thing your week quietly rests on.&lt;/p&gt;
&lt;p&gt;So here is a question worth five minutes: what does that green actually prove?&lt;/p&gt;
&lt;p&gt;We had to answer it ourselves. Our company runs on nearly 700 automated checks that judge every change before it ships — and when we looked closely at our own scoreboard, we found it had been lying to us. Not once. At least four different ways. This is the story of what it took to make green honest — and why the flagship speed change was sequenced last of all.&lt;/p&gt;
&lt;h2&gt;Green was lying, at least four ways&lt;/h2&gt;
&lt;p&gt;Every founder has learned the hard way that &amp;quot;the check passed&amp;quot; and &amp;quot;the thing works&amp;quot; are two different sentences. Ours had drifted apart in four specific, countable places.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A skipped check counted as a pass.&lt;/strong&gt; Some checks can&#39;t run everywhere — they need a tool that isn&#39;t installed, or a fixture the environment doesn&#39;t have. Reporting &amp;quot;skipped&amp;quot; was honest. The scoreboard read it as &amp;quot;passed.&amp;quot; A check that never ran was padding the same column as a check that proved something. Green meant &amp;quot;nothing admitted to failing,&amp;quot; which is a much weaker claim than &amp;quot;everything verified.&amp;quot;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A whole directory of checks was invisible to the finder.&lt;/strong&gt; We built a tool whose entire job is finding checks that nothing runs anymore — orphaned work, quietly rotting. It had a blind spot the size of a directory. Dozens of checks existed — and the tool that was supposed to know about them had never counted them. You can&#39;t trust a census that doesn&#39;t know what it missed.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The map forgot a guard the moment its file was deleted.&lt;/strong&gt; Our runner keeps a map of which checks a change actually needs — the thing that decides a copy edit doesn&#39;t deserve a database audit. The map dropped an entry the moment its file disappeared — so a change that &lt;em&gt;removed&lt;/em&gt; a guarded file never ran the check that watched it. Delete the thing a check protected and the check simply didn&#39;t happen: the safest-looking green a diff can produce.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Nobody knew what green was worth.&lt;/strong&gt; Worst of all: we kept a list of checks that were already failing, treated as known-bad, so the scoreboard could stay green anyway. A &amp;quot;pass&amp;quot; had come to mean &amp;quot;nothing &lt;em&gt;new&lt;/em&gt; is broken — we think.&amp;quot; The baseline of excuses had become invisible. When your standard for success is &amp;quot;no worse than the mess we stopped counting,&amp;quot; green is a shrug.&lt;/p&gt;
&lt;h2&gt;The rest of the dishonesty ledger&lt;/h2&gt;
&lt;p&gt;Those four were the flagship lies. There were three more ways the verdict was weaker than it looked — and six of the seven repairs landed before the first speed change shipped.&lt;/p&gt;
&lt;p&gt;A check that fails one run in five teaches everyone to press &amp;quot;re-run&amp;quot; until it agrees — and a check cut short could file an ordinary-looking verdict, even a false alarm, instead of admitting it never finished. Flaky checks don&#39;t just waste time; they train the whole team to treat red as weather instead of information.&lt;/p&gt;
&lt;p&gt;A throwaway environment we build to run a check safely could, in one path, survive into the real one — a rehearsal space bleeding into the actual stage. The check wasn&#39;t just lying about the verdict; it was contaminating what it measured.&lt;/p&gt;
&lt;p&gt;And an error handler could pass every rehearsal and never fire in production. When work crosses a boundary — one system handing off to another — an error arrives rebuilt, stripped of everything the handler was watching for. So we built a census that counts every place an error can be swallowed crossing that line. A handler that only works in practice is not a handler. The full story of that census — what the swallowed errors cost and how counting every catch site works — is told in &lt;a href=&quot;https://soleur.ai/blog/alarm-passed-every-rehearsal-never-rings/&quot;&gt;Your Alarm Passed Every Rehearsal. In Real Life It Never Rings.&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Seven repairs to what a verdict &lt;em&gt;means&lt;/em&gt;. Six landed before any speed change shipped — and the flagship speed change was gated to land last of all.&lt;/p&gt;
&lt;h2&gt;Only then, speed&lt;/h2&gt;
&lt;p&gt;We could have led with the fast part. Speed is the change everyone can feel, and &amp;quot;we made the checks faster&amp;quot; is an easy thing to say. The flagship change — running checks side by side — was gated to land last on purpose, and only after the rebuilt scoreboard had vouched for every repair before it: speed on top of a dishonest verdict is a faster lie.&lt;/p&gt;
&lt;p&gt;With the scoreboard rebuilt, five changes earned their place. The runner now declines checks the change couldn&#39;t have affected — a copy fix no longer triggers the full hardware audit. Runs split across more workers. A cap keeps a small change from dragging the whole nearly-700-check battery in behind it. And when you ask for it, checks can run side by side instead of one after another: in a controlled experiment on a small fixed set, the opt-in path measured &lt;strong&gt;1.88 times faster&lt;/strong&gt;. The default stays exactly what it was — every founder and every release sees the same serial, boring, trustworthy order unless they choose otherwise.&lt;/p&gt;
&lt;h2&gt;The part that lives in no single fix&lt;/h2&gt;
&lt;p&gt;Here is the detail that makes this story more than a list of repairs: the scoreboard we were fixing was also the thing judging every fix.&lt;/p&gt;
&lt;p&gt;Twelve consecutive changes rebuilt the accounting, the map, the boundaries, and the scheduler. Each one was judged by the very suite it was repairing. The arbiter adjudicated its own rebuild — twelve merges, zero broken mains. A bootstrap problem with a working answer: fix the measuring stick while the measuring stick is checking you.&lt;/p&gt;
&lt;p&gt;This is also why the work compounds instead of repeating. Every repaired lie is now a permanent, automatic check — a ratchet that can only tighten. The next &amp;quot;skipped counts as passed&amp;quot; doesn&#39;t get a second chance, because the scoreboard itself now guards the counting. We&#39;ve written before about turning hard-won lessons into standing rules; extending that habit to the suite that judges everything else was the obvious next turn of the same ratchet. The pattern set out in &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;our last post on the theme&lt;/a&gt; — catch the lesson, write it into the machinery — now covers the machinery&#39;s own verdicts.&lt;/p&gt;
&lt;h2&gt;The honest limits&lt;/h2&gt;
&lt;p&gt;The caveat is part of the story, so we are telling it plainly.&lt;/p&gt;
&lt;p&gt;One repair is still earning its verdict. The change that makes the runner decline irrelevant checks has a soak test running in the background — a probe that watches real runs accumulate and asks whether the savings are real and nothing broke through. It has not answered yet, and we are deliberately not closing the question early. If it comes back negative, that is the machinery working exactly as intended: an honest &amp;quot;not yet&amp;quot; instead of a convenient &amp;quot;sure.&amp;quot;&lt;/p&gt;
&lt;p&gt;The speed claim is bounded the same way. 1.88× is what we measured on a small, controlled set of checks, on the opt-in path. It is not a promise about your workload, and we would rather tell you the shape of the evidence than round it into marketing.&lt;/p&gt;
&lt;h2&gt;The lesson for the founder&lt;/h2&gt;
&lt;p&gt;You do not need to know how any of this works to use the lesson. The order is the whole thing:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Ask what green means before you ask how fast it runs.&lt;/strong&gt; Whatever checks your business leans on — a test suite, a review agent, a report that says &amp;quot;all good&amp;quot; — its verdict is only worth what its accounting is worth. A fast answer you can&#39;t trust is just a quicker way to be wrong.&lt;/p&gt;
&lt;p&gt;We made ours mean something first. Then we made it fast.&lt;/p&gt;
&lt;p&gt;For the technical write-up — the full ledger of what was lying and how each repair was verified — see the &lt;a href=&quot;https://github.com/jikig-ai/soleur/issues/10971&quot;&gt;public engineering record on GitHub&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;And if the reason this resonates is that you are running the whole company yourself: that is the problem Soleur exists for. Your AI team checks its own work, writes down what it learns, and gets measurably better — so the green you see means what you think it means. &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;Join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What does a green check mark actually prove?&lt;/h3&gt;
&lt;p&gt;Only what its accounting is worth. A green verdict means the checks that ran reported no failures — but that can hide checks that never ran, checks that were counted wrong, or known failures quietly excused. Before trusting a green, it is worth asking what &amp;quot;passed&amp;quot; and &amp;quot;skipped&amp;quot; actually counted for.&lt;/p&gt;
&lt;h3&gt;Can an automated check itself be wrong?&lt;/h3&gt;
&lt;p&gt;Yes — in specific, enumerable ways. Ours had at least four: a skipped check counted as a pass, a finder that could not see a whole directory of checks, a map that forgot a guard the moment its file was deleted, and a baseline that made green mean &amp;quot;no worse than the mess we stopped counting.&amp;quot; Every kind of check machinery has an equivalent list worth auditing.&lt;/p&gt;
&lt;h3&gt;Should speed come before accuracy in automated checks?&lt;/h3&gt;
&lt;p&gt;No — and the order matters more than either alone. Speed on top of a dishonest verdict is a faster lie: the same wrong answer, delivered sooner. Fix what the verdict means first, make it fast second, and keep the honest path the default even after a faster one exists.&lt;/p&gt;
&lt;h3&gt;How do I know whether to trust my AI tool&#39;s results?&lt;/h3&gt;
&lt;p&gt;Ask what the tool does when it is uncertain, and whether its record is auditable. Trust signals to look for: it says &amp;quot;not yet&amp;quot; when evidence is thin rather than guessing, its past claims can be checked against what actually happened, and the rules it learned from its own mistakes are written down where you can see them.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What does a green check mark actually prove?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Only what its accounting is worth. A green verdict means the checks that ran reported no failures — but that can hide checks that never ran, checks that were counted wrong, or known failures quietly excused. Before trusting a green, it is worth asking what &#92;&quot;passed&#92;&quot; and &#92;&quot;skipped&#92;&quot; actually counted for.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Can an automated check itself be wrong?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes — in specific, enumerable ways. Ours had at least four: a skipped check counted as a pass, a finder that could not see a whole directory of checks, a map that forgot a guard the moment its file was deleted, and a baseline that made green mean &#92;&quot;no worse than the mess we stopped counting.&#92;&quot; Every kind of check machinery has an equivalent list worth auditing.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Should speed come before accuracy in automated checks?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No — and the order matters more than either alone. Speed on top of a dishonest verdict is a faster lie: the same wrong answer, delivered sooner. Fix what the verdict means first, make it fast second, and keep the honest path the default even after a faster one exists.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How do I know whether to trust my AI tool&#39;s results?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Ask what the tool does when it is uncertain, and whether its record is auditable. Trust signals to look for: it says &#92;&quot;not yet&#92;&quot; when evidence is thin rather than guessing, its past claims can be checked against what actually happened, and the rules it learned from its own mistakes are written down where you can see them.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>A Flaky Test Passed on Retry and the Loop Called It Fixed</title>
    <link href="https://soleur.ai/blog/flaky-test-passed-on-retry-quarantine/" />
    <updated>2026-11-17T00:00:00Z</updated>
    <id>https://soleur.ai/blog/flaky-test-passed-on-retry-quarantine/</id>
    <content type="html">&lt;p&gt;Your AI produces work faster than you can check it, so checks do the checking for you. That arrangement rests on one condition nobody talks about: the check has to mean something. A check that fails on Monday and passes on Tuesday over the same unchanged work is not a weaker check. It is a different object entirely — and inside an AI-driven workflow it does damage a missing check never could.&lt;/p&gt;
&lt;p&gt;This is the story of what we shipped about that, inside the automated fix loop our users&#39; workflows run, and the two ways review caught the machinery lying. It matters even if you never look at a check yourself, because the same question applies to every green light any tool shows you: what does a pass actually prove?&lt;/p&gt;
&lt;h2&gt;The failure mode only an AI learns&lt;/h2&gt;
&lt;p&gt;A check that fails sometimes and passes other times, with nothing changed in between, has a name: flaky. A human engineer watches a failure vanish on re-run and gets suspicious — something underneath is unreliable, and the suspicion is the correct response.&lt;/p&gt;
&lt;p&gt;An AI agent learns the opposite lesson. Failed? Run it again. Green this time? Move on. Every pass on retry is a data point the loop files away: red is negotiable, and persistence is a fix. The agent is not being lazy — it is being rational, responding to exactly the evidence it was given. The evidence is the problem.&lt;/p&gt;
&lt;p&gt;That is why a flaky check left running is worse than no check at all. No check means everyone knows the work is unguarded. A flaky check means the whole system is being trained, one retry at a time, that failures clear themselves if you wait long enough. When we built the fix loop — the machinery that takes a set of failing checks, diagnoses what broke, and iterates until the work passes — this was the one behavior it could never be allowed to learn.&lt;/p&gt;
&lt;h2&gt;One re-run, then quarantine&lt;/h2&gt;
&lt;p&gt;The rule that shipped is deliberately unglamorous: every newly failed check gets exactly one isolated re-run, on the unchanged work. Not a retry loop. One re-run, alone, with nothing fixed first.&lt;/p&gt;
&lt;p&gt;A pass on that re-run is not a fix. It is a flake signature — proof the check fails without a cause — and it triggers three acts.&lt;/p&gt;
&lt;p&gt;Quarantine first. The check comes off the failure list driving the fix work for that run. It is never counted as fixed — no fix was attempted — and it can never trip the loop&#39;s gone-in-circles detector, because it spends no fix budget at all. Quarantine is bookkeeping, not surgery: the check itself is never edited, skipped, or deleted. The bookkeeping is also precise about identity — the same check name in two different files counts as two separate suspects, not one flake.&lt;/p&gt;
&lt;p&gt;File second. A tracker ticket is opened so the flake is owned, visible, and fixable later. The deduplication detail matters: rather than searching for a ticket whose title matches, the loop matches a small invisible marker written inside the ticket itself. A title search silently misses exactly the tickets it exists to find — check names carry punctuation that confuses matching-by-name — and a miss files a duplicate nobody reconciles. A found ticket gets a comment, not a clone. And a ticket that stays open forever trains everyone to ignore the label, so the convention closes it on the next green run of the same suite.&lt;/p&gt;
&lt;p&gt;Report third. The final report must name every quarantined flake — &amp;quot;succeeded, with three checks quarantined and tracked&amp;quot; — never a bare green line. A green that could quietly cover quarantined flakes is the precise failure the screen exists to prevent.&lt;/p&gt;
&lt;p&gt;Two honest boundaries deserve plain statement. A re-run that errors or times out does not count as a pass — the failure stays on the fix path. And quarantine must never launder a real regression the change itself caused: a flake first observed after the loop&#39;s own fix landed, or a check that exercises the files the change touched, is flagged as a possible interaction with the change, not waved through.&lt;/p&gt;
&lt;h2&gt;What review caught&lt;/h2&gt;
&lt;p&gt;The screen went through the same review gauntlet everything here goes through — and review found two defects that matter more than the screen itself.&lt;/p&gt;
&lt;p&gt;The first was a success row that proved nothing. The protocol&#39;s outcome table needed a line meaning &amp;quot;everything that failed turned out to be a flake.&amp;quot; The case had been missing entirely at first — an all-flake run matched no outcome at all and would have been reported as a stall, throwing away real fixes. The row added to cover it then fired on any run that ended with an empty failure list — including a run that produced no readable results whatsoever. Zero quarantines read as green. The fix that shipped requires at least one real quarantine before that row can fire: nothing screened is nothing proven, and a run nobody could read falls through to the failure rows where it belongs.&lt;/p&gt;
&lt;p&gt;The second was a flag that could never fire. The protocol carried a warning meant to catch quarantine laundering a regression — it compared the flaked check&#39;s file against the files the loop&#39;s own changes touched. Reasonable-sounding, and empty by construction: the loop never edits check files, so the comparison could never be true. A warning whose trigger cannot fire is decoration wearing the costume of coverage. The shipped version flags the two signals that can actually fire: the flake was first seen after the loop&#39;s own fix, or the flaked check exercises files the change touched.&lt;/p&gt;
&lt;p&gt;There is a third lesson in the same review, smaller and worth keeping. The first draft of the ticket-deduplication step invented a name-based search — while the codebase already had the marker-inside-the-ticket convention, proven against exactly the inputs that break a name search. When the machinery you need already exists nearby, mirror it; an invented mechanism tends to be weaker on precisely the cases it exists for.&lt;/p&gt;
&lt;h2&gt;Six questions to ask of any suite&lt;/h2&gt;
&lt;p&gt;When a filed flake needs a root cause, the protocol points at a checklist — six axes, and most flakes resolve to exactly one. A founder can audit any suite against them without touching the machinery:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Seeds&lt;/strong&gt; — is randomness reproducible? A check that rolls dice fails on inputs nobody can replay. Every random source needs a fixed seed so a red run can be repeated exactly.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clocks&lt;/strong&gt; — does anything read wall-clock time? That check is unpredictable by construction. Freeze time, and never demand exact equality on a measured quantity — a real clock can tick between two readings.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Network&lt;/strong&gt; — does a check reach the real network? Latency and vendor hiccups make it a flake farm. Stand-ins must fail loudly on an unexpected call, never quietly return a default that passes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;State isolation&lt;/strong&gt; — does every check leave the world as it found it? Shared leftovers between checks are order-dependence waiting for a scheduler to expose it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Output terseness&lt;/strong&gt; — quiet on green, loud on red. A chatty suite buries the failure line in noise, and anything reading the summary needs the last line to be the verdict.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Contention&lt;/strong&gt; — the one axis an isolated re-run cannot see: a check that fails only under load, when siblings race for the same resource. For that class the honest fix is determinism work on the check itself, not another retry.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This post is a spoke of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the four days we spent shipping a trust checklist&lt;/a&gt; — the arc&#39;s day-by-day account.&lt;/p&gt;
&lt;h2&gt;What a pass is worth&lt;/h2&gt;
&lt;p&gt;You never have to run a check to use this. The next time a tool hands you a green light — a verified badge, a fixed ticket, an all-clear — ask the question this whole story hangs on: did it pass, or did it pass eventually? A pass on retry is not a pass. It is a flake asking to be quarantined, tracked, and fixed properly — and a system that reports it as fixed is teaching itself that red was negotiable all along.&lt;/p&gt;
&lt;p&gt;For the full technical checklist — the six axes, the quarantine protocol, and what each signature means — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/plugins/soleur/skills/test-fix-loop/references/determinism-checklist.md&quot;&gt;the determinism reference on GitHub&lt;/a&gt;. And if this is the discipline you want wrapped around the AI work in your own company, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;Isn&#39;t retrying a flaky test harmless?&lt;/h3&gt;
&lt;p&gt;No — a pass on retry teaches the system that red is negotiable. An AI agent does not get suspicious when a failure vanishes on re-run; it learns that persistence is a fix. A flaky check left running is worse than no check, because it quietly trains the whole workflow to retry until green instead of fixing until green.&lt;/p&gt;
&lt;h3&gt;What does quarantine actually do to the test?&lt;/h3&gt;
&lt;p&gt;Nothing to the test itself — it is never edited, skipped, or deleted. Quarantine removes it from the failure list driving the fix work for that run, opens a tracker ticket so the flake is owned and visible, and makes the final report name it. The check is never counted as fixed, and the tracker closes on the next green run so the label never becomes noise.&lt;/p&gt;
&lt;h3&gt;How do I make my suite less flaky in the first place?&lt;/h3&gt;
&lt;p&gt;Audit it against six axes: seeds (is randomness reproducible), clocks (is time frozen), network (are real calls blocked), state isolation (does every check clean up after itself), output terseness (quiet on green, loud on red), and contention (does it fail only under load). Most flakes resolve to exactly one of the six.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Isn&#39;t retrying a flaky test harmless?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No — a pass on retry teaches the system that red is negotiable. An AI agent does not get suspicious when a failure vanishes on re-run; it learns that persistence is a fix. A flaky check left running is worse than no check, because it quietly trains the whole workflow to retry until green instead of fixing until green.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What does quarantine actually do to the test?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Nothing to the test itself — it is never edited, skipped, or deleted. Quarantine removes it from the failure list driving the fix work for that run, opens a tracker ticket so the flake is owned and visible, and makes the final report name it. The check is never counted as fixed, and the tracker closes on the next green run so the label never becomes noise.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How do I make my suite less flaky in the first place?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Audit it against six axes: seeds (is randomness reproducible), clocks (is time frozen), network (are real calls blocked), state isolation (does every check clean up after itself), output terseness (quiet on green, loud on red), and contention (does it fail only under load). Most flakes resolve to exactly one of the six.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>The Map That Only Records What the Agent Saw</title>
    <link href="https://soleur.ai/blog/map-that-only-records-what-the-agent-saw/" />
    <updated>2026-11-12T00:00:00Z</updated>
    <id>https://soleur.ai/blog/map-that-only-records-what-the-agent-saw/</id>
    <content type="html">&lt;p&gt;Ask your AI to check whether your product still works, and it will happily start clicking. Ask it again next week and it starts from zero — re-deriving how your pages fit together, guessing which menu leads where, discarding everything it worked out the moment the session ends. That is not testing. It is tourism.&lt;/p&gt;
&lt;p&gt;The fix is a map: a written inventory of what your product does from the user&#39;s side of the screen, so the agent navigates instead of wandering. But a map an agent writes carries a failure no paper map ever had — it can describe, in confident detail, places the author never visited. This is the story of the convention we wrote to make that impossible, and of the review that caught the check that was supposed to protect it.&lt;/p&gt;
&lt;h2&gt;An agent without a map pays for discovery on every run&lt;/h2&gt;
&lt;p&gt;Without a map, every verification run burns its effort twice over: first discovering that a billing page exists, then discovering how a real user reaches it. Worse, the discovery is ephemeral. Whatever the agent worked out about your navigation evaporates at session end, so the next run guesses again — and a lucky guess is indistinguishable from a verified route.&lt;/p&gt;
&lt;p&gt;The flailing has a signature. The agent mistakes a wrong turn for a missing feature. It takes a shortcut no customer could take and reports a page healthy that a signed-out visitor never sees. It spends its budget re-deriving the shape of your product instead of checking the product itself — and on a solo founder&#39;s budget, that re-derivation is the difference between a check that finished and one that ran out of room. Keeping the map — the thing every run was rebuilding and then throwing away — is the fix, and it is what turns wandering into navigation.&lt;/p&gt;
&lt;h2&gt;One file, one home, one entry per reachable page&lt;/h2&gt;
&lt;p&gt;The convention we shipped is a contract in three parts. The first is the artifact itself: a single map file that lives with the project at one pinned location inside its documentation, so every agent that needs it looks in exactly one place — and a missing map is unambiguous, not merely somewhere else.&lt;/p&gt;
&lt;p&gt;The second is the entry shape. Each reachable page gets one entry written in a fixed grammar: how a real user gets there, whether a menu path or a direct link; the stable on-screen landmarks an agent can check for; what the user should find on arrival; whether a sign-in is needed; and the traps that make a naive check wrong — a table that fills in a beat late, a banner that only appears some days. Every field is written from the user&#39;s side of the screen — what a person reaches and what they see — never from the machinery underneath, because the map exists to steer a check that pretends to be a user. A short section at the end lists the pages deliberately left unmapped, each with a written reason, so &amp;quot;not in the map&amp;quot; never silently means &amp;quot;forgotten.&amp;quot;&lt;/p&gt;
&lt;p&gt;The third part is the pair of rules that decides what the map may claim — and it is the reason the document exists at all.&lt;/p&gt;
&lt;h2&gt;A map may only record what the agent actually saw&lt;/h2&gt;
&lt;p&gt;Here is the failure the convention exists to prevent. An agent asked for a map can produce one — every page, every landmark, every expectation — for routes it has never once visited. The invented entries read exactly like earned ones. Only they are fiction: landmarks that may not exist, outcomes nobody verified, confidence with no walk behind it. A fabricated entry is worse than a missing one, because the next agent navigates straight toward anchors that were never there.&lt;/p&gt;
&lt;p&gt;So the contract draws its line where trust lives: the map may only seed the routes a run actually exercised. No visit, no entry. The agent may still enumerate what exists — a skeleton of pages derived from the product itself — but the fields that make an entry worth believing come only from observed navigation. And if a run exercised nothing at all, no map is written. An empty walk produces no record. A vacuous map certifies nothing.&lt;/p&gt;
&lt;p&gt;That is a harder discipline than &amp;quot;document everything,&amp;quot; and deliberately so. It accepts a partial map — the map is authoritative per page, never a reason to skip a page it does not cover — because a smaller truth beats a larger fiction. What grows the map is use: each real run appends the pages it genuinely walked.&lt;/p&gt;
&lt;h2&gt;Only the fields a run contradicted get corrected&lt;/h2&gt;
&lt;p&gt;A map you cannot annotate is a map you will stop trusting. You will add your own notes — this banner is a deliberate experiment, that landmark is scheduled to change — and a wholesale rewrite would erase them on the next run. So updates are field-scoped: a run corrects only the fields it directly contradicted, and appends entries for newly visited pages. Everything else — including every note you wrote — stays exactly as it was.&lt;/p&gt;
&lt;p&gt;And the guard that keeps the map honest is not a separate checker. It is the update contract itself: a change that adds, renames, or removes a page carries its map edit in the same change, and the map states that rule in its own header, so whoever reads the artifact sees the deal that protects it. Presence, not conformance — an agent that relies on the map looks for it at the one pinned place and reads what is there; the contract, not a gate, is what keeps it true. Teams that want the contract enforced mechanically can add an automated check, and the document sketches what one verifies — the harder version of that story, where we built such a check for our own product&#39;s map and found four ways it could lie, is the companion post&#39;s to tell: &lt;a href=&quot;https://soleur.ai/blog/map-that-cant-drift/&quot;&gt;a map that cannot drift&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;The review that caught the checker&lt;/h2&gt;
&lt;p&gt;The convention shipped through the same review gauntlet as everything else in this arc — which means its own verification was audited too, and the audit caught a lie.&lt;/p&gt;
&lt;p&gt;To prove the map document was properly linked where the system could find it, the plan prescribed a control: on a scratch copy, remove the link, and watch the reachability check turn red. It could not. The check did not look for the link at all — it counted the document&#39;s name appearing anywhere in the surrounding text, and after both links were deleted, the name survived in five other mentions. The control reported green either way. A test that could never fail had certified the link.&lt;/p&gt;
&lt;p&gt;The person running the control and the reviewer re-verifying it hit the same wall independently, and each had to derive the faithful version alone: remove every occurrence of the name — the thing the check actually reads — and only then watch it go red. The rule that came out of it generalizes well past this map: a negative control must remove what the check reads, not what the change happened to add.&lt;/p&gt;
&lt;h2&gt;What this is really about&lt;/h2&gt;
&lt;p&gt;The map is one thread of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the four days we spent shipping a trust checklist&lt;/a&gt; — the day-by-day account this story is a spoke of. But the rule underneath it applies to any document an AI writes for a later reader, human or machine.&lt;/p&gt;
&lt;p&gt;A record is trustworthy in proportion to how hard it is to fabricate. Your AI can draft anything; the question is what it is permitted to assert. Ask it for a map of the whole product and it will oblige — plausibility is its job. The convention&#39;s answer is to narrow what &amp;quot;obliging&amp;quot; is allowed to mean: a map that records only what the agent saw, and corrects only what a run contradicted, cannot claim a visit that never happened. That constraint, not the document&#39;s format, is what makes the next run&#39;s navigation worth trusting — and the same constraint is worth demanding of any report, summary, or inventory your AI hands you: show me the walk, not the confidence.&lt;/p&gt;
&lt;p&gt;For the convention itself — the entry grammar, the seed contract, the field-scoped refresh — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/plugins/soleur/skills/test-browser/references/feature-map.md&quot;&gt;the shipped document on GitHub&lt;/a&gt;. And if you want this kind of discipline wrapped around the AI work in your own company, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;Why can&#39;t the agent write the whole map at once?&lt;/h3&gt;
&lt;p&gt;It can draft the skeleton — the list of pages that exist. What it cannot do honestly is fill in the details for a page it never visited: the landmarks to check, what the user should find, the traps that break a naive check. Those fields are the map&#39;s value, and invented ones send the next agent hunting for things that were never there.&lt;/p&gt;
&lt;h3&gt;What happens to notes I add to the map?&lt;/h3&gt;
&lt;p&gt;They survive. Updates are field-scoped: a run rewrites only the fields it directly contradicted and appends entries for newly visited pages. Untouched entries — and every annotation you added — stay exactly as you left them.&lt;/p&gt;
&lt;h3&gt;Where does the map live in my project?&lt;/h3&gt;
&lt;p&gt;In a single file at one pinned location inside the project&#39;s documentation folder — one place, so every tool that needs the map knows where to look. It is working material for your AI, not a page you publish for customers.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Why can&#39;t the agent write the whole map at once?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;It can draft the skeleton — the list of pages that exist. What it cannot do honestly is fill in the details for a page it never visited: the landmarks to check, what the user should find, the traps that break a naive check. Those fields are the map&#39;s value, and invented ones send the next agent hunting for things that were never there.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What happens to notes I add to the map?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;They survive. Updates are field-scoped: a run rewrites only the fields it directly contradicted and appends entries for newly visited pages. Untouched entries — and every annotation you added — stay exactly as you left them.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Where does the map live in my project?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;In a single file at one pinned location inside the project&#39;s documentation folder — one place, so every tool that needs the map knows where to look. It is working material for your AI, not a page you publish for customers.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Test the Migration, Not Just the Schema</title>
    <link href="https://soleur.ai/blog/test-the-migration-not-just-the-schema/" />
    <updated>2026-11-10T00:00:00Z</updated>
    <id>https://soleur.ai/blog/test-the-migration-not-just-the-schema/</id>
    <content type="html">&lt;p&gt;Every founder who ships software eventually replaces a piece of it that already works. The rewrite is faster, cleaner, built on everything the first version taught you. Its tests pass. You flip the switch. And somewhere downstream, in a place the new system&#39;s tests never looked, something your users depended on quietly stops.&lt;/p&gt;
&lt;p&gt;We have a measured instance of exactly that failure — one that ran undetected for about three weeks inside our own product. This is the story of what it taught us, and the practice we shipped so that an AI planning your next replacement demands better evidence than &amp;quot;it applied cleanly.&amp;quot;&lt;/p&gt;
&lt;h2&gt;The change that applied cleanly&lt;/h2&gt;
&lt;p&gt;Our product&#39;s chat feature broke for about three weeks, and nothing looked broken.&lt;/p&gt;
&lt;p&gt;The cause was a routine database change — the kind that tightens an access rule so every saved record carries the workspace it belongs to. The change applied perfectly. Every check that watches the database said so. What it also did was make one field required, and every place where a person&#39;s typed message got saved omitted that field. Each save was refused — silently, behind a generic &amp;quot;an unexpected error occurred&amp;quot; bubble.&lt;/p&gt;
&lt;p&gt;Here is the part that should worry anyone who runs software alone: the background jobs kept working. Automated writers ran under a different access level that bypassed the rule entirely, so data kept flowing, monitors stayed green, and the product looked healthy — while every human who typed a message hit a wall. The last successful message had been saved weeks before anyone noticed.&lt;/p&gt;
&lt;p&gt;When a user finally reported it, the first fix repaired the wrong thing — the error the monitoring alert happened to name, not the failure the user actually hit. The real defect surfaced only when someone traced the visible symptom to its actual producer.&lt;/p&gt;
&lt;p&gt;The lesson underneath is older than our incident. Your users never relied on the specification — the document that says what the system should do. They relied on what the old thing actually did. A change that applies cleanly proves the change landed. It says nothing about whether everything downstream of it still works.&lt;/p&gt;
&lt;h2&gt;The old system is the referee&lt;/h2&gt;
&lt;p&gt;The discipline that answers this is called differential testing, and its premise is almost insultingly simple: the old system is the oracle. You already own a correct-by-definition reference — the thing that has been running in production, producing the behavior everyone depends on.&lt;/p&gt;
&lt;p&gt;The recipe: feed the old system and the new system the same inputs. Normalize away the differences a human has declared meaningless — timestamps, ordering that carries no information, formatting noise. Then compare the outputs, in both directions, and account for every single difference.&lt;/p&gt;
&lt;p&gt;Every difference is exactly one of two things: a bug in the new system, or an intended change — and an intended change must be written down in a ledger, with a reason. There is no third option. A difference with no ledger entry is a bug by definition. That ledger is what keeps &amp;quot;different&amp;quot; from quietly becoming an acceptable state — and when the migration ends, it doubles as your list of breaking changes, written for free.&lt;/p&gt;
&lt;p&gt;The inputs matter as much as the comparison. Hand-picked cases miss the edges a rewrite actually breaks on, so the recipe draws from two sources: randomly generated inputs across the whole domain — the same practice that caught a real bug the first day we adopted it — and samples of real traffic, recorded from the running system and replayed against both sides. Real traffic carries the long tail nobody invents: strange encodings, legacy clients, inputs nobody remembered were possible. The samples must be recorded, never reconstructed — a reconstructed input is a guess about what production sends.&lt;/p&gt;
&lt;p&gt;Two traps sit on either side of the normalization step. Under-normalize and you drown in meaningless noise; over-normalize and every category you wave away becomes a category where real divergence goes invisible. The rule: normalize only what a human explicitly declared non-semantic, and keep that list where a reviewer can audit it.&lt;/p&gt;
&lt;p&gt;And compare the whole output set, as a one-to-one mapping. &amp;quot;Every new output matches some old output&amp;quot; leaves deletions invisible — a behavior the new system simply dropped reads as clean under that weaker check. Nothing added without a ledger entry; nothing removed without one either.&lt;/p&gt;
&lt;h2&gt;The exit is deletion&lt;/h2&gt;
&lt;p&gt;The end of a migration is not &amp;quot;the comparison has been clean for a while.&amp;quot; The exit criterion is deleting the old system — its code, its deployment, and the comparison harness along with it. The harness is what gives you the confidence to wipe the old thing away; keeping both running &amp;quot;just in case&amp;quot; is twice the work and two ways to break. A system kept around still drifts, still gets paged, and still gets hit by clients nobody rerouted.&lt;/p&gt;
&lt;h2&gt;Where the lesson now lives&lt;/h2&gt;
&lt;p&gt;That recipe now lives inside the planning workflow our users&#39; agents run. When a plan describes a change that rewrites, ports, or replaces a running system, the plan itself must prescribe the run-both-sides comparison and the before-and-after evidence — before a line of implementation starts. The requirement is routed, not broadcast: one conditional sentence inside the planning template&#39;s test-scenario guidance, in each place a test plan gets written, so only migration-shaped plans pick it up. A plan for a landing page never sees it.&lt;/p&gt;
&lt;h2&gt;What this recipe does not cover&lt;/h2&gt;
&lt;p&gt;Honesty about scope is part of the artifact, so here it is plainly. The recipe governs replacing a runnable system — a rewrite in a new language, a pipeline moved to a new engine, a component swapped for an equivalent one. It deliberately does not govern the schema-change scripts that reshape a live database; those keep their own runbooks and their own defect histories.&lt;/p&gt;
&lt;p&gt;The title&#39;s &amp;quot;schema&amp;quot; is the cheap test everyone already runs — did the change apply — not the document&#39;s scope. And yes, our own three-week incident was a schema-shaped change. That is the point rather than a contradiction: whatever layer a change lives in, &amp;quot;it applied cleanly&amp;quot; is evidence the change landed, never evidence that anything downstream still works. The compare-before-you-trust instinct applies everywhere; the full oracle apparatus is for when you replace the machine itself.&lt;/p&gt;
&lt;h2&gt;The harness had to prove it could see red&lt;/h2&gt;
&lt;p&gt;The review story behind the document carries this arc&#39;s usual twist — the verification machinery needed verifying.&lt;/p&gt;
&lt;p&gt;The recipe reaches a user&#39;s plans through a pointer sentence inside template text that gets copied verbatim into every generated plan. The first version of the link inside that sentence was written for where the file lives in our repository — which resolves to nothing inside a plan generated on a user&#39;s machine. The fix anchored the link to the user&#39;s own installation instead.&lt;/p&gt;
&lt;p&gt;But the part worth stealing came next. The plan claimed that existing checks already guaranteed the new document stayed reachable — a coverage claim. That claim was believed only after a negative control ran: on a scratch copy, delete the pointer and confirm the census check went red, naming the missing file. A suite you say covers a thing is evidence only after removing the thing makes it fail.&lt;/p&gt;
&lt;p&gt;The same plan escalated a placement question — full reference document or slimmed-down summary page — to an explicit ruling instead of a guess, and the ruling kept the full version. A decision the agent could not settle locally got decided in the plan, in writing.&lt;/p&gt;
&lt;p&gt;This post tells one thread of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the four days we spent shipping a trust checklist&lt;/a&gt;, the arc&#39;s day-by-day account.&lt;/p&gt;
&lt;h2&gt;What to demand before you retire the old thing&lt;/h2&gt;
&lt;p&gt;You do not have to build the harness to ask the questions it answers. The next time your AI, a contractor, or a vendor proposes replacing a piece of your product that already works, ask three things.&lt;/p&gt;
&lt;p&gt;First: what evidence will you show me that the new system does what the old one did — not &amp;quot;it passed its tests,&amp;quot; but the same inputs through both, with every difference accounted for? Second: where is the written list of the differences you meant to make — because anything not on that list is a bug, not a surprise to negotiate later? Third: when does the old system actually get deleted — because &amp;quot;we will keep both running&amp;quot; is not a migration, it is a mortgage.&lt;/p&gt;
&lt;p&gt;And ask the question this whole arc hangs on: can the check go red? &amp;quot;It applied cleanly&amp;quot; could not. A side-by-side comparison can.&lt;/p&gt;
&lt;p&gt;For the full technical recipe — the normalization contract, the replayed-traffic corpus, the approved-changes ledger, and the delete-the-old exit criterion — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/plugins/soleur/skills/plan/references/differential-migration-testing.md&quot;&gt;the reference document on GitHub&lt;/a&gt;. And if this is the discipline you want wrapped around the AI work in your own company, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What is differential testing?&lt;/h3&gt;
&lt;p&gt;A testing approach where you run an old system and its replacement side by side on the same inputs and compare their outputs. The old system acts as the oracle — the definition of correct — and every difference between the two is either a bug to fix or a deliberate change that must be written down with a reason. There is no third option.&lt;/p&gt;
&lt;h3&gt;How is this different from a migration test?&lt;/h3&gt;
&lt;p&gt;A typical migration test asks whether the change applied cleanly — did the script run, did the data land in the new shape. Differential testing asks whether everything downstream still behaves the way the old system behaved. Our own incident applied cleanly and still broke the feature users touched for about three weeks; &amp;quot;it applied&amp;quot; and &amp;quot;it still works&amp;quot; are different claims that need different evidence.&lt;/p&gt;
&lt;h3&gt;What counts as evidence a replacement is safe?&lt;/h3&gt;
&lt;p&gt;Three things: the same inputs run through the old system and the new one, with outputs compared in both directions; a written ledger where every accepted difference has a stated reason — anything not in the ledger is a bug; and an exit that deletes the old system once the comparison is clean modulo that ledger. A check that can only ever report success is decoration, not coverage.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is differential testing?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;A testing approach where you run an old system and its replacement side by side on the same inputs and compare their outputs. The old system acts as the oracle — the definition of correct — and every difference between the two is either a bug to fix or a deliberate change that must be written down with a reason. There is no third option.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How is this different from a migration test?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;A typical migration test asks whether the change applied cleanly — did the script run, did the data land in the new shape. Differential testing asks whether everything downstream still behaves the way the old system behaved. Our own incident applied cleanly and still broke the feature users touched for about three weeks; &#92;&quot;it applied&#92;&quot; and &#92;&quot;it still works&#92;&quot; are different claims that need different evidence.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What counts as evidence a replacement is safe?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Three things: the same inputs run through the old system and the new one, with outputs compared in both directions; a written ledger where every accepted difference has a stated reason — anything not in the ledger is a bug; and an exit that deletes the old system once the comparison is clean modulo that ledger. A check that can only ever report success is decoration, not coverage.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>A Map That Can&#39;t Drift</title>
    <link href="https://soleur.ai/blog/map-that-cant-drift/" />
    <updated>2026-11-05T00:00:00Z</updated>
    <id>https://soleur.ai/blog/map-that-cant-drift/</id>
    <content type="html">&lt;p&gt;You cannot keep your own product honest by memory. Every founder learns this the embarrassing way: the onboarding tour that describes a screen you renamed three months ago, the help article that walks a customer through a flow you no longer offer, the marketing page still promising a feature you quietly removed. Nothing in a solo company is assigned to keep those documents true, so they drift — politely, invisibly, and always in the direction of promising more than the product actually does.&lt;/p&gt;
&lt;p&gt;An AI helper makes this worse, not better. An agent navigating your product builds its own private map on every run — which page does what, how a user gets there, what they should find — and throws the map away when the session ends. Tomorrow it builds the map again, slightly differently. There is no shared memory to correct, and nothing that ever tells the map it is wrong.&lt;/p&gt;
&lt;p&gt;We hit this wall building our own platform, and the fix we shipped sounds like paperwork until you watch it work: a written map of the product that carries its own enforcement. A map the product cannot drift away from.&lt;/p&gt;
&lt;h2&gt;A map with thirty-two entries — and a reason for every absence&lt;/h2&gt;
&lt;p&gt;The map itself is a single document listing every distinct page a visitor can reach in our product — thirty-two of them. Twenty-seven get a full entry: how a real user reaches the page, which stable landmarks an agent can look for, and what the user should find when they arrive. The other five get no entry, and that is deliberate — each one sits on a short exemption list inside the same document, each with a written reason. An onboarding step that exists only as a funnel page. A public share link that needs a real token before it shows anything. Pages that are real but are not features, and the map says exactly why.&lt;/p&gt;
&lt;p&gt;That last clause — &amp;quot;with a written reason&amp;quot; — is the discipline. An exemption without a reason is just a page somebody forgot, wearing a better name.&lt;/p&gt;
&lt;h2&gt;The guard that checks both directions&lt;/h2&gt;
&lt;p&gt;A document that lists what exists drifts in two directions at once. New things appear that it never recorded. And things it recorded quietly stop existing.&lt;/p&gt;
&lt;p&gt;So the guard that keeps the map honest checks both. Add a page and forget to update the map — the check fails. Delete a page and leave its entry behind — the check fails, because an entry for a page that no longer exists is the most dangerous line in the whole document: it sends whoever reads the map looking for something that is not there, with full confidence.&lt;/p&gt;
&lt;p&gt;The guard also checks that entries are real. Every entry must carry its required fields — how a user reaches the page, what they should see — not just a heading and some prose. A page cannot sit in the mapped list and the exemption list at the same time; the classification has to pick a side. And nothing formatted as an example inside the document may count as a real entry.&lt;/p&gt;
&lt;p&gt;That last rule exists because of what our review panel found. Which is the part of this story that earned the post.&lt;/p&gt;
&lt;h2&gt;Four ways the guard itself could lie&lt;/h2&gt;
&lt;p&gt;Before the map shipped, our reviewers graded the guard the way we now grade every check that will be trusted: name an input that satisfies the check while violating the property it claims to enforce. They found four.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The census that counted thirty-one of thirty-two.&lt;/strong&gt; The guard builds its list of real pages with a file-listing pattern — the kind that says &amp;quot;find me every page file inside every folder.&amp;quot; Measured against an independent count, that pattern returned thirty-one files where there were thirty-two. The missing one was the homepage itself: the pattern requires at least one folder between the starting point and the file, so the single page file sitting at the front door was never counted. A census that under-counts does not look broken — it looks green forever, because the check can only see what the pattern returned.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A completeness check satisfied by a mention.&lt;/strong&gt; Each entry has required fields — how a user reaches the page, what they should see there. The first version of the check searched each entry&#39;s text for those field names. A sentence inside an entry that merely mentioned a field — the equivalent of &amp;quot;see the field list above&amp;quot; — satisfied the check for an entry that had never filled the field in. Presence is not conformance: a check that searches for a word cannot tell the difference between a filled-in field and a sentence about one.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The example that counted as real.&lt;/strong&gt; The map document shows its own format — a worked example of what an entry looks like, included so a human reader can follow the pattern. But the guard reads the document line by line, and a line sitting inside a formatted example block looked exactly like a real entry to it. An illustration of an entry could certify a page nobody had ever verified. The fix was blunt: the document&#39;s own convention now forbids example blocks entirely — the map never needed them, and banning the construct is far cheaper than teaching the checker to ignore it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Two scoreboards for the same question.&lt;/strong&gt; The guard tracked early errors in one tally and check failures in another, then asked &amp;quot;did anything fail?&amp;quot; against a hand-maintained list of counters. Any future check added to the guard that reported a problem but forgot to register its counter would report-and-pass — the finding would print, and the run would still come back green. Now there is one ledger: every check adds to it, and it is the only thing the final verdict reads.&lt;/p&gt;
&lt;p&gt;Four ways for &amp;quot;the map is correct&amp;quot; to be a lie — each subtle, each caught before a single agent relied on it.&lt;/p&gt;
&lt;h2&gt;The general lesson: any count can be quietly short&lt;/h2&gt;
&lt;p&gt;One of those four is worth pulling out on its own, because it reaches far past our document: any check built on a listing can under-count and stay green. A pattern that returns thirty-one results where there are thirty-two looks identical to a pattern that is working — the missing item is invisible precisely because the pattern never produced it. And a check that reports its findings on one channel and its verdict on another can always drift into reporting without failing.&lt;/p&gt;
&lt;p&gt;That yields two habits, both cheap. First, when a check counts things, count them a second way and compare — not because the second count is more trustworthy, but because the disagreement is the signal. Second, before you trust a check, prove it can go red: hand it a tree you know is wrong and watch it fail. A check that has never failed is a hypothesis, not a guard. Ours now ships with a companion battery that does exactly that — it deliberately breaks the tree underneath the check and demands the check notice.&lt;/p&gt;
&lt;h2&gt;What the guard does today — stated plainly&lt;/h2&gt;
&lt;p&gt;Here is the honest scope, because a post about honest verification should not oversell its own machinery. The guard runs as part of our full local test suite, where a disagreement between the map and the product fails the run. In continuous integration — the pipeline that re-runs our checks on every proposed change — it currently reports rather than blocks. That is a deliberate staging choice while the guard earns trust, not an oversight, and the document itself says so in writing.&lt;/p&gt;
&lt;p&gt;Enforcement that reports is still enforcement: the finding prints, names the page, and says which direction it drifted. What the guard does not yet do is stop a merge — and this post will not pretend otherwise. The day it gets promoted, the only thing that changes is the volume. The check itself stays the same.&lt;/p&gt;
&lt;p&gt;The same idea also exists for a founder&#39;s own codebase — a companion post tells that story: &lt;a href=&quot;https://soleur.ai/blog/map-that-only-records-what-the-agent-saw/&quot;&gt;the map that only records what the agent saw&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This story is one spoke of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the four days we spent shipping a trust checklist&lt;/a&gt;, the day-by-day account of the whole arc.&lt;/p&gt;
&lt;h2&gt;What a founder takes from this&lt;/h2&gt;
&lt;p&gt;You do not need our machinery to use the lesson. Whatever keeps your company&#39;s documents honest — a wiki, a help center, an internal handbook, a feature list on your pricing page — ask it the question our reviewers asked: what makes this true? If the honest answer is &amp;quot;somebody remembers to update it,&amp;quot; the document is already drifting; the only question is how far. The fix is rarely a better document. It is a check that can fail — one that counts in both directions, and one you have watched go red on purpose.&lt;/p&gt;
&lt;p&gt;And if you hand your product to an AI to test, audit, or drive, give it the same thing we gave ours: a map it can read, and a guard that keeps the map honest. An agent working from a maintained map navigates. An agent rebuilding its own private map from scratch on every run is guessing — confidently, fluently, and without ever telling you it guessed.&lt;/p&gt;
&lt;p&gt;For the full technical post-mortem — the exact file-listing trap, the three parser defects, and the test battery that now mutates the tree underneath the guard to prove it still sees red — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/knowledge-base/project/learnings/2026-10-20-a-census-guard-under-counts-on-both-enumerators.md&quot;&gt;the write-up on GitHub&lt;/a&gt;. And if this is the kind of verification machinery you want wrapped around the AI work in your own company, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What is a feature map?&lt;/h3&gt;
&lt;p&gt;A feature map is a written inventory of everything your product does from the user&#39;s point of view — each page or screen, how a real person reaches it, and what they should find there. Unlike documentation that is written once and left to rot, a feature map lives next to a check that keeps it honest, so it stays true instead of becoming a confident lie.&lt;/p&gt;
&lt;h3&gt;How can documentation check itself?&lt;/h3&gt;
&lt;p&gt;By comparing the document against the thing it describes, in both directions. Anything real that is missing from the document is a failure, and anything in the document that no longer exists is a failure too. The document becomes something software can verify rather than something a person has to remember to update.&lt;/p&gt;
&lt;h3&gt;Why did the checker need checking?&lt;/h3&gt;
&lt;p&gt;Because a check that under-counts, or that can report a problem and still pass, is worse than no check at all — it shows green while certifying nothing. Reviewers found four distinct ways ours could have lied, from a file listing that skipped the homepage to two separate tallies for &amp;quot;did it fail.&amp;quot; Each was fixed before anyone relied on the map.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is a feature map?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;A feature map is a written inventory of everything your product does from the user&#39;s point of view — each page or screen, how a real person reaches it, and what they should find there. Unlike documentation that is written once and left to rot, a feature map lives next to a check that keeps it honest, so it stays true instead of becoming a confident lie.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How can documentation check itself?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;By comparing the document against the thing it describes, in both directions. Anything real that is missing from the document is a failure, and anything in the document that no longer exists is a failure too. The document becomes something software can verify rather than something a person has to remember to update.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Why did the checker need checking?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Because a check that under-counts, or that can report a problem and still pass, is worse than no check at all — it shows green while certifying nothing. Reviewers found four distinct ways ours could have lied, from a file listing that skipped the homepage to two separate tallies for &#92;&quot;did it fail.&#92;&quot; Each was fixed before anyone relied on the map.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>What Your MVP Is Missing Before Real Users Show Up: A Findings List, Not a Verdict</title>
    <link href="https://soleur.ai/blog/what-your-mvp-is-missing-before-real-users-show-up/" />
    <updated>2026-11-04T00:00:00Z</updated>
    <id>https://soleur.ai/blog/what-your-mvp-is-missing-before-real-users-show-up/</id>
    <content type="html">&lt;p&gt;Your product works. You have clicked through it a hundred times, and it does what you built it to do. Now someone who is not you is about to sign up, with their email address and maybe their card, and a quiet question shows up: what haven&#39;t I checked?&lt;/p&gt;
&lt;p&gt;You can&#39;t answer it by clicking around more. The things that hurt a young product are rarely on the screen. A password saved in your code where anyone with access can read it. No test that tells you a change broke sign-up. No alert when the site goes down at 3 a.m. No written way to undo a bad release. You only find out about these when a user does.&lt;/p&gt;
&lt;h2&gt;What a founder actually needs: a list, not a grade&lt;/h2&gt;
&lt;p&gt;Many readiness tools hand you a score or a certificate. A score invites the wrong question (&amp;quot;am I a 72?&amp;quot;). A certificate promises something no automated check can promise.&lt;/p&gt;
&lt;p&gt;What you need is shorter and more useful: a plain list of the gaps, in the order they matter, so you can pick the next three things to fix and ignore the rest for now.&lt;/p&gt;
&lt;p&gt;That is what Soleur&#39;s launch readiness check produces. You run it on a project you already have, and it writes one report with the gaps it found. It is a findings list, and every report opens with a notice saying it is not a security review or an attestation of any standard.&lt;/p&gt;
&lt;h2&gt;What the list covers: security, testing, upkeep and uptime&lt;/h2&gt;
&lt;p&gt;The report groups what it finds into four areas a founder worries about:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Security and secrets.&lt;/strong&gt; Signs of passwords or keys saved in your project, and whether your project locks its outside software to known versions and has automatic updates set up.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Tests and deployment.&lt;/strong&gt; Signs that something checks your work automatically before it reaches users, and that there is a release path set up or written down.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Tech debt and architecture.&lt;/strong&gt; Whether the big decisions about how your product is put together are written down anywhere, so you (or whoever you hire next) can find them.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Reliability and operations.&lt;/strong&gt; Signs that you would hear about a failure (such as error reporting being set up), that the site has a page that reports whether it is up, and that your files mention how backups and restores work.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These are automatic look-for-signs checks, not proof. A tick means the check found the sign it looks for, not that the thing works.&lt;/p&gt;
&lt;p&gt;Each gap comes with a severity and a plain description of what to do about it. It also looks at which of Soleur&#39;s own working practices your project is missing, such as automatic checks that stop a secret from being saved by accident, so the report tells you what to add as well as what is wrong.&lt;/p&gt;
&lt;h2&gt;Your report never contains your secrets&lt;/h2&gt;
&lt;p&gt;A check that finds a password in your project is the one most worth getting right. So when the report finds a secret, it tells you how many it found, never what they are or where they are. If you want the locations, you run the scan yourself in a separate terminal window, and they appear on your own screen only, never in the report.&lt;/p&gt;
&lt;p&gt;Two limits to know about. The secret scan looks at the version of your project you last saved to its history, not at the whole history (the record of every earlier version). A secret you deleted last month can still be live in that history, so if the report mentions any, change them. And the AI review described below reads files from your project, so a secret written directly into one of those files is part of what goes to your AI provider.&lt;/p&gt;
&lt;h2&gt;The check runs from your own computer&lt;/h2&gt;
&lt;p&gt;The scan runs on your computer, and Soleur&#39;s servers receive nothing. Today you run the check from your own computer, set up through the &lt;a href=&quot;https://soleur.ai/getting-started/&quot;&gt;getting started guide&lt;/a&gt;. It needs one free secret-scanning program installed first, and it stops with a clear message if that is missing.&lt;/p&gt;
&lt;p&gt;The check also has the AI you already use read files from your project, except the ones made to hold passwords and keys, to judge things a scan can&#39;t, such as how people sign in. That content goes to your AI provider. If you accept the offer to file the gaps as tasks, the check reads from and then writes to GitHub with your own login. The report is a file in your project, and because it lists weaknesses you have not fixed yet, you should keep that project private.&lt;/p&gt;
&lt;h2&gt;Turn the gaps into a to-do list, only if you approve it&lt;/h2&gt;
&lt;p&gt;A list is only useful if the gaps become tasks. After the report is written, the check can offer to file them in your own GitHub project: one parent item, and one item per gap, ranked by how much it matters and labelled by area. Run it again later and it will not file a gap it already filed, even if you closed the item, as long as you leave the item&#39;s label in place.&lt;/p&gt;
&lt;p&gt;That step is optional, and it is built so that you are asked first. The check shows you the full list of what it would file, then asks about your project, whether it is public, private or internal, how many items it will create and which labels it will add if they are missing. If your project is public, internal or of unknown visibility, it asks a second, separate question, because filing unfixed gaps there would publish them for everyone who can see the project.&lt;/p&gt;
&lt;p&gt;The command that does the filing cannot tell whether you agreed, so if you ever run it directly, you are the one deciding. It never closes or deletes anything, and it refuses to file when it detects that it is running unattended. If you change your mind, closing the items removes them from your list, though notifications already sent to people watching your project cannot be recalled.&lt;/p&gt;
&lt;p&gt;You can also accept a finding on purpose. If a gap does not apply to your product, say why, and the report moves it to an &amp;quot;accepted&amp;quot; list instead of nagging you.&lt;/p&gt;
&lt;h2&gt;What a clean report does not tell you&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;It does not promise that a clean report means you are safe. A gap it doesn&#39;t list was not found in the files it read, which is different from not being there.&lt;/li&gt;
&lt;li&gt;Its checks can be wrong. Treat each line as a prompt to look, not a ruling.&lt;/li&gt;
&lt;li&gt;Some checks apply only to products built with JavaScript or TypeScript (two common web languages). For other products the report says coverage is partial and lists the checks it could not run, rather than showing them as fine.&lt;/li&gt;
&lt;li&gt;The secret scan cannot see files you have not saved to your project&#39;s history. The other checks read the files in your project folder as they are on disk, including changes and new files you have not yet saved to your project&#39;s history, and skip new files your project is set to ignore.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;div class=&quot;faq-list&quot;&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Does a clean report mean my product is ready for customers?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;No. The report lists gaps it found in the files it read. A gap that is not listed was not found, which is not the same as not being there. It is a findings list, not a score or a certification.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Will my passwords or keys appear in the report?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;No. When the check finds a secret it reports how many, never what they are or where they are. The secret scan looks at the version of your project you last saved to its history, so a secret you deleted earlier can still be live there and should be changed.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Does anything leave my computer?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;The scan runs on your computer and Soleur&#39;s servers receive nothing. The AI you already use reads files from your project, except files made to hold passwords and keys, so that content goes to your AI provider. If you accept the offer to file the gaps as tasks, the check reads from and then writes to GitHub with your own login.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Is this a security audit?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;No. It is a findings list, and every report opens with a notice that it is not a security review or an attestation of any standard.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Will it create tasks in my GitHub project without asking?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;It is built to ask first. It shows you the full list of what it would create and asks, naming your project, whether it is public, private or internal, and how many items it will file. A project that is public, internal or of unknown visibility gets a second, separate question, and a run it detects as unattended is refused.&lt;/p&gt;
  &lt;/details&gt;
&lt;/div&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Does a clean report mean my product is ready for customers?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. The report lists gaps it found in the files it read. A gap that is not listed was not found, which is not the same as not being there. It is a findings list, not a score or a certification.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Will my passwords or keys appear in the report?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. When the check finds a secret it reports how many, never what they are or where they are. The secret scan looks at the version of your project you last saved to its history, so a secret you deleted earlier can still be live there and should be changed.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Does anything leave my computer?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;The scan runs on your computer and Soleur&#39;s servers receive nothing. The AI you already use reads files from your project, except files made to hold passwords and keys, so that content goes to your AI provider. If you accept the offer to file the gaps as tasks, the check reads from and then writes to GitHub with your own login.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Is this a security audit?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. It is a findings list, and every report opens with a notice that it is not a security review or an attestation of any standard.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Will it create tasks in my GitHub project without asking?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;It is built to ask first. It shows you the full list of what it would create and asks, naming your project, whether it is public, private or internal, and how many items it will file. A project that is public, internal or of unknown visibility gets a second, separate question, and a run it detects as unattended is refused.&quot;
      }
    }
  ]
}
&lt;/script&gt;
&lt;h2&gt;Your next step&lt;/h2&gt;
&lt;p&gt;Want an AI team that works through the list while you make the calls? &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;Join the waitlist&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;For the technical write-up, see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/knowledge-base/engineering/architecture/decisions/ADR-314-launch-check-baseline-manifest-and-finding-contract.md&quot;&gt;how the check was designed&lt;/a&gt;.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Before Your AI Team Takes a Recurring Job, It Answers Six Questions</title>
    <link href="https://soleur.ai/blog/before-your-ai-team-takes-a-recurring-job/" />
    <updated>2026-11-04T00:00:00Z</updated>
    <id>https://soleur.ai/blog/before-your-ai-team-takes-a-recurring-job/</id>
    <content type="html">&lt;p&gt;Somewhere in your business, a machine does something on a schedule every night. An invoice reminder. A weekly report. A newsletter draft queued for morning. You set most of them up yourself — probably in a hurry, on a Tuesday, because you needed the thing done more than you needed to think about how it would happen.&lt;/p&gt;
&lt;p&gt;A few months later, could you name every one of them? What each one touches? Whether any two are doing the same job?&lt;/p&gt;
&lt;p&gt;Most founders can&#39;t. The automation pile grows by accretion — each piece added in a moment of need, none ever removed, and the inventory lives in somebody&#39;s head or nowhere at all. That pile works fine right up until it doesn&#39;t, and the failure is always the same shape: something ran twice, something ran and nobody noticed, or something kept running long after it should have stopped.&lt;/p&gt;
&lt;p&gt;We decided that answer can&#39;t live in anyone&#39;s head — including ours. Before our AI team is allowed to give itself a new recurring job, it has to write down six things about the job. Until the answers exist, the job doesn&#39;t exist.&lt;/p&gt;
&lt;h2&gt;The six questions&lt;/h2&gt;
&lt;p&gt;Every new recurring job — a scheduled check, a background process, a monitor that watches other monitors — now declares, before it is allowed to exist:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What wakes it up.&lt;/strong&gt; A clock, an event, a person, or another job finishing. If you can&#39;t say what fires it, you can&#39;t say when it runs.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What it reads and what it changes.&lt;/strong&gt; A job that writes to your billing system deserves a different level of attention than one that writes a draft you&#39;ll review. Naming the surfaces makes the risk legible before it&#39;s live.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What it hands off next.&lt;/strong&gt; Jobs rarely end. They produce something — a ticket, a report, an alert — and something downstream has to own it. &amp;quot;Nothing&amp;quot; is a valid answer, but it has to be a decision, not an omission.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;How a person steps back in.&lt;/strong&gt; When a job goes wrong at 3 a.m., how do you pause it, redirect it, or stop it? If the answer is &amp;quot;dig through the code,&amp;quot; the job isn&#39;t ready to run unattended.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it can&#39;t do the same work twice.&lt;/strong&gt; This is the question everyone gets wrong, and it is worth its own section — see below.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;How the thing that assigns work avoids assigning it twice.&lt;/strong&gt; When a system can file work for itself — an alert that becomes a task — something has to prevent the same alert from becoming two tasks.&lt;/p&gt;
&lt;h2&gt;A line at a bakery is not a deduplicator&lt;/h2&gt;
&lt;p&gt;The subtle one is the fifth question. Most systems that run recurring work offer a way to make jobs wait their turn — one run at a time, in order. That sounds like protection against duplicate work. It isn&#39;t. Making jobs wait their turn is not the same as making sure the work doesn&#39;t happen twice.&lt;/p&gt;
&lt;p&gt;A line at a bakery is orderly. It does not stop two customers from ordering the same birthday cake.&lt;/p&gt;
&lt;p&gt;Our rule makes that distinction a hard requirement: an answer of &amp;quot;the jobs wait their turn&amp;quot; is rejected outright. The job has to name a real mechanism — checking whether the work was already done before starting, leaving a marker so the next run sees it, or keying the work to something that can&#39;t be created twice. Order is a courtesy. Idempotence is a property.&lt;/p&gt;
&lt;h2&gt;Three doors, or seven&lt;/h2&gt;
&lt;p&gt;When we built this rule, the first version watched the three doors we already knew about — the place we schedule automations, the place our background functions live, and our registered checks. It would have caught every new job that entered through any of the three.&lt;/p&gt;
&lt;p&gt;Then we did the boring thing, which turned out to be the important thing: we enumerated every way a recurring job can be born in our system. Not the ways we remembered — the ways that exist.&lt;/p&gt;
&lt;p&gt;Seven. Jobs created inside the database itself. Jobs created by infrastructure files that generate other jobs — a job born one step removed, invisible to anyone watching the final file. Jobs armed by an API call that leaves no file behind at all, so there is nothing for a file-watching rule to see.&lt;/p&gt;
&lt;p&gt;A rule that watches the doors you know is a rule that watches the doors you know. The enumeration is what turned a good intention into a gate.&lt;/p&gt;
&lt;h2&gt;Why this is a creation rule, not a cleanup rule&lt;/h2&gt;
&lt;p&gt;The check only applies to new jobs. You can&#39;t retrofit trust onto jobs that are already running — the honest answer for those is an inventory, and ours runs as an ongoing audit. But every job that enters the system from now on arrives with its answers already written down.&lt;/p&gt;
&lt;p&gt;That asymmetry matters. If the answers are required at creation, the design answers are never a to-do item for a future you. They are written when the job is cheap to change — before it has a reason to exist unchanged.&lt;/p&gt;
&lt;h2&gt;What a recurring job must declare&lt;/h2&gt;
&lt;p&gt;If you run your own automations — in any system, not just ours — the same six questions are worth answering for each one, on paper, before it goes on:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;What wakes it up?&lt;/li&gt;
&lt;li&gt;What does it read, and what does it change?&lt;/li&gt;
&lt;li&gt;What does it hand off when it finishes?&lt;/li&gt;
&lt;li&gt;How do you pause it, redirect it, or stop it at 3 a.m.?&lt;/li&gt;
&lt;li&gt;Why can&#39;t it do the same work twice — a real mechanism, not &amp;quot;it waits its turn&amp;quot;?&lt;/li&gt;
&lt;li&gt;If it can file work for itself, what stops it filing the same work twice?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If your automations have quietly become a system nobody fully remembers, that is the moment to get the answers out of your head and into a place they can be checked. &lt;a href=&quot;https://soleur.ai&quot;&gt;Try Soleur&lt;/a&gt; — we made the answers a condition of the job existing at all.&lt;/p&gt;
&lt;p&gt;For the technical write-up — the map that names every one of the seven doors a background job can enter through — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/knowledge-base/engineering/architecture/factory-map.md&quot;&gt;our factory-map architecture note&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently asked questions&lt;/h2&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How do I find out what automations are running in my business?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Start with an inventory, not an audit. List every scheduled job you can find across every system that runs one — schedulers, databases, background processes, integrations that fire on events. Most founders find several they forgot and at least one doing work nobody owns. The inventory, not the fix, is the first deliverable.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is the difference between scheduling work and duplicating it?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Scheduling decides when work runs. Duplication is whether the same work can happen twice. A queue that runs jobs one at a time is orderly, but it does not check whether the work was already done. Preventing duplication takes a dedicated mechanism — checking first, marking the work done, or keying it to something unique.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>The First Property Test We Wrote Found a Real Bug</title>
    <link href="https://soleur.ai/blog/first-property-test-found-real-bug/" />
    <updated>2026-11-03T00:00:00Z</updated>
    <id>https://soleur.ai/blog/first-property-test-found-real-bug/</id>
    <content type="html">&lt;p&gt;Every check guarding your company&#39;s work is a list of cases somebody remembered to write down. That is true of the checklist your assistant follows, the spot-checks you run before shipping, and the tests your AI writes for its own output. The list is only as good as the imagination behind it — and the imagination doing the listing has the same blind spots the rest of you does.&lt;/p&gt;
&lt;p&gt;We found out how much that costs the day we stopped writing lists and started writing rules. The very first run caught a real bug — one that had already survived two earlier fixes to the same drift, behind a documented promise and a dedicated check that had never once seen it.&lt;/p&gt;
&lt;h2&gt;Your test list is your imagination&lt;/h2&gt;
&lt;p&gt;The ordinary way to check anything — in code, in a spreadsheet, in a process — is the example: pick a case, decide what the right answer looks like, compare. It works until the interesting case is the one nobody thought of. The cases that make the list are the ones the author could picture, and the author can only picture inputs that look reasonable to a human.&lt;/p&gt;
&lt;p&gt;That is the quiet hole in every hand-picked suite. It is not that the wrong answers pass. It is that the question that would produce them is never asked, because &amp;quot;a repository address with a stray space hiding just before its file suffix&amp;quot; is not a shape a person reaches for. No one writes the example they cannot imagine.&lt;/p&gt;
&lt;h2&gt;A rule instead of a list&lt;/h2&gt;
&lt;p&gt;There is a stronger habit, and we adopted it formally: the property test. Instead of listing cases, you state a rule that must always hold — &amp;quot;running a cleanup twice changes nothing the second time&amp;quot; — and a generator throws thousands of random inputs at the rule, trying to break it. The machine does not share our sense of what a reasonable input looks like, so it explores corners no one would think to try: empty inputs, padded inputs, inputs with marks hiding exactly where the cleanup operates.&lt;/p&gt;
&lt;p&gt;When the generator wins, it does something a human tester rarely does: it shrinks the failing input down to the smallest form that still breaks the rule, and that minimized case joins the test list permanently, as a named example the bug can never slip past again. And because a rule that can only ever pass is decoration, each generated check proves itself once at write time against deliberately broken code — it has to show red before it is trusted to show green.&lt;/p&gt;
&lt;h2&gt;Day one&lt;/h2&gt;
&lt;p&gt;The first generated test we aimed was at a small utility that tidies repository addresses — trims stray spaces, strips a trailing suffix, the kind of boring function every product accumulates. Its own documentation promised that running it twice was safe: clean up the cleaned-up value and nothing changes.&lt;/p&gt;
&lt;p&gt;The generator&#39;s first run disagreed. It produced an address with a space sitting just in front of the suffix — an input no human picks, because no human thinks an address looks like that. The cleanup trimmed the ends first, then stripped the suffix — and the space the suffix had been hiding stepped out into the open, left behind at the end of the result, invisible to the eye. Run the tidy-up once, get one answer. Feed that answer back in, and the leading trim moves it again: a third answer. The documented contract said that was impossible. The code did it anyway.&lt;/p&gt;
&lt;p&gt;And the dedicated check we already had — a separate comparison built specifically to keep that cleanup honest — had never seen it either, because its own hand-picked list of cases contained nothing shaped like the offending input. It was the third time the same cleanup had drifted. Every example we had ever written had walked around it.&lt;/p&gt;
&lt;h2&gt;The twin&lt;/h2&gt;
&lt;p&gt;Once we knew the shape to look for, we found the same defect in a second place: the database kept its own copy of the same cleanup, written in the database&#39;s own language, with the same sequence of steps and the same missing final trim. It was the third time the two implementations of one idea had drifted apart — each time in a different way, each time unnoticed until something looked sideways at it.&lt;/p&gt;
&lt;p&gt;Both are fixed now. The application side gained a final tidy-up pass after the stripping, closing the hole the suffix had opened. The database side could not be edited in place — a database change that has already run is history, not a draft — so the fix shipped as a new change that re-cleaned the rows already stored in the broken shape, behind a strict check that only rewrites values both sides agree on. That follow-up has since shipped and closed. The lesson outlived the bug: when the same logic lives in two places, a fix on one side means the other side now disagrees with it.&lt;/p&gt;
&lt;h2&gt;The harness needed fixing too&lt;/h2&gt;
&lt;p&gt;Here is the part of the story we tell carefully, because this post exists to practice what it preaches. The machinery we built to run these generated tests arrived with defects of its own — and every one of them failed in the same direction: it covered less than it claimed while still reporting green.&lt;/p&gt;
&lt;p&gt;One job of the machinery was to read the program under test and collect the rules it was supposed to watch. It understood one of the two ways a rule could be written — so it was watching 31 of the 39 rules while the check stayed green. A rule written in the other style would never join the check at all, and nothing would ever notice.&lt;/p&gt;
&lt;p&gt;A second surface was subtler. A test input that arrived in a form the checker could not read was silently treated as &amp;quot;nothing wrong found&amp;quot; — folding into the machinery as a permanent do-nothing, a check that looks alive and can never fire.&lt;/p&gt;
&lt;p&gt;The third was older and stranger. The built-in random-number source in the language our small helper programs are written in carries a trap on older versions: called one perfectly ordinary way, it keeps handing back the same draw instead of advancing. A run that claimed a hundred draws had really drawn one — and the pass counter printed a hundred beside it.&lt;/p&gt;
&lt;p&gt;All three are fixed: the watcher now counts the rules independently and demands the two counts agree; every collected input must prove it can be read before it is trusted; the generator draws without hitting the trap; and a floor rule insists a run must actually happen at least once before it is allowed to print green.&lt;/p&gt;
&lt;h2&gt;What the first day bought&lt;/h2&gt;
&lt;p&gt;Example tests sample the cases you remembered. Rules plus generation find the one you didn&#39;t. That is the whole trade, and the first run already paid for the practice: the defect it caught had already lived through two fixes, sitting in code that carried a written promise and a purpose-built comparison check, and neither had ever seen the input that broke it.&lt;/p&gt;
&lt;p&gt;That is also the honest way to read any green light your tools show you. The value of a check is measured by what it can still find, not by how long it has stayed green — a check that never goes red might mean the work is good, or it might mean the check cannot see. This post expands one thread of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the four days we spent shipping a trust checklist&lt;/a&gt; — and when we later wrote the guide that teaches this practice, reviewers caught that its own worked example could never fail, which is &lt;a href=&quot;https://soleur.ai/blog/property-test-cannot-fail-teaches-nothing/&quot;&gt;a post-mortem of its own&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;What to take into your own week&lt;/h2&gt;
&lt;p&gt;You do not need to write a test to use this. Wherever a list of remembered cases is what stands between you and a mistake — an audit checklist, a review routine, the prompts you hand your AI — ask whether a rule could stand there instead, and let a machine attack it. Ask of any green light two questions: what input would make this fail, and how much of what it claims to watch is it actually watching? And when the same logic lives in two places — the app and the database, the spreadsheet and the invoicing tool — treat a fix on one side as a promise you now owe the other.&lt;/p&gt;
&lt;p&gt;For the technical write-up — the defect, the twin, the three harness fixes, and the minimized inputs we pinned as permanent examples — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/knowledge-base/project/learnings/logic-errors/2026-10-20-repo-url-retrim-shielded-whitespace-sql-twin.md&quot;&gt;the learning record on GitHub&lt;/a&gt;. And if this is the discipline you want wrapped around the AI work in your own company, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What is a property test in plain terms?&lt;/h3&gt;
&lt;p&gt;It is a check built from a rule instead of a list. Instead of writing &amp;quot;this input should give that answer&amp;quot; for a handful of remembered cases, you state something that must always be true — &amp;quot;running this cleanup twice changes nothing&amp;quot; — and a generator throws thousands of random inputs at the rule trying to break it. When one does, the tool shrinks the failing input to its smallest form, and that case joins the test list permanently.&lt;/p&gt;
&lt;h3&gt;How is a rule better than a list of examples?&lt;/h3&gt;
&lt;p&gt;A list only covers the cases someone imagined. Our own result is the cleanest proof: a documented cleanup utility passed every hand-picked test we had ever written, then a generator&#39;s first run produced an input nobody would have chosen — an address with a space hiding before its suffix — and exposed a real defect. A rule covers the whole space, including the corners your imagination skips.&lt;/p&gt;
&lt;h3&gt;Do I need this if my AI writes my tests?&lt;/h3&gt;
&lt;p&gt;You need it especially then. An AI writes the same kind of list a person does — the cases it can imagine — only faster, so the blind spot arrives sooner. Asking your AI to state rules that must always hold, and to let a generator attack them, is what turns its speed into coverage instead of a longer list with the same holes.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is a property test in plain terms?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;It is a check built from a rule instead of a list. Instead of writing &#92;&quot;this input should give that answer&#92;&quot; for a handful of remembered cases, you state something that must always be true — &#92;&quot;running this cleanup twice changes nothing&#92;&quot; — and a generator throws thousands of random inputs at the rule trying to break it. When one does, the tool shrinks the failing input to its smallest form, and that case joins the test list permanently.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How is a rule better than a list of examples?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;A list only covers the cases someone imagined. Our own result is the cleanest proof: a documented cleanup utility passed every hand-picked test we had ever written, then a generator&#39;s first run produced an input nobody would have chosen — an address with a space hiding before its suffix — and exposed a real defect. A rule covers the whole space, including the corners your imagination skips.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Do I need this if my AI writes my tests?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;You need it especially then. An AI writes the same kind of list a person does — the cases it can imagine — only faster, so the blind spot arrives sooner. Asking your AI to state rules that must always hold, and to let a generator attack them, is what turns its speed into coverage instead of a longer list with the same holes.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>A Property Test That Cannot Fail Teaches Nothing</title>
    <link href="https://soleur.ai/blog/property-test-cannot-fail-teaches-nothing/" />
    <updated>2026-10-29T00:00:00Z</updated>
    <id>https://soleur.ai/blog/property-test-cannot-fail-teaches-nothing/</id>
    <content type="html">&lt;p&gt;Your AI produces work faster than you can check it. That is the bargain every founder makes with these tools, and most days it is a good one. The uncomfortable part is where your confidence actually comes from: checks wrapped around the work — checks you never wrote, cannot see, and mostly take on faith.&lt;/p&gt;
&lt;p&gt;Here is the question almost nobody asks of those checks: can this one actually fail?&lt;/p&gt;
&lt;p&gt;We learned to ask it inside a guide we wrote to teach our own AI a better way to check its work. This is the story of a teaching document that failed its own lesson, twice, before a single reader saw it. It matters even if you never write a test in your life, because the same question applies to every piece of guidance you hand an AI — and to every green light your tools show you.&lt;/p&gt;
&lt;h2&gt;Teaching the AI to test rules, not examples&lt;/h2&gt;
&lt;p&gt;Ordinary tests check examples: feed in these inputs, expect these answers. That works until the interesting case is the one nobody thought to write down. So we taught the AI in our workflows a stronger habit — the property test. Instead of listing cases, you state a rule that must always hold: &amp;quot;running a cleanup twice changes nothing the second time,&amp;quot; or &amp;quot;the sorted result contains the same items that went in.&amp;quot; Then a generator throws thousands of random inputs at the rule, trying to break it — including inputs no human would pick, because the machine does not share our sense of what a reasonable input looks like.&lt;/p&gt;
&lt;p&gt;The practice proved itself before any teaching was needed: the very first rule it ran caught a real bug on day one — which is why the guidance exists at all. Adoption first, pedagogy second. Then we wrote the guide that would hand the practice to our users&#39; workflows.&lt;/p&gt;
&lt;h2&gt;The example that could never fail&lt;/h2&gt;
&lt;p&gt;The guide went through our review panel — seventeen passes across the reviewers and the fix round — and three reviewers independently flagged the same finding, the release&#39;s one real defect at the highest severity grade we assign.&lt;/p&gt;
&lt;p&gt;The guide&#39;s worked example — the one meant to show what a good property test looks like — had been written so that every possible answer, including a completely wrong one, would pass it. Read the assertion slowly and it says, in effect: &amp;quot;the sorted list equals the original, or it doesn&#39;t, or sorting twice works.&amp;quot; Every branch of that sentence is a way to pass. There is no input that can fail it — not because sorting is easy, but because the sentence asks for nothing.&lt;/p&gt;
&lt;p&gt;An example that cannot fail teaches a practice that cannot catch anything. And because a worked example in a guide is read as endorsement — &amp;quot;this is how we do it&amp;quot; — a vacuous one is worse than no example at all.&lt;/p&gt;
&lt;h2&gt;The fix that still let nothing fail&lt;/h2&gt;
&lt;p&gt;Here is where the story earns its length. Removing the tautology was not enough.&lt;/p&gt;
&lt;p&gt;The repaired example checked two genuine rules: the sorted output must contain the same items that went in, and running the routine twice must change nothing the second time. Both are true of a correct sort. Both are also true of a program that returns its input untouched.&lt;/p&gt;
&lt;p&gt;A do-nothing program — take the list, hand it straight back — passes &amp;quot;same items out as went in.&amp;quot; It passes &amp;quot;running it twice changes nothing.&amp;quot; It sorts nothing at all. The reviewers&#39; fix round caught that second layer: only a third rule — the output must actually be in order — closed the example to trivial implementations.&lt;/p&gt;
&lt;p&gt;That is the deeper half of the lesson. A check is not cleansed by removing the obviously broken assertion. It is sound only when nothing lazy survives it — when &amp;quot;do nothing and still pass&amp;quot; is no longer on the table.&lt;/p&gt;
&lt;h2&gt;The same disease, twice more&lt;/h2&gt;
&lt;p&gt;Pulled on, the thread kept coming. The same guide carried two more instances of the same shape.&lt;/p&gt;
&lt;p&gt;The guide advised checking a round-trip — transform a value, transform it back — by comparing the result against an answer produced by the same machinery being tested. A consistently wrong program passes that check by being consistently wrong on both sides of the comparison. The expected answer has to come from somewhere independent — the plain form the input was supposed to become — or the grader and the contestant are the same entity.&lt;/p&gt;
&lt;p&gt;And a small table of recommended tools in the guide made claims nobody had verified. Those rows now carry an explicit unverified label, and the verified rows name the date they were checked. Publishing &amp;quot;we believe this&amp;quot; as though it were &amp;quot;we measured this&amp;quot; is its own species of cannot-fail — a claim with no failure condition attached.&lt;/p&gt;
&lt;p&gt;One subtler twin surfaced in the same review: a check elsewhere in the workflow required a phrase to appear, where the ruling demanded it appear exactly once. &amp;quot;Must appear&amp;quot; cannot see a phrase appear twice — a presence check cannot count. When the requirement is &amp;quot;exactly one,&amp;quot; the check has to count, not just look.&lt;/p&gt;
&lt;h2&gt;The honest ledger&lt;/h2&gt;
&lt;p&gt;The full count, because this post exists to practice what it preaches. Seventeen review passes across the panel and the fix round. One genuine defect at our highest grade — what the ledger calls a P1 — the example that could never fail. One grade below it, a P2: the self-grading comparison. Roughly ten smaller findings. And zero follow-up issues filed afterward — every finding was fixed before a single reader saw the guide.&lt;/p&gt;
&lt;p&gt;That last number is the one that matters. The point of the gauntlet is not that the guide was perfect — it visibly was not — but that its defects were caught where only reviewers could see them.&lt;/p&gt;
&lt;h2&gt;Guidance is graded like code&lt;/h2&gt;
&lt;p&gt;The lesson underneath all of it: instructions you hand to an AI are graded like code. A worked example is not prose — it executes in the reader&#39;s head, and whatever it permits becomes the practice it teaches. A vacuous example is not a typo to shrug at; it is a defect in the teaching, with the same blast radius as a bug in the thing being taught.&lt;/p&gt;
&lt;p&gt;So the discipline we demand of tests now applies to the documents that teach them. Can this example fail? Does a do-nothing answer still pass it? Is the expected answer produced by the machinery being graded? Three questions, cheap to ask, expensive to skip — and they apply to any guidance you hand an AI, a contractor, or yourself.&lt;/p&gt;
&lt;p&gt;This post expands one thread of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the four days we spent shipping a trust checklist&lt;/a&gt; — the day-by-day account this story is a spoke of.&lt;/p&gt;
&lt;h2&gt;What to ask of your own tools&lt;/h2&gt;
&lt;p&gt;You never have to write a test to use this lesson. The next time an AI tool hands you a check, a score, a green &amp;quot;verified&amp;quot; — ask the one question this whole story hangs on: what would make this fail? If the honest answer is &amp;quot;nothing could,&amp;quot; you are looking at decoration wearing the costume of protection. If the honest answer is &amp;quot;we never checked,&amp;quot; you know what the check is worth.&lt;/p&gt;
&lt;p&gt;And when you write instructions for your own AI tools — a style guide, a checklist, an example of what good looks like — grade the instructions the way you would grade the work. An example that cannot be wrong teaches a standard that cannot be met.&lt;/p&gt;
&lt;p&gt;For the full technical post-mortem — every finding, every fix, the three checks we now run on any shipped example — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/knowledge-base/project/learnings/best-practices/2026-10-21-shipped-guidance-examples-must-survive-trivial-implementations-and-self-oracles.md&quot;&gt;the write-up on GitHub&lt;/a&gt;. And if this is the discipline you want wrapped around the AI work in your own company, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What does it mean for a test to be unable to fail?&lt;/h3&gt;
&lt;p&gt;It means the test was written so that every outcome counts as a pass — including a completely wrong answer. It looks like protection and behaves like decoration: it runs, it reports green, and it can never catch anything. The only test worth keeping is one that can show you a red light.&lt;/p&gt;
&lt;h3&gt;How can instructions to an AI be wrong if they read correctly?&lt;/h3&gt;
&lt;p&gt;Because an example in a guide is not prose — it executes in the reader&#39;s head. If the worked example permits a do-nothing answer, the reader learns to permit do-nothing work. A wrong-looking instruction is easy to catch; a plausible-sounding one that demands nothing is the dangerous kind.&lt;/p&gt;
&lt;h3&gt;What should I ask about the checks my AI tools rely on?&lt;/h3&gt;
&lt;p&gt;Ask what would make each check fail, and when it last did. A check with no failure condition is decoration; a claim nobody verified should be labelled as such; and the expected answer should come from somewhere independent of the machinery being graded. Whoever or whatever reviews the checks is the layer that makes &amp;quot;verified&amp;quot; mean something.&lt;/p&gt;
&lt;h3&gt;Does this apply to work that is not software?&lt;/h3&gt;
&lt;p&gt;Yes. Any instruction you hand off — to an AI, a freelancer, a template — is only as good as its failure condition. &amp;quot;The report should cover our metrics&amp;quot; cannot fail if any metrics count. &amp;quot;These three numbers must reconcile&amp;quot; can. The discipline is the same wherever the example lives.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What does it mean for a test to be unable to fail?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;It means the test was written so that every outcome counts as a pass — including a completely wrong answer. It looks like protection and behaves like decoration: it runs, it reports green, and it can never catch anything. The only test worth keeping is one that can show you a red light.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How can instructions to an AI be wrong if they read correctly?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Because an example in a guide is not prose — it executes in the reader&#39;s head. If the worked example permits a do-nothing answer, the reader learns to permit do-nothing work. A wrong-looking instruction is easy to catch; a plausible-sounding one that demands nothing is the dangerous kind.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What should I ask about the checks my AI tools rely on?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Ask what would make each check fail, and when it last did. A check with no failure condition is decoration; a claim nobody verified should be labelled as such; and the expected answer should come from somewhere independent of the machinery being graded. Whoever or whatever reviews the checks is the layer that makes &#92;&quot;verified&#92;&quot; mean something.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Does this apply to work that is not software?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes. Any instruction you hand off — to an AI, a freelancer, a template — is only as good as its failure condition. &#92;&quot;The report should cover our metrics&#92;&quot; cannot fail if any metrics count. &#92;&quot;These three numbers must reconcile&#92;&quot; can. The discipline is the same wherever the example lives.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>We Audited 102 Guardrails. Only Four Belong in Every Repository.</title>
    <link href="https://soleur.ai/blog/audited-102-guardrails-four-portable/" />
    <updated>2026-10-27T00:00:00Z</updated>
    <id>https://soleur.ai/blog/audited-102-guardrails-four-portable/</id>
    <content type="html">&lt;p&gt;Your AI writes code faster than you can read it. That is the deal you made, and most days it is a good one — until the question arrives that every founder eventually asks: what is actually checking this work?&lt;/p&gt;
&lt;p&gt;If you are running a company alone, the honest answer is usually &amp;quot;some automated checks I did not write and cannot see.&amp;quot; Maybe your AI tool set them up. Maybe a template did. Either way, you are trusting them the same way you trust the AI itself — on faith, and on the fact that nothing has gone visibly wrong yet.&lt;/p&gt;
&lt;p&gt;That faith deserves a receipt. So before we shipped any safety checks into the repositories our users work in, we audited our own. All of them. This post is what the audit found — and it is a useful story even if you never install a single check, because the question underneath it applies to every automated safeguard you rely on: can it prove it catches anything?&lt;/p&gt;
&lt;h2&gt;The audit: count your own first&lt;/h2&gt;
&lt;p&gt;Our own codebase has been accumulating automated checks for months — every mistake that ever got through left a check behind to stop it from happening twice. Before we told founders which of these belong in their repositories, we had to answer an uncomfortable question ourselves: how many of ours actually would?&lt;/p&gt;
&lt;p&gt;On October 21 we counted the whole corpus: 102 automated checks, in two families. Fifty-six of them are tripwires — they fire in the moment work is about to be saved, the last instant a mistake can still be stopped cheaply. The other forty-six are sweeps — they read the whole repository and flag anything that has drifted out of line.&lt;/p&gt;
&lt;p&gt;Then we graded every single one against a blunt question: would this mean anything in someone else&#39;s repository? Each check got one of three verdicts:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Portable as it stands&lt;/strong&gt; — the check would work in any repository with at most a path changed.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Portable after repackaging&lt;/strong&gt; — the idea is universal, but our copy is wired into machinery that does not exist anywhere else, so it would have to be rebuilt as a clean, self-contained version.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Welded to our machinery&lt;/strong&gt; — the check&#39;s whole purpose is ours: it enforces one of our own rules, reads one of our own file layouts, or watches a vendor in our own stack. In another repository it would check nothing that means anything.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;The honest count&lt;/h2&gt;
&lt;p&gt;Here is what the verdicts came back as:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Where the check runs&lt;/th&gt;
&lt;th&gt;Portable as it stands&lt;/th&gt;
&lt;th&gt;Portable after repackaging&lt;/th&gt;
&lt;th&gt;Welded to our machinery&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Tripwires that fire before work is saved (56 checks)&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;46&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Repo-wide sweeps (46 checks)&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;29&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Read that first column again. Of the fifty-six checks guarding the exact moment work gets saved — the highest-value position in the whole system — not one could be carried into another repository as it stood. Zero. Every one of them had grown roots into our machinery: our internal rules, our file layout, our own bookkeeping, our vendors.&lt;/p&gt;
&lt;p&gt;This is the part of the story most teams never publish, because the honest version is unflattering. The reusable fraction of a mature safety practice turns out to be startlingly small — not because the checks are bad, but because a check earns its place by knowing the codebase it guards. Knowledge of your repository is precisely what does not transfer.&lt;/p&gt;
&lt;p&gt;From the portable candidates we shipped four — the four with the highest value for any codebase, each rebuilt as a clean, self-contained template. Most teams in our position would have promised a hundred and shipped four quietly. We would rather show the count.&lt;/p&gt;
&lt;h2&gt;The four that belong in every repository&lt;/h2&gt;
&lt;p&gt;What survived the audit, translated into what each one does for you:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;A scan for secrets left in work about to be saved.&lt;/strong&gt; Passwords, keys, access credentials — the class of mistake that is free to catch and catastrophic to miss. A widely used secrets scanner reads what is about to be saved and flags anything that looks like a credential.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A check that work never lands straight on the main line.&lt;/strong&gt; Every change arrives through a side branch where it can be reviewed — yours, your tools&#39;, anyone&#39;s. Direct saves to the project&#39;s backbone get refused.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A check that unfinished work is never quietly shelved and forgotten.&lt;/strong&gt; &amp;quot;Tuck this away for later&amp;quot; is how work-in-progress silently dies. The check catches the shelving move at the moment it happens and asks whether you meant it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A check that a test suite with nothing left to run cannot pass silently.&lt;/strong&gt; The guard that guards the guards: it watches whether your other checks are still wired into the machinery that runs them, because a check that exists but never runs is worse than no check — it looks like coverage.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Notice what the four share: none of them knows anything about our company, our rules, or our stack. Each encodes a mistake every software project can make. That is what earned them a place on the list — and it is why the list is so short.&lt;/p&gt;
&lt;h2&gt;They arrive as warnings, not walls&lt;/h2&gt;
&lt;p&gt;Every check that ships installs in advisory mode first: it runs the identical tests it always would, but it warns instead of blocks. Same evidence, different verb.&lt;/p&gt;
&lt;p&gt;That is deliberate, and it is the part most safety tooling skips. A check you did not write, dropping into your repository and refusing to let work save, is a frustration waiting to happen — and worse, it teaches you to distrust the whole idea of checks. So each one arrives watching and reporting. Promoting a check from warning to blocking is a one-line change, and it requires at least one clean observed run first: proof that the check behaves correctly in your repository before it is allowed to refuse anything there.&lt;/p&gt;
&lt;h2&gt;Proven before it is allowed to guard anything&lt;/h2&gt;
&lt;p&gt;Here is the contract underneath all of it — the part we think matters most.&lt;/p&gt;
&lt;p&gt;When one of these checks installs, it does not get trusted. It gets rehearsed. The installer makes a throwaway copy of the repository, plants a violation the check is supposed to catch, and watches: the check must go off. Then the violation is removed, and the check must go quiet. Only after it has proven both — that it fires when it should and stays silent when it should — is it written into your repository for real.&lt;/p&gt;
&lt;p&gt;Two more refusals sit underneath that. The installer will not overwrite anything already sitting where a check would go — there is no force flag, no &amp;quot;clobber and continue.&amp;quot; And it refuses to rehearse inside the real repository it is meant to protect: the rehearsal happens in the throwaway copy, or it does not happen, and the installer says so out loud rather than skipping the proof quietly.&lt;/p&gt;
&lt;p&gt;A guard that cannot be proven to fire is a document, not a guard.&lt;/p&gt;
&lt;h2&gt;What the review found in the machinery itself&lt;/h2&gt;
&lt;p&gt;Because the checks are machinery, the checks had defects too. This is the part of the build log nobody else publishes, so we will:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;A branch name containing a certain punctuation mark could slip past the main-line check&#39;s pattern. Under blocking mode the hole ran the worst possible way: instead of stopping nothing, it would have waved everything through.&lt;/li&gt;
&lt;li&gt;Each rehearsal overwrote the previous run&#39;s error log, so a check that failed could erase the evidence of why it failed.&lt;/li&gt;
&lt;li&gt;The remover reported tidying up machinery that was never there — a small lie in the cleanup report, but a lie in a report is a lie in a report.&lt;/li&gt;
&lt;li&gt;Several of the refusal cases had no automated test pinned on them — the rules existed, but nothing verified they still worked.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Every one of those was filed and fixed before release. And the sharpest finding was not any single defect: three different surfaces that record derived facts about the codebase — an architecture diagram, a test scorecard, and a headline number in the project&#39;s public description — all went red for the same reason. One regeneration step had run on a subset of what it was meant to update. Three red lights, one root cause, one fix. It is the audit&#39;s lesson again, viewed from inside: the machinery that verifies things needs verifying itself.&lt;/p&gt;
&lt;h2&gt;The lesson worth taking&lt;/h2&gt;
&lt;p&gt;If you take one idea from the count, take this one: the portable fraction of a real safety corpus is far smaller than anyone assumes, and you only learn it by measuring. We would have guessed half our checks were generic. The audit said fewer than a third were even candidates — and only four were ready to stand alone in somebody else&#39;s repository.&lt;/p&gt;
&lt;p&gt;The same question transfers to whatever automated checks protect your own work today — in your AI tools, your automations, your bookkeeping integrations. Not &amp;quot;does a check exist?&amp;quot; but &amp;quot;when did it last prove it catches anything?&amp;quot; A safeguard that cannot show you a real catch is a decoration with a green light.&lt;/p&gt;
&lt;p&gt;This post expands one section of &lt;a href=&quot;https://soleur.ai/blog/agentic-verification-checklist-shipped/&quot;&gt;the build log we published last week&lt;/a&gt; — the day-by-day account of shipping the trust machinery this audit was part of.&lt;/p&gt;
&lt;p&gt;For the full technical write-up — every check named, every verdict explained — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/blob/main/plugins/soleur/skills/verification-scaffold/references/guardrail-audit.md&quot;&gt;the audit itself, on GitHub&lt;/a&gt;. And if this is the kind of safety you want wrapped around your own AI work, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What is a guardrail for AI-written code?&lt;/h3&gt;
&lt;p&gt;An automated check that watches the work instead of trusting it — a scan that fires before a mistake is saved, or a sweep that notices when something has drifted out of line. The point is not to slow the AI down; it is to give you evidence you can review instead of work you have to re-do.&lt;/p&gt;
&lt;h3&gt;Why did only four of the 102 checks qualify?&lt;/h3&gt;
&lt;p&gt;Because a check earns its keep by knowing the codebase it guards — our rules, our file layouts, our vendors — and that knowledge is exactly what does not transfer. Fewer than a third of our checks were even candidates for portability, and the four that shipped were the highest-value ones that could be rebuilt as clean, self-contained templates and proven on a throwaway copy before activation.&lt;/p&gt;
&lt;h3&gt;Can an automated check itself be wrong?&lt;/h3&gt;
&lt;p&gt;Yes — ours were, and that is the point. The review found a check that could wave bad work through, a log that erased its own evidence, and a report that claimed cleanup it never did. Verification machinery needs verification, which is why every check we ship has to prove it catches a planted violation before it is allowed to guard anything.&lt;/p&gt;
&lt;h3&gt;Do these checks replace a human reviewing the work?&lt;/h3&gt;
&lt;p&gt;No — they change what reviewing costs. The checks absorb the mechanical watching (credentials left in code, changes landing on the main line, forgotten work, test suites that quietly stopped running) so your judgment lands on the decisions that need it. You stay the reviewer; you stop being the tripwire.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is a guardrail for AI-written code?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;An automated check that watches the work instead of trusting it — a scan that fires before a mistake is saved, or a sweep that notices when something has drifted out of line. The point is not to slow the AI down; it is to give you evidence you can review instead of work you have to re-do.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Why did only four of the 102 checks qualify?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Because a check earns its keep by knowing the codebase it guards — our rules, our file layouts, our vendors — and that knowledge is exactly what does not transfer. Fewer than a third of our checks were even candidates for portability, and the four that shipped were the highest-value ones that could be rebuilt as clean, self-contained templates and proven on a throwaway copy before activation.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Can an automated check itself be wrong?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes — ours were, and that is the point. The review found a check that could wave bad work through, a log that erased its own evidence, and a report that claimed cleanup it never did. Verification machinery needs verification, which is why every check we ship has to prove it catches a planted violation before it is allowed to guard anything.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Do these checks replace a human reviewing the work?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No — they change what reviewing costs. The checks absorb the mechanical watching (credentials left in code, changes landing on the main line, forgotten work, test suites that quietly stopped running) so your judgment lands on the decisions that need it. You stay the reviewer; you stop being the tripwire.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Run Your AI Company on Google Antigravity Without Losing Your Business Memory</title>
    <link href="https://soleur.ai/blog/run-your-ai-company-on-google-antigravity/" />
    <updated>2026-10-26T00:00:00Z</updated>
    <id>https://soleur.ai/blog/run-your-ai-company-on-google-antigravity/</id>
    <content type="html">&lt;p&gt;Running a company alone means making hundreds of decisions every week. As AI agentic coding tools evolve, solo founders often find themselves facing a frustrating dilemma: every time a faster model or a more powerful harness launches, switching means leaving your company&#39;s context and workflow behind.&lt;/p&gt;
&lt;p&gt;Your marketing guidelines, your legal guardrails, your financial models, and your product roadmap are scattered across chat tabs. You start from scratch on every new tool.&lt;/p&gt;
&lt;p&gt;Soleur was built to solve this problem. Today, we are bringing Soleur natively to &lt;a href=&quot;https://github.com/google/antigravity&quot;&gt;Google Antigravity CLI&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Your Company&#39;s Memory Stays Where It Belongs: With You&lt;/h2&gt;
&lt;p&gt;Soleur turns a solo founder into a full organization — AI specialists across all eight departments: engineering, marketing, legal, finance, operations, product, sales, and support, all sharing one persistent memory.&lt;/p&gt;
&lt;p&gt;Instead of locking your business context inside one vendor&#39;s proprietary system, Soleur stores your company&#39;s institutional memory as plain files in your own repository. Every decision, brand choice, and engineering standard is remembered and shared across all departments.&lt;/p&gt;
&lt;p&gt;With native Google Antigravity CLI support, you can now run that entire organization on Google&#39;s agentic harness:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;No context loss:&lt;/strong&gt; Your marketing agent knows your brand guide. Your legal agent references your compliance standards. Your engineering team follows your architectural rules.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Asynchronous delegation:&lt;/strong&gt; Complex tasks run in the background with native reactive wakeups, alerting you only when input is needed or work is ready.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Freedom to choose:&lt;/strong&gt; Run on Claude Code, Grok Build, Codex, Devin CLI, OpenCode, or Google Antigravity CLI — and switch between them without losing a single past decision.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Running a company alone shouldn&#39;t mean doing everything alone. And it shouldn&#39;t mean being locked into a single AI vendor.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;Does Soleur run on Google Antigravity CLI?&lt;/h3&gt;
&lt;p&gt;Yes. Soleur installs natively on Google Antigravity CLI (&lt;code&gt;agy&lt;/code&gt;), allowing you to run your full AI team across engineering, marketing, legal, finance, operations, product, sales, and support directly within Google&#39;s agentic harness.&lt;/p&gt;
&lt;h3&gt;Which models can I use with Google Antigravity?&lt;/h3&gt;
&lt;p&gt;Google Antigravity CLI connects natively to Google&#39;s Gemini models, including Gemini 2.5 Pro and Gemini Flash, giving your AI specialists fast inference and large context windows while keeping your business memory stored locally.&lt;/p&gt;
&lt;h3&gt;Do I lose my company&#39;s memory if I switch tools?&lt;/h3&gt;
&lt;p&gt;No. Your company&#39;s memory lives in plain markdown files inside your own Git repository. You can run Soleur on Google Antigravity CLI today and switch between any supported coding harness without losing your business context, brand guidelines, or operational history.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Does Soleur run on Google Antigravity CLI?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes. Soleur installs natively on Google Antigravity CLI (agy), allowing you to run your full AI team across engineering, marketing, legal, finance, operations, product, sales, and support directly within Google&#39;s agentic harness.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Which models can I use with Google Antigravity?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Google Antigravity CLI connects natively to Google&#39;s Gemini models, including Gemini 2.5 Pro and Gemini Flash, giving your AI specialists fast inference and large context windows while keeping your business memory stored locally.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Do I lose my company&#39;s memory if I switch tools?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. Your company&#39;s memory lives in plain markdown files inside your own Git repository. You can run Soleur on Google Antigravity CLI today and switch between any supported coding harness without losing your business context, brand guidelines, or operational history.&quot;
      }
    }
  ]
}
&lt;/script&gt;
&lt;hr&gt;
&lt;p&gt;For the technical write-up, see &lt;a href=&quot;https://github.com/jikig-ai/soleur/pull/10362&quot;&gt;how we built Antigravity CLI support&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Ready to run your company with an AI team that remembers? &lt;a href=&quot;https://soleur.ai/getting-started/#google-antigravity-cli&quot;&gt;Get started with Soleur&lt;/a&gt; or &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt;.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>What Happens to Your Business When Your AI Provider Changes the Rules?</title>
    <link href="https://soleur.ai/blog/own-your-ai-team-without-vendor-lock-in/" />
    <updated>2026-10-23T00:00:00Z</updated>
    <id>https://soleur.ai/blog/own-your-ai-team-without-vendor-lock-in/</id>
    <content type="html">&lt;p&gt;You are three months into building your company. Your AI assistant drafts your contracts, plans your marketing campaigns, remembers your customers, and has learned your brand voice from a hundred conversations. Then on a Tuesday morning, the provider changes its pricing model, throttles your account, or swaps the underlying model — and every workflow you rely on breaks at once.&lt;/p&gt;
&lt;p&gt;The contracts it drafted were good. The campaigns it planned were working. The institutional memory it accumulated was real. But all of it lived inside one company&#39;s closed system, and you had no way to take it with you.&lt;/p&gt;
&lt;p&gt;This is the risk most founders do not see until it is too late.&lt;/p&gt;
&lt;h2&gt;The Hidden Risk of Renting Your Company&#39;s Brain&lt;/h2&gt;
&lt;p&gt;Every time you use an AI assistant for business work — writing a privacy policy, structuring a financial model, planning a product roadmap — you are not just generating output. You are accumulating institutional knowledge. Your brand voice sharpens. Your legal posture becomes consistent. Your engineering conventions solidify. Your competitive intelligence deepens.&lt;/p&gt;
&lt;p&gt;That knowledge has value. It is the difference between a founder who starts every prompt from scratch and one whose AI partner already understands the business.&lt;/p&gt;
&lt;p&gt;But when that knowledge lives inside a single vendor&#39;s chat history or proprietary workspace, you do not own it. You are renting your company&#39;s brain. The vendor can change the terms, raise the price, deprecate the model, or shut down the service — and your institutional memory goes with it.&lt;/p&gt;
&lt;p&gt;For a solo founder, this is a familiar SaaS risk: the tool you depend on can become the tool that controls you.&lt;/p&gt;
&lt;h2&gt;What Changes When You Separate Your Team from the Engine&lt;/h2&gt;
&lt;p&gt;Soleur was built on a different premise: your company&#39;s institutional memory should live in plain files inside your own repository, readable by any tool, and your AI team should work across model providers supported by your chosen harness.&lt;/p&gt;
&lt;p&gt;Soleur gives a single founder the leverage of a full organization — eight departments covering engineering, marketing, legal, finance, operations, product, sales, and support. Each department has specialist AI agents that share one knowledge base. Every decision, brand guide, legal audit, financial model, and engineering plan is stored as a plain document in your own Git repository.&lt;/p&gt;
&lt;p&gt;The founder still decides everything that matters: strategy, approvals, budget, and which model provider to trust. Soleur&#39;s AI specialists handle the execution — drafting, planning, reviewing, and documenting across every department — while the institutional memory compounds with every session.&lt;/p&gt;
&lt;p&gt;The key architectural decision is separation. Your company&#39;s knowledge and your AI team&#39;s workflows are not tied to any single model or harness. They live in your repository as plain files. You can run Soleur on Claude Code, Grok Build, Codex, Devin CLI, OpenCode, or Google Antigravity CLI — and switch between them without losing a single past decision.&lt;/p&gt;
&lt;h2&gt;Why Open Weights Models Change the Equation&lt;/h2&gt;
&lt;p&gt;For founders who want maximum control, open-weights models change the equation. &lt;a href=&quot;https://opencode.ai/docs/providers/&quot;&gt;OpenCode&#39;s provider documentation&lt;/a&gt; lists more than 75 providers and support for running local models through Ollama, llama.cpp, and LM Studio. You can run Soleur workflows on open-weights models configured through those providers, or through other providers supported by your harness.&lt;/p&gt;
&lt;p&gt;This means three things:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Less model-provider lock-in.&lt;/strong&gt; Your brand voice, legal posture, financial models, and engineering conventions are not trapped inside one provider&#39;s ecosystem. They live in your repository.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Cost control.&lt;/strong&gt; You can choose a provider and model tier that fit your budget — from frontier APIs to self-hosted open-weights inference — without losing your file-based knowledge base.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Operational sovereignty.&lt;/strong&gt; If a provider changes its terms, raises prices, or goes offline, you can switch to another supported provider and your file-based knowledge base remains intact and available.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Soleur&#39;s knowledge base is the same regardless of which model powers the conversation. The memory is yours. The team is yours. The model is your choice.&lt;/p&gt;
&lt;h2&gt;Plain Files You Can Read, Move, and Keep Forever&lt;/h2&gt;
&lt;p&gt;Your knowledge-base artifacts — brand guides, legal documents, financial models, engineering plans, competitive analyses, and marketing campaigns — are stored as plain documents in your repository. You can read them, edit them, version them, and move them between projects.&lt;/p&gt;
&lt;p&gt;This is not a feature. It is the architecture. Soleur does not store your company&#39;s knowledge in a proprietary database or a vendor-locked workspace. It stores it in files you can open in any text editor, search with standard tools, and back up to any cloud provider.&lt;/p&gt;
&lt;p&gt;When you switch AI providers, your company&#39;s file-based memory comes with you. When you hire a human contractor, they can read the same files. Committed changes to those records are auditable in your Git history.&lt;/p&gt;
&lt;h2&gt;What This Means for Solo Founders&lt;/h2&gt;
&lt;p&gt;The solo founder&#39;s biggest risk is not running out of ideas or running out of time. It is building a company whose institutional memory is held hostage by a single vendor.&lt;/p&gt;
&lt;p&gt;Soleur eliminates that risk by separating three things that most AI tools bundle together:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;The memory&lt;/strong&gt; (your company&#39;s knowledge base) — owned by you, stored in your repo.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The team&lt;/strong&gt; (67 specialist agents across 8 departments) — runs on any supported harness.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The engine&lt;/strong&gt; (the underlying AI model) — your choice, switchable at any time.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You get the leverage of a full organization without the dependency on any single provider. Your AI team compounds its knowledge with every session, and that knowledge stays with you regardless of which model or harness you use tomorrow.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What happens to my company&#39;s memory if I switch AI models in Soleur?&lt;/h3&gt;
&lt;p&gt;Your company&#39;s memory lives in plain files inside your own Git repository. Switching models or harnesses does not affect your knowledge base — every decision, brand guide, legal document, and engineering plan remains intact and readable by your new AI team. The memory is yours, not the model provider&#39;s.&lt;/p&gt;
&lt;h3&gt;Do I need to run my own servers to use open-weights models with my AI team?&lt;/h3&gt;
&lt;p&gt;No. Soleur runs on supported harnesses — Claude Code, Grok Build, Codex, Devin CLI, OpenCode, and Google Antigravity CLI. Each harness connects to providers it supports. OpenCode&#39;s provider documentation lists support for local models through Ollama, llama.cpp, and LM Studio, as well as hosted providers. You can use frontier APIs, bring-your-own-key providers, or self-hosted open-weights models depending on your preference and technical comfort. Soleur does not require you to operate Soleur-hosted infrastructure; self-hosted model inference has its own requirements.&lt;/p&gt;
&lt;h3&gt;How does Soleur keep my marketing, legal, and finance specialists aligned across different AI providers?&lt;/h3&gt;
&lt;p&gt;All specialists share the same knowledge base — your company&#39;s plain-file repository. When you switch providers, the new model can read the same brand guide, legal posture, financial models, and engineering conventions. The institutional memory is stored in files, not in the model&#39;s context window, so it remains available to your specialists regardless of which supported provider powers the conversation.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What happens to my company&#39;s memory if I switch AI models in Soleur?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Your company&#39;s memory lives in plain files inside your own Git repository. Switching models or harnesses does not affect your knowledge base — every decision, brand guide, legal document, and engineering plan remains intact and readable by your new AI team. The memory is yours, not the model provider&#39;s.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Do I need to run my own servers to use open-weights models with my AI team?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. Soleur runs on supported harnesses — Claude Code, Grok Build, Codex, Devin CLI, OpenCode, and Google Antigravity CLI. Each harness connects to providers it supports. OpenCode&#39;s provider documentation lists support for local models through Ollama, llama.cpp, and LM Studio, as well as hosted providers. You can use frontier APIs, bring-your-own-key providers, or self-hosted open-weights models depending on your preference and technical comfort. Soleur does not require you to operate Soleur-hosted infrastructure; self-hosted model inference has its own requirements.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How does Soleur keep my marketing, legal, and finance specialists aligned across different AI providers?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;All specialists share the same knowledge base — your company&#39;s plain-file repository. When you switch providers, the new model can read the same brand guide, legal posture, financial models, and engineering conventions. The institutional memory is stored in files, not in the model&#39;s context window, so it remains available to your specialists regardless of which supported provider powers the conversation.&quot;
      }
    }
  ]
}
&lt;/script&gt;
&lt;hr&gt;
&lt;p&gt;For the technical write-up on how we built the OpenCode runtime adapter and installer, see &lt;a href=&quot;https://github.com/jikig-ai/soleur/pull/10723&quot;&gt;how we built the OpenCode harness support&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Ready to build a company you actually own? &lt;a href=&quot;https://soleur.ai/getting-started/&quot;&gt;Get started with Soleur&lt;/a&gt;.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>A Checklist for Trusting AI Work Was Just Published. We Shipped All of It.</title>
    <link href="https://soleur.ai/blog/agentic-verification-checklist-shipped/" />
    <updated>2026-10-23T00:00:00Z</updated>
    <id>https://soleur.ai/blog/agentic-verification-checklist-shipped/</id>
    <content type="html">&lt;p&gt;If you use AI tools to run parts of your business, you have already made a quiet bargain: you stopped checking every line, because checking every line is the job you were trying to stop doing. The leverage is real. So is the question underneath it — what if it&#39;s wrong?&lt;/p&gt;
&lt;p&gt;That question is the number-one hesitation founders have about delegating to AI, and it deserves a better answer than &amp;quot;trust me.&amp;quot; It just got a much better answer. Addy Osmani — an engineering leader whose writing on AI-assisted development is among the most followed in the industry — published &lt;a href=&quot;https://addyo.substack.com/p/agentic-verification&quot;&gt;a checklist for agentic verification&lt;/a&gt;: the practice of giving an AI system ways to check its own work, so a human reviews evidence instead of re-doing the work.&lt;/p&gt;
&lt;p&gt;We read it the day it came out, measured ourselves against every row, and shipped what was missing. Seven changes in four days — some in our own codebase, most in the workflows we hand to the founders who use Soleur. But the speed is not the story worth telling. The story is that almost every piece of verification machinery we shipped arrived with a real defect in the machinery itself — caught by reviewers before a single user saw it. A checklist for trusting AI work turns out to be the easy half. The hard half is trusting the verifiers.&lt;/p&gt;
&lt;h2&gt;What the checklist actually asks for&lt;/h2&gt;
&lt;p&gt;Osmani&#39;s argument in one sentence: the bottleneck has moved from producing work to knowing the work is correct, so the durable investment is giving the agent ways to verify its own output. The checklist is concrete:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Test what a real user sees.&lt;/strong&gt; End-to-end checks that sign up, click through, and confirm the outcome — the ground truth a unit check can share a misunderstanding with and still pass.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Test the rules, not the examples.&lt;/strong&gt; Property-based testing: instead of checking a handful of hand-picked inputs, you state a rule that must always hold — &amp;quot;running this twice changes nothing the second time&amp;quot; — and a generator throws thousands of random inputs at it trying to break it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Keep a feature map.&lt;/strong&gt; A written inventory of what your product does from the user&#39;s point of view — what exists, how a real person reaches it, what they should see — so an agent driving the product works from a map, not a guess.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;When you replace something, compare old and new.&lt;/strong&gt; Differential testing: run the old system and the new system side by side over the same inputs, and treat every difference as either a bug or a deliberate, documented change.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Make the loop fast and predictable.&lt;/strong&gt; Checks that are slow or flaky teach the agent the wrong lesson — it learns to retry instead of fix. A flaky check gets isolated and removed, not tolerated.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Write the check before the work.&lt;/strong&gt; Define what &amp;quot;correct&amp;quot; means first; the agent iterates toward a target instead of a vibe.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Make taste compound.&lt;/strong&gt; When the agent gets something wrong, write the lesson down as a permanent, automatic check so it never happens twice.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Measured honestly, we already had three rows solid — the user-perspective test suite, the check-before-work gate, and a written-down-lessons practice that has grown into more than a hundred automatic checks on our own repository. Four rows were missing or only lived inside our walls, invisible to the people we build for. So we shipped all four — and packaged each one so a founder&#39;s own codebase gets the same benefit, not just ours.&lt;/p&gt;
&lt;h2&gt;The map that guards itself — and the four ways the guard was wrong&lt;/h2&gt;
&lt;p&gt;The feature map row sounds like paperwork. It is actually the difference between an AI helper that navigates your product and one that bluffs. Our platform has 32 distinct pages and screens a visitor can reach; the map records each one — how a real user gets there and what they should find — and an automatic guard fails the build the moment the map and the product disagree, in either direction.&lt;/p&gt;
&lt;p&gt;Our review panel found four defects in that guard before it shipped:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The counter that enumerates the screens &lt;strong&gt;silently missed the homepage&lt;/strong&gt; — it counted 31 of 32, because the pattern it used to list pages skipped the one sitting at the front door. A check that under-counts looks green forever.&lt;/li&gt;
&lt;li&gt;A stray line of descriptive text could &lt;strong&gt;satisfy the completeness check&lt;/strong&gt; without the required information actually being present — presence is not conformance.&lt;/li&gt;
&lt;li&gt;An example shown inside the map document — included to teach the format — could be &lt;strong&gt;counted as a real entry&lt;/strong&gt;, certifying a page nobody had verified.&lt;/li&gt;
&lt;li&gt;The guard kept &lt;strong&gt;two separate scoreboards&lt;/strong&gt; for &amp;quot;did everything pass,&amp;quot; and a future check could report a problem and still come back green.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Four ways for &amp;quot;the map is correct&amp;quot; to be a lie, each subtle, each caught before anyone relied on it. This became the theme of the whole four days.&lt;/p&gt;
&lt;p&gt;We wrote up &lt;a href=&quot;https://soleur.ai/blog/map-that-cant-drift/&quot;&gt;the guard that keeps the map honest&lt;/a&gt; — the four defects it caught in itself, and what a coverage check has to prove — as a companion to this post. And for your own agent, &lt;a href=&quot;https://soleur.ai/blog/map-that-only-records-what-the-agent-saw/&quot;&gt;the convention that lets it keep an honest map&lt;/a&gt; is written up as a companion too — what the map records, where it lives, and how it stays true.&lt;/p&gt;
&lt;h2&gt;The test that found a bug on day one — and the example that could never fail&lt;/h2&gt;
&lt;p&gt;Property-based testing is the row most teams skip, because &amp;quot;generate thousands of random inputs&amp;quot; sounds exotic. It is not. You write one sentence — &amp;quot;cleaning up a value twice must give the same answer as cleaning it once&amp;quot; — and a machine spends its afternoon trying to prove you wrong.&lt;/p&gt;
&lt;p&gt;The very first generated test we ran caught a real bug on day one. A small utility that tidies repository addresses — trimming spaces, stripping a trailing suffix — could leave an invisible trailing space behind. Run it on certain inputs, get a subtly different answer, run it again, get a third. The documentation said running it twice was safe; the code disagreed. And once we looked, the same defect lived in a second place where the same cleanup logic ran. No human would have hand-picked that input. A generator finds it immediately because it does not share our imagination about what &amp;quot;reasonable input&amp;quot; looks like.&lt;/p&gt;
&lt;p&gt;Then the row taught its own second lesson. When we wrote the guide that teaches our users&#39; workflows to do the same thing, three reviewers independently flagged the guide&#39;s own worked example: it was written so that every possible answer — including a completely wrong one — would pass it. An example that cannot fail teaches a practice that cannot catch anything. And after we fixed that, the fix-round found a second layer: a do-nothing program still passed the improved example. Teaching material gets graded like code now, because to a reader it is code.&lt;/p&gt;
&lt;p&gt;We wrote up &lt;a href=&quot;https://soleur.ai/blog/property-test-cannot-fail-teaches-nothing/&quot;&gt;the full post-mortem&lt;/a&gt; — how the example shipped, what the reviewers caught, and what it changed — as a companion to this post. We also wrote up &lt;a href=&quot;https://soleur.ai/blog/first-property-test-found-real-bug/&quot;&gt;the day-one bug itself&lt;/a&gt; — the input nobody would have hand-picked — as a companion to this post.&lt;/p&gt;
&lt;h2&gt;Old versus new, side by side&lt;/h2&gt;
&lt;p&gt;The differential-testing row exists because of a failure mode anyone who has replaced a system knows: the new version passes its own tests while quietly diverging from the old one in ways nobody documented. Users never relied on the specification; they relied on what the old thing actually did.&lt;/p&gt;
&lt;p&gt;The answer is to treat the old system as the referee: feed both versions the same inputs and account for every difference. We shipped that practice into the workflow our users get — any change that moves data through a new path now has to show before-and-after evidence, born from an incident class we had measured where a change applied cleanly and broke everything downstream of it. The companion rule the reviews surfaced: a check that cannot go red is not coverage. If your comparison can only ever report success, it is a formality wearing a seatbelt costume. We wrote up &lt;a href=&quot;https://soleur.ai/blog/test-the-migration-not-just-the-schema/&quot;&gt;the differential-testing practice&lt;/a&gt; — the old system as referee, before-and-after evidence for every change — as a companion to this post.&lt;/p&gt;
&lt;h2&gt;A pass on retry is not a pass&lt;/h2&gt;
&lt;p&gt;Here is a failure mode unique to AI-driven work. When a check fails sometimes and passes other times with nothing changed, a human knows to be suspicious. An agent learns something worse: retrying is a strategy that works. Osmani&#39;s checklist is blunt about it — a flaky check left in the main suite is worse than no check, because it quietly teaches the whole system that &amp;quot;red&amp;quot; is negotiable.&lt;/p&gt;
&lt;p&gt;So we shipped a quarantine rule into the fix-loop workflow users get: every newly failed check gets exactly one isolated re-run, and a pass-on-retry on unchanged work is filed as a flake to be quarantined and tracked — never reported as fixed.&lt;/p&gt;
&lt;p&gt;And here the reviews caught the sharpest defect of the four days. The rule&#39;s own &amp;quot;all clear&amp;quot; row — the line meant to mean &amp;quot;everything that failed turned out to be a flake&amp;quot; — could fire on a run that produced no readable results at all. Zero quarantines, read as green. The success row proved nothing, which is exactly the failure mode the row existed to prevent. A second reviewer found a warning flag whose trigger was empty by construction — it compared the failing check against a list of things the workflow never touches, so it could never be true. Decoration that reads as coverage. We wrote up &lt;a href=&quot;https://soleur.ai/blog/flaky-test-passed-on-retry-quarantine/&quot;&gt;the quarantine protocol&lt;/a&gt; — one isolated re-run, then the file — as a companion to this post.&lt;/p&gt;
&lt;h2&gt;Safety rails you can install&lt;/h2&gt;
&lt;p&gt;The last row is the one closest to the founder&#39;s version of the problem. We run more than a hundred automated checks on our own codebase — the accumulated taste of every mistake that ever got through. Those checks are why our platform&#39;s AI work is trustworthy. But a founder&#39;s codebase is not our codebase, so we asked the honest question: how much of this is actually portable?&lt;/p&gt;
&lt;p&gt;We audited all 102 of our automated checks one by one. The answer: four. Four were generic enough to belong in any repository — a scan for accidentally committed secrets, a check that work never lands straight on the main line, a check against stashing away unfinished work, and a check that catches tests abandoned with nothing left to run. Nearly everything a mature verification system accumulates turns out to be welded to the machinery around it. The audit is the honest part of the story — most teams would have promised a hundred and shipped four quietly.&lt;/p&gt;
&lt;p&gt;The four that shipped each have to prove themselves before they are allowed to guard anything. Installing one triggers a rehearsal on a throwaway copy of the repository: the check must catch a real planted violation, then go quiet when the violation is removed. A guard that cannot be proven to fire is a document, not a guard — and it installs nothing if something already sits where it would go, rather than quietly overwriting what you have. We wrote up &lt;a href=&quot;https://soleur.ai/blog/audited-102-guardrails-four-portable/&quot;&gt;the audit&#39;s full story&lt;/a&gt; — every check classified, every verdict explained — as a companion to this post.&lt;/p&gt;
&lt;h2&gt;The real lesson: verify the verifiers&lt;/h2&gt;
&lt;p&gt;Step back and count. The feature map&#39;s guard shipped with four defects. The testing guide&#39;s own example could never fail — and its first fix still let a do-nothing program pass. The quarantine rule&#39;s &amp;quot;all clear&amp;quot; could fire on an unreadable run. The very first property test found a bug humans had been shipping for months. Every row of the checklist arrived with at least one way the verifier itself could lie.&lt;/p&gt;
&lt;p&gt;That is not an indictment of the checklist — it is the checklist&#39;s deepest row, the one underneath the others. Verification machinery is machinery. It has defects like anything else we build, and a defect in a verifier is worse than a defect in the product, because it converts &amp;quot;unverified&amp;quot; into &amp;quot;verified-looking.&amp;quot; A check that cannot fail is worse than no check at all.&lt;/p&gt;
&lt;p&gt;The reason our four days produced something trustworthy is not that we built fast. It is that everything we built got reviewed by people and panels whose only job was to ask &amp;quot;how could this be wrong?&amp;quot; — and the build log, defects and all, is public. Trust does not come from the checklist existing. It comes from the machinery being honest about where it almost failed.&lt;/p&gt;
&lt;h2&gt;What this means when the company is yours&lt;/h2&gt;
&lt;p&gt;You cannot personally check everything your AI tools produce. Nobody running a company alone can — that is the whole reason for the tools. So the question was never whether to trust; it is what trustworthy looks like.&lt;/p&gt;
&lt;p&gt;It looks like this: checks that behave like a real user, rules tested against thousands of cases you would never think to try, a map of the product that cannot quietly drift, proof when new replaces old, and flakes quarantined instead of retried into silence. And then — the part those four days taught us — reviewers who grade the checks themselves, so a green light means something was actually tested, not merely run.&lt;/p&gt;
&lt;p&gt;You stay the decision-maker. What changes is what deciding costs: evidence arrives already examined, and the examiners were examined too. If &amp;quot;what if it&#39;s wrong?&amp;quot; is the question keeping you from delegating real work to AI, &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;join the waitlist&lt;/a&gt; — this is the system we built to answer it.&lt;/p&gt;
&lt;p&gt;For the technical write-up — every defect named, every fix shown — see &lt;a href=&quot;https://github.com/jikig-ai/soleur/issues/10397&quot;&gt;the public tracking record on GitHub&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The pattern continued after this post: the same write-the-lesson-into-the-machinery habit went on to rebuild the suite&#39;s own scoreboard — read the sequel, &lt;a href=&quot;https://soleur.ai/blog/first-we-made-green-mean-something/&quot;&gt;First We Made &#39;Green&#39; Mean Something. Then We Made It Fast.&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Frequently Asked Questions&lt;/h2&gt;
&lt;h3&gt;What does &amp;quot;agentic verification&amp;quot; mean in plain terms?&lt;/h3&gt;
&lt;p&gt;It means giving an AI system ways to check its own work before a human sees it — tests that behave like real users, generated inputs that try to break the rules, comparisons against what worked before. Your job shifts from inspecting every line of output to reviewing evidence that the work already passed.&lt;/p&gt;
&lt;h3&gt;How can I tell whether the AI tools I use verify their work?&lt;/h3&gt;
&lt;p&gt;Ask four questions. Does it check outcomes the way a user would experience them, not just internal steps? Does it test rules against cases nobody hand-picked? Does a check that passes on retry count as fixed — or get quarantined as a flake? And who checks the checks — does anything review the verification machinery itself?&lt;/p&gt;
&lt;h3&gt;Isn&#39;t verifying the AI&#39;s work the same as doing the work twice?&lt;/h3&gt;
&lt;p&gt;No — verification is where machines are strongest and humans are weakest. Defining what &amp;quot;correct&amp;quot; means takes your judgment once; proving it against thousands of cases takes the machine seconds. You keep the deciding; the system absorbs the checking, which is the part that never scaled.&lt;/p&gt;
&lt;h3&gt;Do I need to be a programmer for this to matter to me?&lt;/h3&gt;
&lt;p&gt;No. The checklist is about software, but the principle is about delegation: anything you hand off — code, contracts, campaigns — is only as trustworthy as the way its output gets checked. When you evaluate an AI tool for your business, &amp;quot;how does it prove its work?&amp;quot; is the question that separates a system you can rely on from one you have to babysit.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What does &#92;&quot;agentic verification&#92;&quot; mean in plain terms?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;It means giving an AI system ways to check its own work before a human sees it — tests that behave like real users, generated inputs that try to break the rules, comparisons against what worked before. Your job shifts from inspecting every line of output to reviewing evidence that the work already passed.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How can I tell whether the AI tools I use verify their work?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Ask four questions. Does it check outcomes the way a user would experience them, not just internal steps? Does it test rules against cases nobody hand-picked? Does a check that passes on retry count as fixed — or get quarantined as a flake? And who checks the checks — does anything review the verification machinery itself?&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Isn&#39;t verifying the AI&#39;s work the same as doing the work twice?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No — verification is where machines are strongest and humans are weakest. Defining what &#92;&quot;correct&#92;&quot; means takes your judgment once; proving it against thousands of cases takes the machine seconds. You keep the deciding; the system absorbs the checking, which is the part that never scaled.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Do I need to be a programmer for this to matter to me?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. The checklist is about software, but the principle is about delegation: anything you hand off — code, contracts, campaigns — is only as trustworthy as the way its output gets checked. When you evaluate an AI tool for your business, &#92;&quot;how does it prove its work?&#92;&quot; is the question that separates a system you can rely on from one you have to babysit.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Your First Beta Testers Went Quiet. Now What?</title>
    <link href="https://soleur.ai/blog/beta-testers-went-quiet/" />
    <updated>2026-09-26T00:00:00Z</updated>
    <id>https://soleur.ai/blog/beta-testers-went-quiet/</id>
    <content type="html">&lt;p&gt;You posted in a few communities, and four people said they&#39;d try your product. That was three weeks ago. One of them sent a bug report on day two — and since then, nothing. You don&#39;t know if the other three opened the product once, got confused, and left, or whether they&#39;re quietly getting value every day.&lt;/p&gt;
&lt;p&gt;Asking feels desperate. Not asking feels like flying blind.&lt;/p&gt;
&lt;p&gt;This is the beta-test problem nobody warns you about: getting people to say yes is the easy part. Knowing what they actually did — and whether it worked — is the hard part, and most founders solve it with hope.&lt;/p&gt;
&lt;h2&gt;What a quiet beta really costs you&lt;/h2&gt;
&lt;p&gt;Every day a tester stays silent, you&#39;re making a decision in the dark. Is the onboarding broken? Is the product fine and they just got busy? If you guess wrong in either direction, you burn the scarcest resource an early product has: real users who volunteered.&lt;/p&gt;
&lt;p&gt;The founders who get this right don&#39;t guess. They design the beta so the answer arrives on its own — who signed up, who activated, who went quiet, and when to nudge them.&lt;/p&gt;
&lt;h2&gt;The two rules that make a beta program work&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;First: know who your testers are, in the product, not in your memory.&lt;/strong&gt; If your early users live in a spreadsheet or a Slack thread, you&#39;ll never measure them as a group. Give the product itself a way to say &amp;quot;this person is part of the test group&amp;quot; — then every question about the cohort is a question you can actually answer: how many activated, who&#39;s quiet, what they used.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Second: schedule the check-ins before they&#39;re needed.&lt;/strong&gt; A &amp;quot;remembered&amp;quot; follow-up is a follow-up that never happens. The nudge to a tester who&#39;s gone quiet has to be armed the day they start, so it fires even when you&#39;re heads-down on the next feature.&lt;/p&gt;
&lt;h2&gt;The part nobody expects: what the product should keep on the user&#39;s machine&lt;/h2&gt;
&lt;p&gt;Here&#39;s the uncomfortable version of measuring testers: the moment you want to know what your product did for someone, you&#39;re collecting information about them. Do it carelessly and you&#39;ve built a privacy problem inside a helpful feature.&lt;/p&gt;
&lt;p&gt;There&#39;s a better shape. Your product can keep a small, honest log on the &lt;em&gt;tester&#39;s own machine&lt;/em&gt; — a list of what it did and when, with nothing personal in it. No messages they typed, no file names, nothing identifying. When it&#39;s time for the two-week check-in, the tester pulls the log themselves and shares the summary. They can read every line first. They can turn it off with one switch.&lt;/p&gt;
&lt;p&gt;That&#39;s the deal a beta tester actually wants: &amp;quot;help me build this&amp;quot; without &amp;quot;watch me while I work.&amp;quot;&lt;/p&gt;
&lt;h2&gt;What we learned running our own&lt;/h2&gt;
&lt;p&gt;We rebuilt our early-tester program around exactly this. Testers are tagged in our product, so a real-time view of the group exists at all — who activated, who&#39;s been quiet for three days, who&#39;s untracked. Quiet testers get an automatic nudge; every tester gets a scheduled check-in. And our product writes a private, metadata-only activity log on each tester&#39;s machine — the first two-week checkpoint for our beta group will be based on that log, not on our memory of it.&lt;/p&gt;
&lt;p&gt;The honest part: this is a beta program, so we&#39;re testing the program too. The first checkpoint report will tell us whether testers actually found value — or whether the whole thing is a story we were telling ourselves.&lt;/p&gt;
&lt;h2&gt;The takeaway for your beta&lt;/h2&gt;
&lt;p&gt;You don&#39;t need our exact setup. You need the same three pieces, in whatever form fits your product:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;A way to mark who&#39;s in the test group&lt;/strong&gt; — a flag on their account, anything — so &amp;quot;how is the beta going?&amp;quot; is a question with an answer.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check-ins that are scheduled, not remembered&lt;/strong&gt; — a quiet-tester nudge that fires whether or not you think of it that week.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A respectful way to see what happened&lt;/strong&gt; — a log the user owns and can inspect, shared only when they choose.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;A beta that can&#39;t tell you anything is just a waiting list with extra steps. A beta that answers back is how a product learns to deserve its second hundred users.&lt;/p&gt;
&lt;p&gt;For the technical write-up, see &lt;a href=&quot;https://github.com/jikig-ai/soleur/pull/8868&quot;&gt;the pull request&lt;/a&gt;.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How many beta testers do I need for an early-stage product?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;For the first signal, three to five testers is enough — the goal at this stage is learning whether the product works for one real person, not statistics. A small group you can actually measure beats a large one you can&#39;t.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How do I get feedback from beta testers without pestering them?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Schedule the check-ins before you need them: a quiet-tester nudge after a few days of silence and a checkpoint after a couple of weeks. When the follow-up is armed at signup, it fires even when you&#39;re heads-down building.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Is it okay to log what beta testers do in my product?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Only if the tester stays in control. Keep the log on their machine, keep it to metadata (what the product did, never their content), let them read every line, and give them a one-switch opt-out. &#39;Help me build this&#39; is a different deal than &#39;watch me while I work.&#39;&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Parked or Forgotten? How to Tell Before It Costs You</title>
    <link href="https://soleur.ai/blog/parked-vs-forgotten-ideas/" />
    <updated>2026-09-25T00:00:00Z</updated>
    <id>https://soleur.ai/blog/parked-vs-forgotten-ideas/</id>
    <content type="html">&lt;p&gt;I lose track of what I decided and what I just forgot. From the outside, a parked idea and a forgotten one look exactly the same. So I re-argue choices I already made, or find an idea months later and realize I never decided anything about it.&lt;/p&gt;
&lt;p&gt;The usual fix is an idea parking lot: one list where every &amp;quot;not now&amp;quot; goes, so it stops nagging you. A parking lot solves half the problem. It holds the ideas you set aside, but it can&#39;t tell you which of them you actually decided on, or which ideas never made it onto the list at all.&lt;/p&gt;
&lt;p&gt;This post is the fix I use now. It works in a notebook, in Notion or in a spreadsheet, and you set it up once.&lt;/p&gt;
&lt;h2&gt;What it costs when you can&#39;t tell them apart&lt;/h2&gt;
&lt;p&gt;When a parked idea and a forgotten one look the same, you pay for it in two ways.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You re-decide things you already decided.&lt;/strong&gt; A few months ago you chose not to launch a second product yet. You had good reasons, but you didn&#39;t write them down. Now a customer asks for it, and you lose an evening re-arguing a question you had already answered. Worse, you can end up reversing a good decision because you can&#39;t remember why you made it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You lose ideas you meant to keep.&lt;/strong&gt; A strong idea comes up on a call. You jot it somewhere, meaning to come back to it. It never gets a decision, so it never gets a place, and by the time you find it again the moment has passed.&lt;/p&gt;
&lt;p&gt;Neither cost shows up on a dashboard. Both come out of the one resource a solo founder can&#39;t hire more of: your attention.&lt;/p&gt;
&lt;h2&gt;Every idea belongs in one of four places&lt;/h2&gt;
&lt;p&gt;The fix is a rule, not a tool. Every idea you have goes in exactly one of four places:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Doing now.&lt;/strong&gt; You&#39;re working on it this week or this month.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Later, chosen on purpose.&lt;/strong&gt; You decided it matters, and you decided it waits. That&#39;s a decision, not a delay.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Not yet defined.&lt;/strong&gt; You can&#39;t yet say what question the idea would answer. The test is whether you can state the question, not whether you can answer it. &amp;quot;Do a podcast&amp;quot; has no question yet, so it stays here. &amp;quot;Should a podcast replace my newsletter?&amp;quot; is a question you can state today, even if you don&#39;t know the answer, so that idea goes under Later.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Decided against, with one line on why.&lt;/strong&gt; You closed it, and you wrote down the reason. If you change your mind later, that&#39;s a new decision, not a reopened one.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Anything in none of the four wasn&#39;t parked. It was forgotten.&lt;/p&gt;
&lt;p&gt;Two of those places, not yet defined (with its &amp;quot;can you state the question&amp;quot; test) and decided against, come from &lt;a href=&quot;https://github.com/mattpocock/skills&quot;&gt;Matt Pocock&#39;s free, public collection of methods for working with AI&lt;/a&gt;. I added them to the Doing now and Later lists I already had, and added the rule above.&lt;/p&gt;
&lt;p&gt;The rule works because each place answers a different question. Doing now and Later say when. Not yet defined says what&#39;s missing before you can decide. Decided against says why not, in one line you can reread the next time the idea comes back.&lt;/p&gt;
&lt;h2&gt;Sort your list this week, with any tool&lt;/h2&gt;
&lt;p&gt;You don&#39;t need new software. Open wherever your ideas live now and work through them once:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;List everything.&lt;/strong&gt; Pull every idea, request and &amp;quot;we should&amp;quot; into one place: notes, your inbox, sticky notes, the back of a notebook.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Place each item in one of the four places.&lt;/strong&gt; Be honest. If you haven&#39;t decided, it isn&#39;t Later.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;For each &amp;quot;not yet defined&amp;quot; item, write the question it would answer.&lt;/strong&gt; If you can write it, the idea moves to Later or Doing now. If you can&#39;t write it yet, it stays where it is, and that&#39;s fine.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;For each &amp;quot;decided against&amp;quot; item, write one line on why.&lt;/strong&gt; One line is enough. It&#39;s for the version of you who meets this idea again in six months.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Anything left unplaced was forgotten.&lt;/strong&gt; Decide on it now, or place it.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The first pass is the slow one. After that, each new idea needs one question: which of the four places does it go in?&lt;/p&gt;
&lt;h2&gt;My AI team had blind spots too&lt;/h2&gt;
&lt;p&gt;I run my company with an AI team, so I assumed nothing on my list could slip. The part of my AI team that plans our roadmap once pointed me at the wrong priority. It also saw only about a quarter of our current plan, and it never warned me about either. Now it reads all of it, and tells me when it can&#39;t.&lt;/p&gt;
&lt;p&gt;I fixed both in the same round of work that gave my plan its two missing places, not yet defined and decided against. Together they changed how I think about any list: it only helps if whoever reads it, whether that&#39;s you, a cofounder or an AI, can see all of it and can tell what was decided.&lt;/p&gt;
&lt;h2&gt;What changed: decisions stay decided&lt;/h2&gt;
&lt;p&gt;Here is what&#39;s different now:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Ideas I can&#39;t phrase yet go on a &amp;quot;not yet defined&amp;quot; list instead of into a half-made plan.&lt;/li&gt;
&lt;li&gt;Anything I rule out is closed with one line on why, so I stop re-arguing it.&lt;/li&gt;
&lt;li&gt;My AI team sees all of the current plan, and tells me when it can&#39;t.&lt;/li&gt;
&lt;li&gt;When it suggests what to work on next, it&#39;s something I can start today.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;I still make every call about which place an idea goes. My AI team keeps the list and flags anything on it that doesn&#39;t have a place yet. It&#39;s the same split I use for &lt;a href=&quot;https://soleur.ai/blog/how-to-run-every-department-with-ai-agents/&quot;&gt;the rest of the company&lt;/a&gt;: I decide, and the team keeps track.&lt;/p&gt;
&lt;p&gt;Running a company alone shouldn&#39;t mean remembering everything alone.&lt;/p&gt;
&lt;h2&gt;Frequently asked questions&lt;/h2&gt;
&lt;div class=&quot;faq-list&quot;&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;What is the difference between a parked idea and a forgotten idea?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;A parked idea has a decision attached: you chose to do it later, or you need to define it before you can decide. A forgotten idea has no decision and no place on your list. From the outside they look the same, which is why founders re-argue settled choices and lose good ideas.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;When is an idea &#39;not yet defined&#39; instead of &#39;later&#39;?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;An idea is not yet defined when you cannot state the question it would answer. It belongs under Later when you can state that question, even if you cannot answer it yet. &quot;Do a podcast&quot; is not yet defined. &quot;Should a podcast replace my newsletter?&quot; is a question, so it goes under Later.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;How do I stop going back over decisions I&#39;ve already made?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;Write one line on why every time you decide against an idea, and keep that line next to the idea. When it comes back, read the reason before you debate it again. If your situation has changed, make a new decision instead of reopening the old one.&lt;/p&gt;
  &lt;/details&gt;
&lt;/div&gt;
&lt;p&gt;Want an AI team that keeps the list while you make the calls? &lt;a href=&quot;https://soleur.ai/pricing/#waitlist&quot;&gt;Join the waitlist&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;For the technical write-up, see &lt;a href=&quot;https://github.com/jikig-ai/soleur/pull/8536&quot;&gt;how we fixed it&lt;/a&gt;.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is the difference between a parked idea and a forgotten idea?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;A parked idea has a decision attached: you chose to do it later, or you need to define it before you can decide. A forgotten idea has no decision and no place on your list. From the outside they look the same, which is why founders re-argue settled choices and lose good ideas.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;When is an idea &#39;not yet defined&#39; instead of &#39;later&#39;?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;An idea is not yet defined when you cannot state the question it would answer. It belongs under Later when you can state that question, even if you cannot answer it yet. &#92;&quot;Do a podcast&#92;&quot; is not yet defined. &#92;&quot;Should a podcast replace my newsletter?&#92;&quot; is a question, so it goes under Later.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How do I stop going back over decisions I&#39;ve already made?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Write one line on why every time you decide against an idea, and keep that line next to the idea. When it comes back, read the reason before you debate it again. If your situation has changed, make a new decision instead of reopening the old one.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Best AI Tools for Solo Founders 2026: The Stack Worth Running</title>
    <link href="https://soleur.ai/blog/best-ai-tools-for-solo-founders-2026/" />
    <updated>2026-06-15T00:00:00Z</updated>
    <id>https://soleur.ai/blog/best-ai-tools-for-solo-founders-2026/</id>
    <content type="html">&lt;p&gt;The solo founder has more leverage in 2026 than at any point before it. Carta&#39;s data shows single-founder startups now make up roughly a third of new company formations on its cap-table platform (&lt;a href=&quot;https://carta.com/data/&quot;&gt;Carta, State of Private Markets&lt;/a&gt;), and the tooling has caught up with the ambition: one person can now ship what used to take a small team. The constraint is no longer headcount. It is choosing the tools that earn their place in a one-person workflow.&lt;/p&gt;
&lt;p&gt;This guide ranks the AI tools and platforms that genuinely move the needle for solo founders in 2026 — the agents, the editors, the infrastructure — and then draws the line between adding another capable tool and changing the scope of what you can run alone.&lt;/p&gt;
&lt;h2&gt;How to Evaluate an AI Tool as a Solo Founder&lt;/h2&gt;
&lt;p&gt;A team can absorb a tool that only pays off occasionally. A solo founder cannot. Every tool you adopt is a tool you alone configure, maintain, and switch context into. Three questions before anything earns a place in the stack:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Does it remove work, or relocate it?&lt;/strong&gt; A tool that automates a task but demands constant supervision has relocated the work, not removed it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Does it compound?&lt;/strong&gt; The best tools get more valuable as they learn your codebase, your customers, or your process — not merely faster at a fixed task.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Does it cover a whole job, or one step of one job?&lt;/strong&gt; Single-step tools are worth it when the step is frequent. Whole-job coverage is worth far more when you are the only person doing every job.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;With that filter, here are the tools that survive.&lt;/p&gt;
&lt;h2&gt;The Best AI Tools for Solo Founders in 2026&lt;/h2&gt;
&lt;h3&gt;1. Claude Code — The Agentic Engineering Runtime&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://docs.anthropic.com/en/docs/claude-code&quot;&gt;Claude Code&lt;/a&gt; is the foundation most solo technical founders build on in 2026. It runs in the terminal, reads and edits your actual repository, runs tests, and drives multi-step engineering work end to end. What separates it from earlier coding tools is extensibility: skills, agents, hooks, and &lt;a href=&quot;https://modelcontextprotocol.io/&quot;&gt;Model Context Protocol (MCP)&lt;/a&gt; integrations let you teach it your stack and your conventions once, then reuse that knowledge on every task.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; it covers a whole job — engineering — rather than autocomplete on one file. For a founder writing all the code, that is the difference between faster typing and a second engineer.&lt;/p&gt;
&lt;h3&gt;2. Cursor — The AI-Native Editor&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://cursor.com/&quot;&gt;Cursor&lt;/a&gt; remains the strongest AI-native IDE for founders who prefer an editor-centric workflow over a terminal agent. Inline edits, codebase-aware chat, and fast multi-file refactors make it excellent for the tight read-edit-run loop of early product work. Many founders run Cursor and a terminal agent side by side — the editor for exploratory work, the agent for structured, multi-step changes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; the lowest-friction on-ramp to AI-assisted coding, with a learning curve measured in minutes.&lt;/p&gt;
&lt;h3&gt;3. Linear — Planning That Keeps Pace With Shipping&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://linear.app/&quot;&gt;Linear&lt;/a&gt; is where solo founders keep the plan when the plan changes daily. Its 2026 AI features — issue drafting, automatic triage, and cycle planning — matter most precisely because there is no project manager to keep the board honest. The tool does the maintenance a team would otherwise distribute.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; it keeps planning overhead near zero, which is the only acceptable level when one person owns both the roadmap and the work.&lt;/p&gt;
&lt;h3&gt;4. Vercel — Ship Without an Ops Hire&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://vercel.com/&quot;&gt;Vercel&lt;/a&gt; collapses deployment, hosting, and preview environments into a git push. For a founder with no infrastructure team, the value is the absence of one: no pipeline to babysit, no servers to patch, preview URLs on every branch for fast feedback.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; it removes a whole category of work — operations — rather than speeding one task within it.&lt;/p&gt;
&lt;h3&gt;5. Supabase — The Backend a Solo Founder Can Actually Run&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://supabase.com/&quot;&gt;Supabase&lt;/a&gt; gives a single founder a production Postgres database, authentication, storage, and edge functions behind one dashboard and one set of credentials. Row-level security and generated APIs mean the data layer is something one person can reason about and keep secure.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; whole-job coverage of the backend, with security primitives that do not require a dedicated platform engineer.&lt;/p&gt;
&lt;h3&gt;6. Stripe — Revenue Infrastructure That Runs Itself&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://stripe.com/&quot;&gt;Stripe&lt;/a&gt; handles payments, subscriptions, invoicing, and tax so the founder does not. Its agent-facing APIs and 2026 tooling let billing logic be built and adjusted as fast as the rest of the product, which matters when pricing is still moving.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; it turns the entire finance-operations surface of taking money into configuration instead of a function to staff.&lt;/p&gt;
&lt;h3&gt;7. Perplexity — Research at Founder Speed&lt;/h3&gt;
&lt;p&gt;&lt;a href=&quot;https://www.perplexity.ai/&quot;&gt;Perplexity&lt;/a&gt; compresses market research, competitive checks, and technical lookups into cited answers. For a founder who is also the analyst, it shortens the loop from question to sourced answer from hours to minutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why it earns its place:&lt;/strong&gt; frequent, whole-loop coverage of research, with citations that let you verify rather than trust.&lt;/p&gt;
&lt;h2&gt;Beyond Tools: The Organization Layer&lt;/h2&gt;
&lt;p&gt;Every tool above makes you faster at a job you already do. None of them does the jobs you do not have time to do at all.&lt;/p&gt;
&lt;p&gt;That is the gap a solo founder feels by the second month: the code ships, but the contracts go unread, the competitors go unwatched, the pipeline goes untracked, and the learnings from last week never inform next week. No single-purpose tool closes that gap, because the gap is not a missing step in one job — it is a missing organization.&lt;/p&gt;
&lt;p&gt;That is the category &lt;a href=&quot;https://soleur.ai&quot;&gt;Soleur&lt;/a&gt; occupies. It is not a better IDE or a faster deploy. It is a &lt;a href=&quot;https://soleur.ai/company-as-a-service/&quot;&gt;Company-as-a-Service&lt;/a&gt; platform that runs as a Claude Code extension: specialist &lt;a href=&quot;https://soleur.ai/ai-agents-for-solo-founders/&quot;&gt;AI agents for solo founders&lt;/a&gt; across eight business departments — engineering, marketing, legal, finance, operations, product, sales, and support — coordinated through one compounding knowledge base, so each task informs every task that follows.&lt;/p&gt;
&lt;p&gt;We rank Soleur honestly: not as the best editor or the best database — the tools above win those categories on their merits — but as a distinct layer above them. The tools make the work faster. The organization layer changes which work a single person can take on at all. For the fuller picture of how these pieces fit together, see the &lt;a href=&quot;https://soleur.ai/solo-founder-ai-stack/&quot;&gt;solo founder AI stack&lt;/a&gt; and our breakdown of the &lt;a href=&quot;https://soleur.ai/blog/billion-dollar-solo-founder-stack/&quot;&gt;billion-dollar one-person company stack&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;The Stack Worth Running&lt;/h2&gt;
&lt;p&gt;For a solo founder building a software product in 2026, the practical shortlist:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool&lt;/th&gt;
&lt;th&gt;Job it covers&lt;/th&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Claude Code&lt;/td&gt;
&lt;td&gt;Engineering, end to end&lt;/td&gt;
&lt;td&gt;Agentic engineering runtime&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cursor&lt;/td&gt;
&lt;td&gt;AI-assisted editing&lt;/td&gt;
&lt;td&gt;IDE&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Linear&lt;/td&gt;
&lt;td&gt;Planning and triage&lt;/td&gt;
&lt;td&gt;Project management&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vercel&lt;/td&gt;
&lt;td&gt;Deploy and hosting&lt;/td&gt;
&lt;td&gt;Infrastructure&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Supabase&lt;/td&gt;
&lt;td&gt;Database, auth, storage&lt;/td&gt;
&lt;td&gt;Backend&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Stripe&lt;/td&gt;
&lt;td&gt;Payments and billing&lt;/td&gt;
&lt;td&gt;Revenue infrastructure&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Perplexity&lt;/td&gt;
&lt;td&gt;Research&lt;/td&gt;
&lt;td&gt;Knowledge&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Soleur&lt;/td&gt;
&lt;td&gt;Every business department&lt;/td&gt;
&lt;td&gt;Organization layer (Company-as-a-Service)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The first seven make each job faster. The last one changes how many jobs one person can hold.&lt;/p&gt;
&lt;h2&gt;FAQ&lt;/h2&gt;
&lt;h3&gt;What are the best AI tools for solo founders in 2026?&lt;/h3&gt;
&lt;p&gt;The strongest single-purpose tools for solo founders in 2026 are Claude Code and Cursor for engineering, Linear for planning, Vercel for deployment, Supabase for the backend, Stripe for payments, and Perplexity for research. Above the single-purpose tools, Soleur provides an organization layer — a Company-as-a-Service platform with specialist agents across eight business departments — that covers the work no single-purpose tool addresses.&lt;/p&gt;
&lt;h3&gt;What is the difference between an AI tool and an AI organization layer?&lt;/h3&gt;
&lt;p&gt;An AI tool makes one job faster — writing code, deploying, taking payments. An AI organization layer does the jobs a solo founder has no time to staff at all: reviewing contracts, watching competitors, tracking pipeline, and compounding what was learned. Tools speed up existing work; the organization layer expands the range of work one person can take on.&lt;/p&gt;
&lt;h3&gt;Do solo founders need a full AI stack, or only one tool?&lt;/h3&gt;
&lt;p&gt;Most solo founders start with one engineering tool (Claude Code or Cursor) and add infrastructure (Vercel, Supabase) and revenue tools (Stripe) as the product takes shape. The single-purpose stack covers building and shipping. Founders feel the second gap — the business operations no tool touches — once the product is live, which is where an organization-layer platform like Soleur fits.&lt;/p&gt;
&lt;h3&gt;How much does an AI stack for solo founders cost in 2026?&lt;/h3&gt;
&lt;p&gt;Most of the tools in this guide offer free or low-cost entry tiers that scale with usage, so an early-stage solo founder can run a capable stack for a modest monthly cost. Pricing changes frequently — check each tool&#39;s current pricing page. The larger cost for a solo founder is usually time and context-switching, not subscription fees, which is why whole-job coverage tends to pay for itself faster than single-step tools.&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What are the best AI tools for solo founders in 2026?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;The strongest single-purpose tools for solo founders in 2026 are Claude Code and Cursor for engineering, Linear for planning, Vercel for deployment, Supabase for the backend, Stripe for payments, and Perplexity for research. Above the single-purpose tools, Soleur provides an organization layer — a Company-as-a-Service platform with specialist agents across eight business departments — that covers the work no single-purpose tool addresses.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is the difference between an AI tool and an AI organization layer?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;An AI tool makes one job faster — writing code, deploying, taking payments. An AI organization layer does the jobs a solo founder has no time to staff at all: reviewing contracts, watching competitors, tracking pipeline, and compounding what was learned. Tools speed up existing work; the organization layer expands the range of work one person can take on.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Do solo founders need a full AI stack, or only one tool?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Most solo founders start with one engineering tool (Claude Code or Cursor) and add infrastructure (Vercel, Supabase) and revenue tools (Stripe) as the product takes shape. The single-purpose stack covers building and shipping. Founders feel the second gap — the business operations no tool touches — once the product is live, which is where an organization-layer platform like Soleur fits.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;How much does an AI stack for solo founders cost in 2026?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Most of the tools in this guide offer free or low-cost entry tiers that scale with usage, so an early-stage solo founder can run a capable stack for a modest monthly cost. Pricing changes frequently — check each tool&#39;s current pricing page. The larger cost for a solo founder is usually time and context-switching, not subscription fees, which is why whole-job coverage tends to pay for itself faster than single-step tools.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Loop Engineering for Your Whole Company, Not Just Your Codebase</title>
    <link href="https://soleur.ai/blog/loop-engineering-for-your-whole-company/" />
    <updated>2026-06-12T00:00:00Z</updated>
    <id>https://soleur.ai/blog/loop-engineering-for-your-whole-company/</id>
    <content type="html">&lt;p&gt;In June 2026, Addy Osmani — a director on Google&#39;s Cloud AI team — gave a name to a shift that was already underway. He called it &lt;strong&gt;&lt;a href=&quot;https://addyo.substack.com/p/loop-engineering&quot;&gt;loop engineering&lt;/a&gt;&lt;/strong&gt;, and his one-line definition is the clearest framing of the idea anyone has written:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&amp;quot;Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead.&amp;quot;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The leverage has moved. The skill that mattered last year was writing a good prompt. The skill that matters now is designing the system that prompts the agent for you — on a schedule, in isolation, with memory that survives between runs and a second agent that checks the first one&#39;s work.&lt;/p&gt;
&lt;p&gt;This post does two things. First, it credits Osmani&#39;s framework and the engineers who shaped it. Then it extends the idea past the boundary Osmani drew around it — because loop engineering does not stop at code.&lt;/p&gt;
&lt;h2&gt;What Osmani named&lt;/h2&gt;
&lt;p&gt;Osmani&#39;s essay lays out five building blocks for systems that prompt agents on their own, plus a sixth element that ties them together:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Automations&lt;/strong&gt; — scheduled tasks that discover and triage work on a cadence, without a human kicking them off.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Worktrees&lt;/strong&gt; — isolated working directories that let several agents run in parallel without colliding on the same files.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Skills&lt;/strong&gt; — documented procedural knowledge, captured once so the agent never has to be re-taught the same task.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Connectors&lt;/strong&gt; — integrations, over the &lt;a href=&quot;https://modelcontextprotocol.io/&quot;&gt;Model Context Protocol&lt;/a&gt;, that give agents access to real tools and systems.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Sub-agents&lt;/strong&gt; — a separate agent that verifies the work, so the maker never grades its own homework.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The sixth is &lt;strong&gt;external memory&lt;/strong&gt; — markdown files, issue boards, a knowledge base — because models forget everything between sessions, and the loop only compounds if state outlives the run.&lt;/p&gt;
&lt;p&gt;He is not alone in seeing the shift. Boris Cherny, who leads Claude Code, describes his own working day this way (as quoted in Osmani&#39;s essay):&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&amp;quot;I don&#39;t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops&amp;quot;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Peter Steinberger puts the same idea as a directive (also quoted in the essay):&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&amp;quot;You shouldn&#39;t be prompting coding agents anymore. You should be designing loops that prompt your agents.&amp;quot;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Osmani closes by keeping the human in frame: &lt;strong&gt;&amp;quot;Build the loop. But build it like someone who intends to stay the engineer, not just the person who presses go.&amp;quot;&lt;/strong&gt; That instinct — automate the prompting, keep the judgment — is exactly right, and it is the spirit this post extends.&lt;/p&gt;
&lt;h2&gt;The boundary Osmani drew&lt;/h2&gt;
&lt;p&gt;Read the essay closely and you will notice it lives entirely inside software engineering. Every example is a coding agent: fixing a failing test, triaging a pull request, refactoring a module. That is the natural place for loop engineering to start, because engineers built the substrate — worktrees, skills files, the Model Context Protocol — for their own work first.&lt;/p&gt;
&lt;p&gt;But nothing in the architecture is specific to code. A scheduled automation does not care whether it triages a bug or a churn signal. A skill file is as happy documenting a refund policy as a migration. External memory is as useful to a legal review as to a release. The six blocks are domain-neutral; only the examples were not.&lt;/p&gt;
&lt;h2&gt;Loop engineering for the whole company&lt;/h2&gt;
&lt;p&gt;Soleur is what loop engineering looks like when you stop scoping it to the codebase. It runs the same six pieces Osmani named — across marketing, sales, legal, finance, operations, support, and product, not just engineering.&lt;/p&gt;
&lt;p&gt;Four of the six already span every department:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;External memory&lt;/strong&gt; is the foundation, and it is the strongest piece. A single git-committed knowledge base holds the state of the whole company — brand guides, deal notes, compliance posture, plans, learnings — and every agent in every department reads and writes it. State outlives the run by default, because it is version-controlled, auditable, and yours.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Skills&lt;/strong&gt; number more than 80 documented procedures, spanning eight departments. A legal-document generator, a competitive-intelligence scan, a campaign calendar, a budget analysis — each one captured once.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Connectors&lt;/strong&gt; reach real systems over MCP. Cloudflare, Stripe, and Vercel ship in the box, with more available — so an agent can read live revenue or change a DNS record, not just talk about it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Worktrees&lt;/strong&gt; give every agent an isolated place to work in parallel, the same substrate Osmani describes, available to any department&#39;s work.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The remaining two — scheduled automations that dispatch agents on their own, and the maker/checker split where a separate agent verifies the work — run today in engineering, and Soleur is extending them outward across the company. That is the honest shape of it: the memory, skills, and connectors are company-wide now; the autonomous dispatch and verification are proven where they started and generalizing from there.&lt;/p&gt;
&lt;p&gt;This is the difference worth naming. Osmani&#39;s loop replaces &lt;em&gt;you, the engineer who prompts the coding agent&lt;/em&gt;. Soleur&#39;s loop replaces &lt;em&gt;you, the founder who prompts every department&lt;/em&gt; — the one person doing marketing on Monday, finance on Tuesday, and legal review whenever the contract lands. Same architecture. Wider blast radius.&lt;/p&gt;
&lt;p&gt;If you want the ground underneath this — why the leverage moved from prompting to system design in the first place — that is the larger story of &lt;a href=&quot;https://soleur.ai/agentic-engineering/&quot;&gt;agentic engineering&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Build the loop. Run the company.&lt;/h2&gt;
&lt;p&gt;Osmani&#39;s instinct is the right one to keep: build the loop, but stay the engineer who presses go on purpose, not by reflex. The judgment stays human; the prompting becomes a system you design. Soleur takes that instinct and points it at the whole org chart — an AI organization where the loops run the departments and you run the loops.&lt;/p&gt;
&lt;p&gt;The leverage point Osmani identified for code is the same leverage point for a company. The question is no longer how to prompt your agents. It is which loops to build first.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;Soleur is not affiliated with or endorsed by Addy Osmani, Boris Cherny, Peter Steinberger, Google, or Anthropic. &amp;quot;Loop engineering&amp;quot; was coined by Addy Osmani; the quotes above appear in his essay and are reproduced here with attribution. Source: &lt;a href=&quot;https://addyo.substack.com/p/loop-engineering&quot;&gt;Addy Osmani, &amp;quot;Loop Engineering&amp;quot; (June 2026)&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is loop engineering?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Loop engineering, a term coined by Addy Osmani in June 2026, is the practice of designing systems that prompt AI agents autonomously instead of prompting them yourself by hand. Its building blocks are scheduled automations, isolated worktrees, documented skills, MCP connectors, verifier sub-agents, and external memory that persists state across sessions.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Can loop engineering work outside of code?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes. Osmani&#39;s essay scopes loop engineering to software engineering, but the six building blocks are domain-neutral. Soleur applies the same architecture across marketing, sales, legal, finance, operations, support, and product. External memory, skills, and MCP connectors span every department today; autonomous scheduling and maker/checker verification run in engineering and are extending across the company.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Is Soleur affiliated with Addy Osmani, Google, or Anthropic?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;No. Soleur is not affiliated with or endorsed by Addy Osmani, Boris Cherny, Peter Steinberger, Google, or Anthropic. The term loop engineering was coined by Addy Osmani, and the quotes from Boris Cherny and Peter Steinberger appear in Osmani&#39;s essay, reproduced here with attribution.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Claude Code Plugin vs Skill vs MCP: A Clear Disambiguation</title>
    <link href="https://soleur.ai/blog/claude-code-plugin-vs-skill-vs-mcp/" />
    <updated>2026-06-01T00:00:00Z</updated>
    <id>https://soleur.ai/blog/claude-code-plugin-vs-skill-vs-mcp/</id>
    <content type="html">&lt;p&gt;Three terms get used interchangeably when people talk about extending Claude Code: &lt;strong&gt;plugin&lt;/strong&gt;, &lt;strong&gt;skill&lt;/strong&gt;, and &lt;strong&gt;MCP server&lt;/strong&gt;. They are not the same thing. They operate at different layers, load at different times, and ship through different channels. Conflating them leads to bad architecture decisions — packaging a one-off instruction as an MCP server, or reaching for a plugin when a skill would do.&lt;/p&gt;
&lt;p&gt;This post draws the lines cleanly. Each primitive answers a different question: a &lt;strong&gt;skill&lt;/strong&gt; answers &amp;quot;what should the model know how to do,&amp;quot; an &lt;strong&gt;MCP server&lt;/strong&gt; answers &amp;quot;what external systems can the model reach,&amp;quot; and a &lt;strong&gt;plugin&lt;/strong&gt; answers &amp;quot;how do I distribute a bundle of capabilities to a team.&amp;quot;&lt;/p&gt;
&lt;h2&gt;The one-line definitions&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;A &lt;strong&gt;skill&lt;/strong&gt; is a packaged unit of procedural knowledge — a &lt;code&gt;SKILL.md&lt;/code&gt; file (plus optional scripts and references) that teaches the model how to perform a task. It is loaded on demand when the task is relevant. (&lt;a href=&quot;https://docs.claude.com/en/docs/agents-and-tools/agent-skills&quot;&gt;Agent Skills documentation&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;An &lt;strong&gt;MCP server&lt;/strong&gt; is a separate process that exposes tools, resources, and prompts over the &lt;a href=&quot;https://modelcontextprotocol.io/&quot;&gt;Model Context Protocol&lt;/a&gt;, an open standard for connecting AI applications to external systems. It is how the model reaches databases, APIs, and services. (&lt;a href=&quot;https://docs.claude.com/en/docs/claude-code/mcp&quot;&gt;MCP in Claude Code&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;A &lt;strong&gt;plugin&lt;/strong&gt; is a distributable bundle that can contain commands, agents, skills, hooks, and MCP server configuration, installed as a single unit and shared across a team via a marketplace. (&lt;a href=&quot;https://docs.claude.com/en/docs/claude-code/plugins&quot;&gt;Claude Code plugins documentation&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Plugin vs Skill vs MCP: the disambiguation table&lt;/h2&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Dimension&lt;/th&gt;
&lt;th&gt;Skill&lt;/th&gt;
&lt;th&gt;MCP server&lt;/th&gt;
&lt;th&gt;Plugin&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;What it is&lt;/td&gt;
&lt;td&gt;Packaged procedural knowledge (&lt;code&gt;SKILL.md&lt;/code&gt; + assets)&lt;/td&gt;
&lt;td&gt;A process exposing tools, resources, and prompts over an open protocol&lt;/td&gt;
&lt;td&gt;A distributable bundle of commands, agents, skills, hooks, and MCP config&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scope&lt;/td&gt;
&lt;td&gt;Teaches the model how to do a task&lt;/td&gt;
&lt;td&gt;Connects the model to external systems and data&lt;/td&gt;
&lt;td&gt;Packages and distributes many capabilities at once&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lifecycle&lt;/td&gt;
&lt;td&gt;Loaded on demand when the task is relevant&lt;/td&gt;
&lt;td&gt;Runs as a long-lived process; connected for the session&lt;/td&gt;
&lt;td&gt;Installed once; its contents load per their own rules&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Runs where&lt;/td&gt;
&lt;td&gt;In-context, no separate process&lt;/td&gt;
&lt;td&gt;A separate local or remote process&lt;/td&gt;
&lt;td&gt;N/A — a container; its parts run per their type&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Distribution&lt;/td&gt;
&lt;td&gt;A directory or repo; bundled inside plugins&lt;/td&gt;
&lt;td&gt;Configured per project or user; published to MCP registries&lt;/td&gt;
&lt;td&gt;Installed from a plugin marketplace&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Answers&lt;/td&gt;
&lt;td&gt;&amp;quot;What should the model know how to do?&amp;quot;&lt;/td&gt;
&lt;td&gt;&amp;quot;What systems can the model reach?&amp;quot;&lt;/td&gt;
&lt;td&gt;&amp;quot;How do I ship a capability set to a team?&amp;quot;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The key insight: these are &lt;strong&gt;composable, not competing&lt;/strong&gt;. A plugin can ship skills and wire up MCP servers. A skill can describe how to use the tools an MCP server exposes. You rarely choose one &lt;em&gt;instead of&lt;/em&gt; another — you choose which layer a given capability belongs to.&lt;/p&gt;
&lt;h2&gt;How to choose&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Reach for a &lt;strong&gt;skill&lt;/strong&gt; when you are encoding a repeatable procedure — a workflow, a checklist, a domain method — that the model should apply when relevant. No new process, no external connection.&lt;/li&gt;
&lt;li&gt;Reach for an &lt;strong&gt;MCP server&lt;/strong&gt; when the model needs to &lt;em&gt;touch something outside itself&lt;/em&gt;: query a database, call an API, read a file store, hit a SaaS product. If it requires authentication or a network call to a system, it is an MCP concern.&lt;/li&gt;
&lt;li&gt;Reach for a &lt;strong&gt;plugin&lt;/strong&gt; when you have a &lt;em&gt;set&lt;/em&gt; of capabilities — several commands, a few agents, a handful of skills, maybe an MCP server — that you want to install and version as one unit and share with a team.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For a ranked, opinionated tour of what is actually worth installing across all three, see &lt;a href=&quot;https://soleur.ai/blog/best-claude-code-plugins-2026/&quot;&gt;Best Claude Code Plugins 2026&lt;/a&gt;. For the deeper distinction between portable skill libraries and end-to-end workflow plugins — the two dominant &lt;em&gt;shapes&lt;/em&gt; a plugin takes — read &lt;a href=&quot;https://soleur.ai/blog/skill-libraries-vs-workflow-plugins/&quot;&gt;Skill Libraries vs. Workflow Plugins&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Soleur itself is a plugin: it ships 69 agents and 108 skills across 8 departments, and configures the MCP servers its agents rely on — one install, the whole &lt;a href=&quot;https://soleur.ai/company-as-a-service/&quot;&gt;Company-as-a-Service&lt;/a&gt; organization.&lt;/p&gt;
&lt;h2&gt;Frequently asked questions&lt;/h2&gt;
&lt;div class=&quot;faq-list&quot;&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;What is the difference between a Claude Code plugin, skill, and MCP server?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;A skill is packaged procedural knowledge that teaches the model how to do a task and loads on demand. An MCP server is a separate process that connects the model to external systems and data over an open protocol. A plugin is a distributable bundle that can contain commands, agents, skills, hooks, and MCP server configuration, installed as a single unit. They operate at different layers and are composable, not competing.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;When should I use a skill instead of an MCP server?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;Use a skill when you are encoding a repeatable procedure the model should apply when relevant, with no external connection required. Use an MCP server when the model needs to reach something outside itself, such as a database, an API, or a SaaS product, which requires a running process and usually authentication. Skills teach know-how; MCP servers provide reach.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Can a plugin contain skills and MCP servers?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;Yes. A plugin is a container. It can bundle commands, agents, skills, hooks, and MCP server configuration and install them all as one versioned unit. This is why the three primitives are composable rather than mutually exclusive: the plugin is the distribution layer, while skills and MCP servers are the capability layers inside it.&lt;/p&gt;
  &lt;/details&gt;
  &lt;details class=&quot;faq-item&quot;&gt;
    &lt;summary class=&quot;faq-question&quot;&gt;Which one do I distribute to my team?&lt;/summary&gt;
    &lt;p class=&quot;faq-answer&quot;&gt;Distribute a plugin. Plugins install from a marketplace as a single versioned unit and can carry many skills, agents, and MCP server configurations at once, so a team installs one thing and gets the whole capability set. Individual skills and MCP servers can be shared on their own, but a plugin is the channel built for packaging and sharing a set of capabilities.&lt;/p&gt;
  &lt;/details&gt;
&lt;/div&gt;
&lt;script type=&quot;application/ld+json&quot;&gt;
{
  &quot;@context&quot;: &quot;https://schema.org&quot;,
  &quot;@type&quot;: &quot;FAQPage&quot;,
  &quot;mainEntity&quot;: [
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;What is the difference between a Claude Code plugin, skill, and MCP server?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;A skill is packaged procedural knowledge that teaches the model how to do a task and loads on demand. An MCP server is a separate process that connects the model to external systems and data over an open protocol. A plugin is a distributable bundle that can contain commands, agents, skills, hooks, and MCP server configuration, installed as a single unit. They operate at different layers and are composable, not competing.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;When should I use a skill instead of an MCP server?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Use a skill when you are encoding a repeatable procedure the model should apply when relevant, with no external connection required. Use an MCP server when the model needs to reach something outside itself, such as a database, an API, or a SaaS product, which requires a running process and usually authentication. Skills teach know-how; MCP servers provide reach.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Can a plugin contain skills and MCP servers?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Yes. A plugin is a container. It can bundle commands, agents, skills, hooks, and MCP server configuration and install them all as one versioned unit. This is why the three primitives are composable rather than mutually exclusive: the plugin is the distribution layer, while skills and MCP servers are the capability layers inside it.&quot;
      }
    },
    {
      &quot;@type&quot;: &quot;Question&quot;,
      &quot;name&quot;: &quot;Which one do I distribute to my team?&quot;,
      &quot;acceptedAnswer&quot;: {
        &quot;@type&quot;: &quot;Answer&quot;,
        &quot;text&quot;: &quot;Distribute a plugin. Plugins install from a marketplace as a single versioned unit and can carry many skills, agents, and MCP server configurations at once, so a team installs one thing and gets the whole capability set. Individual skills and MCP servers can be shared on their own, but a plugin is the channel built for packaging and sharing a set of capabilities.&quot;
      }
    }
  ]
}
&lt;/script&gt;
</content>
  </entry>
  <entry>
    <title>Skill Libraries vs. Workflow Plugins: Two Shapes of Claude Code Extension</title>
    <link href="https://soleur.ai/blog/skill-libraries-vs-workflow-plugins/" />
    <updated>2026-05-15T00:00:00Z</updated>
    <id>https://soleur.ai/blog/skill-libraries-vs-workflow-plugins/</id>
    <content type="html">&lt;p&gt;Search &amp;quot;Claude Code skills&amp;quot; on GitHub and two very different shapes come back. One repository is a catalog: a long list of self-contained skills you can drop into any Claude Code install, picked individually, used independently. Another shape is the opposite — a single bundle of agents, skills, and a shared knowledge base, opinionated end-to-end, with cross-references between every component.&lt;/p&gt;
&lt;p&gt;The instinct is to rank them on shelf size. The catalog looks bigger, the bundle looks narrower, and the comparison ends before the actual question gets asked: which shape fits the work you&#39;re doing?&lt;/p&gt;
&lt;p&gt;These are two different categories. Each answers a different question. Choosing between them — or running both — depends on what you need Claude Code to do this week.&lt;/p&gt;
&lt;h2&gt;What a Skill Library Is&lt;/h2&gt;
&lt;p&gt;A skill library is a portable catalog of self-contained Claude Code skills. The defining traits: breadth-first coverage, intentional unopinionatedness about how the skills get used, pick-and-mix consumption, MIT-friendly portability across teams and projects.&lt;/p&gt;
&lt;p&gt;The canonical exemplar is the &lt;a href=&quot;https://github.com/alirezarezvani/claude-skills&quot;&gt;alirezarezvani/claude-skills&lt;/a&gt; repository — the reference implementation for the portable-library shape. It catalogs a broad collection of skills covering common developer needs (research workflows, data wrangling, documentation generation, project scaffolding) and ships them as independent units. Each skill is its own folder, its own README, its own contract. You can copy one skill into your project, ignore everything else in the catalog, and never touch the rest.&lt;/p&gt;
&lt;p&gt;A skill library answers one question: &amp;quot;what&#39;s the next useful capability I can add to Claude Code without changing how I work?&amp;quot; The library is a shelf. You walk down it, you take what you need, you go. There is no expectation that skill A is aware of skill B. There is no shared memory between them. Orchestration is your job; the library hands you the pieces.&lt;/p&gt;
&lt;p&gt;That shape has real strengths. The library is portable: a skill written for one team&#39;s Claude Code install runs on another team&#39;s install with no migration. The catalog scales horizontally — adding capabilities widens coverage without changing the core promise. And the unopinionated design respects the user&#39;s existing workflow, imposing no convention.&lt;/p&gt;
&lt;h2&gt;What a Workflow Plugin Is&lt;/h2&gt;
&lt;p&gt;A workflow plugin is opinionated orchestration across a lifecycle. Where a library hands you pieces, this shape runs an organization.&lt;/p&gt;
&lt;p&gt;Soleur is the canonical exemplar. Soleur ships 69+ agents, 108+ skills, and a compounding knowledge base, organized across 8 business departments — engineering, marketing, legal, finance, operations, product, sales, support, community. The agents are not interchangeable parts. The marketing agent reads what the legal agent decided. The competitive-intelligence agent feeds the brand-architect agent. The compound skill captures learnings from every session and replays them into the next one.&lt;/p&gt;
&lt;p&gt;The agents and skills are wired through a brainstorm → plan → work → review → compound lifecycle. Each stage is a gate. Brainstorm surfaces ambiguity before plans get written. Plans get deepened, then executed by work. Review catches what work missed. Compound writes the lesson into a knowledge base that every future session reads from. Decisions in session 1 shape what session 50 produces.&lt;/p&gt;
&lt;p&gt;The compounding knowledge base is the structural component that separates a workflow plugin from a pile of skills. It is a git-tracked directory of markdown files that every agent reads from and writes to. The brand guide informs the copywriter agent&#39;s tone. The competitive-intelligence audit informs the growth-strategist agent&#39;s positioning. The institutional learnings from a debugged migration inform the data-integrity-guardian agent the next time someone touches the schema. None of this requires the founder to wire it manually. The cross-domain coherence is the default.&lt;/p&gt;
&lt;p&gt;This shape answers a different question: &amp;quot;how do I execute every department of my company without hiring a team?&amp;quot; That is the &lt;a href=&quot;https://soleur.ai/company-as-a-service/&quot;&gt;Company-as-a-Service&lt;/a&gt; shape — a full AI organization, opinionated end-to-end, designed for the founder running every job alone.&lt;/p&gt;
&lt;h2&gt;Why Two Shapes (Not One Hierarchy)&lt;/h2&gt;
&lt;p&gt;The temptation is to put these on a leaderboard and pick a winner. Resist it. Skill libraries and workflow plugins optimize for different things. Comparing them on the same axis is like comparing a hardware store to a general contractor — same building materials, completely different unit of value.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Optimizes for&lt;/th&gt;
&lt;th&gt;Skill libraries&lt;/th&gt;
&lt;th&gt;Workflow plugins&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Distribution model&lt;/td&gt;
&lt;td&gt;Pick-and-mix, individual skills&lt;/td&gt;
&lt;td&gt;End-to-end lifecycle, all-or-nothing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unit of value&lt;/td&gt;
&lt;td&gt;A single capability&lt;/td&gt;
&lt;td&gt;A coordinated organization&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Knowledge flow&lt;/td&gt;
&lt;td&gt;Per-skill, ephemeral&lt;/td&gt;
&lt;td&gt;Cross-session, compounding&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Adoption pattern&lt;/td&gt;
&lt;td&gt;Copy what you need&lt;/td&gt;
&lt;td&gt;Install once, use everything&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Best fit&lt;/td&gt;
&lt;td&gt;Augmenting an existing workflow&lt;/td&gt;
&lt;td&gt;Replacing the workflow itself&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Neither column is a deficiency. The library shape exists because some users have a workflow they like and want more capabilities to drop into it. The workflow-plugin shape exists because some users — solo founders especially — don&#39;t have a workflow yet and need one that already works.&lt;/p&gt;
&lt;p&gt;The wrong question is &amp;quot;which is better?&amp;quot; The right question is &amp;quot;which one does what I need this week?&amp;quot; Sometimes the answer is one. Sometimes both. Sometimes the answer changes as the work changes.&lt;/p&gt;
&lt;h2&gt;When a Skill Library Wins&lt;/h2&gt;
&lt;p&gt;Three founder moments make the portable-library shape unambiguously the right call.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You need one capability, not a system.&lt;/strong&gt; You&#39;re working on something that&#39;s mostly working. You need a single skill — clean up an Excel export, normalize a JSON schema, generate boilerplate for a new package. You don&#39;t want a new lifecycle. You want a tool you can install in two minutes and forget you have until you need it again. The skill library is the shelf you walk to.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You&#39;re evaluating Claude Code itself.&lt;/strong&gt; If you haven&#39;t decided whether Claude Code is the right substrate for your team, you don&#39;t want to commit to an opinionated bundle yet. Pulling individual skills from a portable library is a low-risk way to feel out the platform — the cost of removal is one folder. Library skills are evaluation-friendly by design.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You already have an orchestration layer.&lt;/strong&gt; Some teams have invested years in their own internal workflow — their own brainstorming process, their own QA gates, their own knowledge management. Layering an opinionated lifecycle on top would conflict. A portable library slots in without claiming the orchestration territory.&lt;/p&gt;
&lt;p&gt;In each of those cases, the opinionated shape would be the wrong fit. The user already has what it would replace.&lt;/p&gt;
&lt;h2&gt;When a Workflow Plugin Wins&lt;/h2&gt;
&lt;p&gt;The other three moments belong to this shape.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You&#39;re a solo founder running every department.&lt;/strong&gt; Marketing campaigns, legal contracts, financial planning, competitive analysis, customer support — every job that a venture-backed company spreads across eight hires. You don&#39;t have a workflow that absorbs context across departments because you don&#39;t have departments. An opinionated organization is the substitute for the team you didn&#39;t hire.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You need decisions to compound.&lt;/strong&gt; A skill-library session 100 starts where session 1 started — fresh, no memory, no accumulated context. For prototyping that&#39;s fine. For running a business it&#39;s a structural problem: every session re-litigates what was already decided. A compounding knowledge base means the brand decisions from January shape the marketing copy in May without the founder pasting them back into the prompt.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You want the lifecycle, not only the tools.&lt;/strong&gt; Brainstorm before plan, plan before work, review before commit, compound before move-on. The lifecycle is opinionated on purpose: it enforces the discipline a solo founder can&#39;t enforce on themselves at 11pm before a release. The skills inside are the executors; the lifecycle is the structure that makes the executors useful.&lt;/p&gt;
&lt;p&gt;These are the moments where the &lt;a href=&quot;https://soleur.ai/company-as-a-service/&quot;&gt;Company-as-a-Service&lt;/a&gt; shape — not a tool, an organization — wins.&lt;/p&gt;
&lt;h2&gt;They Stack&lt;/h2&gt;
&lt;p&gt;The two shapes are not mutually exclusive. A founder running Soleur for cross-domain orchestration can install individual skills from a portable library for one-off capabilities the workflow-plugin shape doesn&#39;t cover. Different layers of the stack: the organization runs the lifecycle, the library skills fill specific gaps.&lt;/p&gt;
&lt;p&gt;The category-creation point holds: choose the shape based on what you need it to do. Run both when both answer different questions.&lt;/p&gt;
&lt;h2&gt;FAQ&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;How does alirezarezvani/claude-skills relate to Soleur?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;It is the canonical exemplar of the portable-library category — a different shape, answering a different question. Soleur is the workflow-plugin shape. They sit in different categories, not on the same leaderboard.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Can I use both?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Yes. The shapes stack at different layers. The organization runs the lifecycle; library skills fill capability gaps the lifecycle doesn&#39;t cover.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Why isn&#39;t there a head-count comparison?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Counting catalog entries against a coordinated organization&#39;s component list is comparing inventory against architecture. The number of library entries says nothing about whether a lifecycle exists, whether knowledge compounds across sessions, or whether agents share context. The right comparison is shape-against-shape, not count-against-count.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Which shape is right for a solo founder running a company alone?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The workflow-plugin shape. The defining trait of solo-founder work — running every department without a team to absorb context — requires the compounding knowledge base and cross-domain orchestration that a portable library, by design, does not provide. Use a library to augment specific capabilities; use an organization to replace the team you don&#39;t have.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;What about installing both?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Common and reasonable. The organization should be the substrate; the library can extend specific capabilities the substrate doesn&#39;t cover. Order matters: install the workflow plugin first, then layer library skills on top where the gaps appear.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;If you&#39;re a solo founder building toward the company that takes every job off your plate, the workflow-plugin shape is what you want as the substrate. &lt;a href=&quot;https://soleur.ai/company-as-a-service/&quot;&gt;See how Company-as-a-Service runs every department&lt;/a&gt; — and decide where the library skills slot in around it.&lt;/p&gt;
</content>
  </entry>
</feed>