Study · Agents

The Conversational Software Factory

Study · Conversational pipeline — naive input · €84/mo server class

AgentForge · · The detailed companion to our blog article, The Software Factory Class

Prefer the short version? The Software Factory Class — the same findings for everyday readers, with an audio reading.

Question: What is the best software-production pipeline for naive conversational input — a non-technical owner talks plainly, working software comes out — on the cheapest reasonable budget, built around a €84/mo Hetzner dedicated server (EX101 class: Intel i9-13900, 24 cores, 64 GB RAM, 2× NVMe, no GPU)?

Candidate architecture under test: OpenWebUI (the owner's window) → Hermes (Nous Research's agent, acting as foreman: translates plain talk into technical briefs, delegates, keeps the builders in sync with the owner's intent, reports back in plain words) → production engines (Claude Code headless as primary builder, Codex CLI as second builder/reviewer, OpenHands for parallel and local-model work), every job isolated in Docker + a git worktree, with automated test gates before any work is accepted.

Researched: 24–25 October 2026. Sources cited inline; retrieval dates are October 2026 unless noted. Companion pieces: our earlier studies on agent runtimes (agentic-coding-powerhouse-runtime-choice-20261025-0522) and local inference on CPU-only servers (local-llm-efficiency-and-gpu-servers-20261025-0517), whose verified findings are reused and marked as such.

Revision (25 Oct 2026): the research below was first read as recommending a local-model foreman with a cloud second brain. On review we reversed that call: the evidence in §1 shows the local model's weakness sits exactly on the foreman's core duty (long dependent tool chains, ~33–38% vs ~44–55% for rented pro-tier models), and the foreman is the owner's daily point of contact. The specified design (§§4, 6, 7) therefore puts a rented Sonnet-class brain in the foreman's seat by default, paid through the Nous Portal wallet, and demotes the local model to fallback and grunt duty. The §1 evidence is unchanged — it is what forced the reversal.

Names, in plain English: Hermes is the foreman software — the body: it listens, remembers, and dispatches. A brain is a language model rented separately and plugged into it. Nous (Nous Research) is the company that makes Hermes. Portal is Nous's prepaid wallet — $20/mo buys $22 of thinking across 300+ brains. Sonnet is Anthropic's pro-tier brain (the senior professional); Opus is Anthropic's top tier (the expensive genius).

1. Can the foreman run on a free local brain? (The all-local question)

1.1 What the job actually demands

Hermes-the-foreman does four things: (1) hold a plain-language conversation with the owner; (2) translate intent into a precise technical brief; (3) operate tools — dispatch jobs to engines, read results, run checks, manage files and sessions; (4) report back in plain words. It does not write the production code. The naive hope is that (1)–(4) are "just talking and organising," so a free local model should suffice. The evidence says the job is really two jobs with very different difficulty:

1.2 The benchmark evidence (BFCL / τ-bench)

Berkeley's Function Calling Leaderboard (BFCL) is the standard tool-use exam. For Qwen3-30B-A3B-Instruct-2507 — the strongest realistic local candidate for this server (see §1.4) — independent evaluations run under the official BFCL protocol report (arXiv 2610.23148, appendix L, BFCL v3, temperature 0.7):

BFCL v3 categoryQwen3-30B-A3B-Instruct-2507
Non-live (single calls)87.0%
Live (real-world calls)77.2%
Multi-turn37.6%
Irrelevance detection85.4%
Overall67.3%

A second independent study (TDScaling, arXiv 2602.03219) tells the same story with different numbers: BFCL multi-turn 33.2% for Qwen3-30B-A3B-Instruct, and τ-bench agentic scores of 30.5 (Airline), 53.3 (Retail), 21.9 (Telecom). A third (AgenticQwen, arXiv 2604.21590) puts its BFCL-V4 multi-turn average at 36.2%. For calibration, the frontier models are better but not magical on the same multi-turn tests: Claude Sonnet 4 scores 54.8%, GPT-5 43.8% (arXiv 2602.03219, Table 1). Single-call function calling, by contrast, is a solved problem at this size: ~83–87% across BFCL and OrchestrationBench (ICLR 2026) measurements.

Reading of the evidence: per-call, the local foreman is right roughly 6 times out of 7 on easy calls and only ~1 time in 3 across a long dependent chain. Errors compound: a dispatch sequence of eight dependent steps at per-step reliability implied by these scores succeeds end-to-end only about a third of the time without correction. Hermes's own error handling (retries, self-correction, checkpointing) recovers some of this, but the direction is unambiguous — the longer and more autonomous the local foreman's own tool chains, the more often the factory floor stumbles for reasons that have nothing to do with the builders.

1.3 What Hermes itself requires and recommends

From Nous Research's own documentation (hermes-agent.nousresearch.com, retrieved Oct 2026):

1.4 Why Qwen3-30B-A3B is the only sane local candidate on this box

Generation on CPU is memory-bandwidth-bound. The i9-13900's DDR5-5600 ceiling is 89.6 GB/s theoretical. A dense 31B model at Q4 reads ~19 GB of weights per token → ~3–5 tok/s in practice: unusable for conversation. Qwen3-30B-A3B is a mixture-of-experts model that activates only ~3.3B parameters per token (~1.9 GB of active weights), which is why it — uniquely in its class — reaches an estimated 15–30 tok/s at moderate context on this exact server class, falling toward 8–18 tok/s as context fills toward 64K (our local-inference study, anchored on published bandwidth arithmetic and community benchmarks; treat as ±2× estimates, not measurements of this exact box). Weights at Q4_K_M ≈ 18.6 GB; total footprint with 64K KV cache ≈ 21.8 GB (q8_0 KV) to 24.9 GB (f16 KV) — comfortably inside 64 GB alongside the rest of the factory, provided llama.cpp's server (not Ollama) is used for its prefix caching, which makes the foreman's long stable system prompt nearly free to re-read each turn.

1.5 Community field reports

No large public corpus exists yet of Hermes specifically running a manager workload on a local model (a gap we flag in §8), but the sibling ecosystem — OpenClaw is architecturally the same pattern (agent harness + tool registry + local OpenAI-compatible endpoint) — has produced a consistent failure taxonomy from production write-ups (clawmage.ai's OpenClaw/Ollama production-patterns series; dev.to post-mortems; Edgible's deployment guides, all retrieved Oct 2026):

  1. Transport traps beat model quality as the #1 cause of "my local agent does nothing." Ollama's OpenAI-compatible /v1 path silently drops/breaks tool calls for agent harnesses (the agent prints raw JSON as chat text instead of executing); the fix is the native API surface. The llama.cpp equivalent for our stack: tool calling requires the --jinja flag, without which the model likewise prints JSON as text (Hermes local-models guides). Lesson: after any stack change, smoke-test a real tool call, not a chat reply.
  2. Context blowout, not stupidity, is the classic mid-task failure. An agent turn carries system prompt + tool schemas + skills + memory — tens of thousands of tokens before the user says a word. If the served context is smaller than the harness believes, truncation silently eats the end of the prompt — where the user's actual message lives. Symptoms: the agent ignores instructions, "forgets" its tools, drifts off-topic. A direct curl chat test passes; the real agent turn falls over (dev.to, "My fresh OpenClaw install kept failing. The model wasn't the problem."). This is precisely why Hermes hard-rejects <64K contexts, and why the local server must be configured with an honest, fixed 64K window.
  3. Format fragility on complex sequences. Models that chat fine emit almost-correct, unparseable tool calls when sequences get complex; small models (7B class) "drop tool calls" outright on multi-step turns. Guides converge on ~27–32B tool-trained models as the practical local floor for agent work — exactly the class §1.4 selects.
  4. Cold starts. Runtimes unload idle models (Ollama's default keep-alive is 5 minutes); reloading ~19 GB costs tens of seconds and reads as a hang. On a 64 GB box the fix is free: keep the foreman's model permanently resident.
  5. The hybrid consensus. Practitioner guides land in the same place: local for the cheap, frequent, private beats; a cheap cloud model (a few dollars a month at Flash/Luna-class prices) where reliability matters — "Most teams settle on hybrid" (betterclaw.io). Our final design sits inside that consensus with the emphasis the BFCL numbers dictate: the rented brain takes the conversation and coordination (the reliability-critical traffic), and the local model takes the bulk work and failure recovery (§4). The closest published twin of our full architecture — a DevOps engineer running Hermes as orchestrator delegating real work to the Claude Code CLI, with Hermes doing QA, git and phone reporting (dev.to, "How I Built a 24/7 AI DevOps Coding Agent for About CAD 200") — likewise uses a cloud model as the orchestrator brain rather than a local one, and reports the pattern works around the clock. That is evidence for the architecture; our §1 analysis says the foreman's seat specifically belongs to the rented brain.

1.6 Conclusion on the all-local foreman

The local model can hold the conversation, take dictation, translate ordinary requests into briefs, dispatch simple jobs and narrate results — the factory's front-office work — at zero marginal cost. What it cannot do is match the rented pro-tier brains at the foreman's core duty: multi-turn chains land at ~33–38% versus ~44–55% for the frontier class (§1.2), and the gap widens precisely when a job gets tangled. We therefore do not seat the local model as foreman by default. The design answer (§4) is a rented Sonnet-class default brain, paid through a prepaid wallet, with the local model kept on staff for the two jobs where it is genuinely the right tool: automatic fallback when the rented brain fails, and the grunt tier (Bay 3 bulk work), where cost-per-word is zero and its weaknesses do not matter.

2. The biggest budget lever: how the engines are billed

The builders — not the foreman — burn the serious tokens. There are two ways to pay for each builder: a flat consumer subscription, or metered API billing. Whether the subscription route is legitimate on a headless server turns out to be sharply different for "the vendor's own CLI" versus "a third-party harness holding the vendor's login." This distinction is the whole ballgame, and as of October 2026 it is documented in the vendors' own words.

2.1 Claude Code on a Claude subscription — YES, through Anthropic's own front door

2.2 The trap: Hermes must not hold the Claude login itself

Hermes's own provider menu offers an "Anthropic (Claude Max + extra usage credits via OAuth)" route. The button existing is not the same as the route being permitted:

2.3 Codex on a ChatGPT plan — YES, same shape

2.4 The foreman's brain, priced

OptionCostNotes
Rented Sonnet-class via Nous Portal Plus (the default)$20/mo wallet ($22 credits); budget $20–50/mo by chat volumeOne OAuth covering 300+ models + tool gateway (portal.nousresearch.com, retrieved Oct 2026). The same wallet can also rent Opus-class thinking for the foreman's own hardest escalations — but it cannot feed Claude Code; the builder bays keep their own subscriptions (§2.1–2.2), and flat plans beat metered Opus pricing for heavy building regardless
Local Qwen3-30B-A3B (llama.cpp)$0/moElectricity only; quality per §1. Seats held: automatic fallback + Bay 3 grunt engine
Cheap API direct (e.g. GPT-6 Luna $0.10/$0.50 per M; Gemini Flash class)~$2–15/mo at light volumesCheapest raw tokens; more keys to manage; a viable wallet substitute
Frontier API foreman at heavy manager traffic~$100/mo hybrid / ~$300/mo all-frontier (our earlier estimate at ~120 agent-hrs of manager traffic)The ceiling this design avoids — the foreman's is conversational traffic, not 120 hours of building

2.5 Why two builder subscriptions, not one top tier

The obvious simplification — drop Codex, put everything on the single top Claude tier (Max 20x, $200/mo) — loses on three counts. Price: the two wages (Max 5x $100 + ChatGPT Plus $20 = $120) undercut the one top tier by $80/mo. Fit: even Max 20x rations Opus (24–40 Opus-hours/wk), so an all-Opus diet runs dry mid-week at any tier; the split design matches models to jobs instead of fighting the allowance shape. Independence: Codex's seat is independent checker — a second vendor's model reviewing Bay 1's diffs catches classes of error that a builder reviewing its own work waves through (§3, §5). And consolidation was never fully available in any case: §2.2's rule means no subscription may be spent inside Hermes, so the foreman needs its own Portal wallet regardless of how the bays are paid.

3. Does this factory already exist in the wild? (Workflows & deployment reality)

Yes — every joint in the candidate architecture is a documented, travelled path, which matters more for a budget build than any benchmark:

4. The foreman's brain design (rented default, local understudy)

Because Hermes has no complexity router (§1.3), brain assignment is assembled from the parts that do exist. The design that falls out of the documentation — and out of the §1 evidence, which is why the rented brain, not the local one, holds the default seat:

  1. Default (rented): foreman on a Sonnet-class model via the Nous Portal wallet — conversation, brief drafting, dispatch, supervision, reporting. The wallet is Portal Plus ($20/mo, $22 of credits); the realistic budget is $20–50/mo depending on chat volume. Rationale: the foreman is the owner's daily point of contact, and its core duty — long dependent tool chains — is exactly where the local model scores ~33–38% against the rented class's ~44–55% (§1.2). Staffing the desk with the evidently weaker brain to save the price of the wallet is false economy.
  2. Failure fallback (automatic): fallback_providers: chain — Portal model → local Qwen3-30B-A3B endpoint. Documented behaviour: triggers on rate limits, server errors, auth failures; swaps mid-session without losing the conversation; one-shot per session. The local model is the understudy: if the rented brain falls over, the factory keeps talking.
  3. Hard-moment escalation (process, not machinery): the foreman's standing instructions (its SOUL/skill layer) define escalation triggers — multi-product scope; anything touching production data, money, or deployment; requirements still ambiguous after one clarification round; a brief whose own checklist fails. On trigger, Hermes re-drafts the brief with an Opus-class model rented from the same Portal wallet and presents the brief itself in Open WebUI for the owner's approval before dispatch. The owner approves words, not code — which is the entire point of the naive-input design. Note the wallet's limit: Portal rents brains for the foreman only; it cannot feed Claude Code, whose bay keeps its own subscription (§2.5).
  4. The local model's standing jobs: besides fallback duty (item 2), Qwen3-30B-A3B is the grunt engine for Bay 3 — OpenHands bulk and parallel work, where zero cost-per-word beats chain reliability — plus bursty background chores (compression, titles); summarisation is its strength. Model pinned resident on the server; nothing quality-critical rides on it.
  5. Engine-side second opinions: Codex reviews Claude Code's diffs as a standing order; the test gate is mechanical and cannot be talked past. Two consecutive gate failures on one job bounce the job back to the owner with a plain-language incident note rather than a third silent retry.

Expected escalation volume beyond the default brain is small — a handful of hard briefs a week — which is why the one $20 credit wallet (Portal Plus carries $22 of credits) covers both the default seat and the Opus-class escalations at one-owner chat volumes, with headroom to ~$50/mo before the wallet size, not the design, needs revisiting.

5. Failure modes & mitigations

#Failure modeEvidenceMitigation in the blueprint
1Foreman fumbles long tool chainsBFCL multi-turn 33–38% local / ~44–55% frontier (§1.2)Rented pro-tier brain in the foreman's seat (§4); short chains only; delegate whole jobs to engines; retries + §4 escalation
2Silent context truncation mid-shiftCommunity failure taxonomy (§1.5)Fixed honest 64K ctx on llama-server; lean skill set; Hermes's startup floor respected
3Tool calls print as text, never executeOllama /v1 trap; llama.cpp needs --jinjallama-server (not Ollama) with --jinja; tool-call smoke test after any change
4Cold-start hangsModel unload defaultsModel pinned resident in RAM (64 GB makes this free)
5Chat slows while builders compileOne box, 24 coresDokploy/systemd CPU fences: inference guaranteed a slice; agent world capped at ~⅓ of the box (standing fencing design)
6A builder's weekly subscription cap lands mid-project§2.1–2.3 limit structuresTwo vendors (caps don't align); API overflow wallet; heavy work scheduled early in the allowance week; usage checked weekly
7Terms-of-service tripwireAnthropic Legal page, Feb–Apr 2026 (§2.2)Hermes never holds engine logins; engines are the vendors' own CLIs, self-authenticated; single owner, own use
8A mistranslated brief builds the wrong thing wellThe telephone-game risk, named in our architecture sessionsBrief is a written artefact; owner approves big briefs in plain words before dispatch (§4.3)
9Builder forgets the project over a long jobContext-decay case study (§3)Fresh worktree + fresh engine session per job; durable memory lives in the foreman, not the builder
10The one server diesSingle-box designEverything in git (pushed to GitLab continuously); configs in the Dokploy project; factory rebuild = one afternoon from this blueprint

6. The full monthly bill

Workload assumption (stated so the table can be argued with): ~120 engine-hours/month of real building, split roughly ⅔ Claude Code / ⅓ Codex + OpenHands; a handful of hard briefs/week escalated. Currency: €1 = $1.12 (25 Oct 2026, Finnhub). Server price: Hetzner's own launch listing — EX101, €84/mo + €39 one-time setup (Hetzner press release; spec confirmed in Hetzner's hardware docs: i9-13900, 64 GB DDR5 ECC, 2× 1.92 TB NVMe Gen4). Re-confirm the order-page price at checkout; €84 is a launch price and secondary listings show the model trading higher.

LineLean buildRecommended factoryAll-API equivalent
Server (EX101)$94 (€84)$94 (€84)$94 (€84)
Foreman brain (Sonnet-class via Portal wallet)Portal Plus — $20Portal wallet — $20–50Frontier API manager — ~$300
Local model (fallback + Bay 3 grunt)Qwen — $0Qwen — $0—
Claude CodePro — $20 ⚠ tightMax 5x — $100Metered API (in engine line)
CodexPlus — $20Plus — $20 (Pro $100 on evidence)Metered API (in engine line)
Engines on API instead of plans——~$325 (Sonnet-class, same workload, our token model)
OpenWebUI / Dokploy / Hermes / llama.cpp$0 (open source)$0$0
Total / month≈ $154≈ $235–265≈ $719

Notes on the table: the Lean column works only with Sonnet discipline and tolerance for cap friction — Pro's weekly envelope is shared with the owner's personal Claude use. The Recommended column's headroom is deliberate: Max 5x's nominal weekly Sonnet envelope (140–280 h) covers the Claude share (~80 h/mo ≈ 18 h/wk) several times over even under the class action's pessimistic re-rating, and Codex Plus is the starting tier with a defined upgrade trigger. The foreman-brain line is a range because the Portal wallet is metered credit: the $20 Plus tier ($22 of credits) covers an ordinary month of one-owner conversation, and $50 is the honest top of the expected band before the wallet size — not the design — needs revisiting (§4). The All-API column uses our earlier verified token model (~1.5M tokens per agent-hour, cache-adjusted, Sonnet-class $2/$10) — it is what this same factory costs with no subscriptions and every brain at full metered price, i.e. roughly 3× the recommended build. One-time costs: €39 server setup (≈ $44). Electricity, bandwidth and the owner's own internet are inside the rent.

7. Verdict — the software factory, specified

The verdict in one breath: build the factory exactly as the candidate architecture describes, with one correction the research forced — the foreman's brain is rented, not local: a Sonnet-class brain on a $20–50 prepaid wallet staffs the desk, the free local model stands by as fallback and does the grunt work, and hard briefs escalate to Opus-class thinking from the same wallet under a human-approval process. That is the difference between a charming demo and a factory that survives its first tangled order. Everything else in the candidate design survived contact with the evidence, including the part that surprised us: the engines' flat subscriptions are not a loophole — used through the vendors' own CLIs, they are the documented, intended way to run this, and they are worth ~$200–480/mo against metered billing at this workload.

The blueprint (Dokploy project: "the factory"):

What we are NOT claiming: that the free local brain was good enough to face the owner all day — the evidence says it is not (§1.2), and the design never asks it to. The local model works where its weaknesses do not matter: bulk grunt work and emergency fallback. The factory works because judgement-facing work — the conversation, the briefs, the building, the review — rides on rented pro-tier brains and mechanical gates, while the foreman's own job stays narrowed: talk, translate, dispatch, narrate.

8. Claims we could NOT verify (flagged, not buried)

  1. Exact local tok/s on the real EX101. No public benchmark of Qwen3-30B-A3B Q4 on an i9-13900 at 64K exists; our 15–30 / 8–18 tok/s figures are estimates synthesised from bandwidth arithmetic and adjacent benchmarks (our local-inference study). First act after the server lands: measure, don't trust. (The figure now bounds Bay 3's grunt throughput and fallback quality rather than the daily conversation, which rides on the rented brain.)
  2. BFCL V4 official-leaderboard overall for Qwen3-30B-A3B-Instruct-2507. The official board is script-rendered and did not yield its table to text retrieval; §1.2 therefore relies on three independent replications run under official BFCL protocols (v3 and V4 subsets), which agree with each other (multi-turn 33–38%) but are not the board itself.
  3. Anthropic's 6 May 2026 limit-doubling and current effective allowances. The doubling is reported by multiple secondary trackers, not confirmed here from an Anthropic page; and the effective weekly allowance is genuinely contested (see the Kahn filing's allegations vs Anthropic's marketing hours). The blueprint survives either way — Max 5x has headroom under both accounts — but the exact hours should be treated as soft.
  4. OpenAI's Codex weekly allowance. OpenAI publishes 5-hour estimates and says weekly limits "may apply"; it does not publish the weekly numbers. Plus-tier sufficiency at our workload is therefore a judgement, with Pro $100 as the defined fallback.
  5. Hermes-as-manager field reports, specifically. The field-report corpus (§1.5) is dominated by OpenClaw (same pattern, different harness) plus general Hermes setups, most of them with cloud brains; a large body of Hermes-foreman production reports does not yet exist in either brain configuration. The BFCL evidence is model-level and solid; the harness-level interaction (Hermes's retries and scaffolding recovering model errors, local or rented) is plausible but unmeasured in public.
  6. Long-term tolerance of setup-token-driven workloads. claude setup-token is documented by Anthropic for CI/scripts today (Oct 2026). Whether Anthropic further restricts unattended, agent-dispatched subscription workloads in future terms revisions is unknowable; the fallback if it ever changes is metered API for Bay 1 (~$325/mo at this workload) — painful, not fatal, and the reason the blueprint keeps an API wallet warm.

Sources (retrieved 24–25 Oct 2026 unless noted)

← All field notes