Forge

\n\n## Why Small Models Crash and Burn on Agentic Tasks\n\nHere's a pattern anyone running a local LLM knows too well: you ask your little 8B model to book a trip. It handles the flight lookup, then tries to call the hotel API with a malformed JSON payload, and the whole workflow dies mid-execution. Not because the model is stupid—it just ignored the tool's expected format.\n\nThis is the core problem Forge solves. Author zambelli understood it clearly: small models don't lack knowledge, they lack behavioral guardrails. So instead of upgrading to GPT-4 or Claude, Forge gives the model structural constraints that keep its actions within legal boundaries.\n\nThe HN thread (687 points, 237 comments) sparked niche but technical discussion. User c0rruptbytes tested Forge with ternary bonsai 8b lcm 2bit (a quantized-to-2-bit small model), and the numbers were telling:\n\n| Metric | Bare | With Forge |\n| --- | --- | --- |\n| Accuracy | 25.0% | 66.7% |\n| Completion rate | 33.3% | 100% |\n| ToolCall errors | 62 | 2 |\n\nThat's not prompt engineering magic—that's structural error correction catching malformed tool calls and nudging the model back on track. The improvement comes from reducing execution failure, not boosting reasoning capability.\n\n## Not Prompt Tricks, Structural Constraints\n\nForge is a pure Python framework with serious infrastructure: proxy layer, OpenAICompatClient for multi-backend support, middleware architecture, 67 commits, 10 version tags. From v0.7.0 improving error channels to v0.7.1 adding Claude Code proxy mode (Anthropic -> Forge -> OpenAI format translation), this is not a weekend demo.\n\nThe technical depth matters. zambelli told tommica a concrete example: if your Pi agent asks to book travel and suggests flights, hotels, cars—it gets flights right but sends malformed JSON to the hotel API. The whole workflow dies with partial completion. Forge catches that and injects context to get the model back on track. This is mechanistic error recovery, not content improvement.\n\nAs blurbleblurble put it: " , ". zambelli agreed: big models already have decent harnesses—Claude Code works because Opus has that foundation. Small models aren't there yet; Forge nudges them back onto the road. Whether they have a good sense of direction is a separate question.\n\n## Who This Is For\n\nPerfect fit: developers quantizing small models locally, cost-conscious environments, anyone hitting tool call format errors with 7B-13B models—essentially the same crowd as schaefer who said: " :"#.*1\\."