Conceptual architecture; not a claim about the linked repository’s exact implementation.
The question behind the prototype
A coding assistant is not one indivisible reasoning problem. It must identify relevant files, localize a defect, inspect code structure, generate a candidate fix, review that fix, produce tests and package a patch. These stages differ in how much context they need, how constrained their outputs are and how independently their results can be checked.
My slm-cascade project explores whether smaller language models can handle the bounded stages while larger models focus on open-ended planning and code generation. The hypothesis is about division of labor, not a claim that smaller models are universally equivalent to larger ones.
What the code actually implements
The uploaded implementation defines a seven-stage LangGraph pipeline: orchestrator → fault localizer → code analyzer → code generator → code reviewer → test generator → patch formatter. It also defines a conditional review loop: when review fails, the graph can return to analysis and generation with a bounded retry counter. Shared workflow state carries intermediate outputs across stages.
Tree-sitter and other AST utilities expose structural information to the localization and analysis stages. Python syntax checks and diff utilities provide deterministic support for parts of the review and patch-generation process. This is important: a small model need not solve an unconstrained problem when conventional software can narrow its inputs or validate part of its output.
Model specialization, not one-size-fits-all inference
The current configuration in files/config/models.py assigns Qwen2.5-72B-Instruct to orchestration and DeepSeek-Coder-V2-Instruct to code generation as primary larger-model choices. Several bounded stages are configured with Phi-4-mini, including localization, analysis, review, test generation and patch formatting. The registry also specifies fallback configurations. These are configuration choices in the uploaded code, not evidence that each provider endpoint was available or that all stages completed successfully in a production run.
The README documents an earlier design using Mistral-7B, Phi-3-mini, Qwen2.5-Coder-1.5B and DeepSeek-Coder-6.7B. I treat that as a design iteration rather than presenting the older table as the current implementation.
Why structured inputs change the model-selection problem
Consider fault localization. Asking a model to reason over an entire repository is different from asking it to rank a short list of functions extracted by an AST parser. The second task has a narrower search space and a more checkable output. Similarly, patch formatting is better handled primarily by deterministic diff generation than by paying a large model to reproduce syntax that a library can generate.
This suggests a general principle: before upgrading the model, reduce the task’s ambiguity. Supply typed context, limit the decision surface and use external checks wherever possible. Model size should be chosen after designing the task interface, not before.
Where verification helps—and where it does not
A parser can reject syntactically invalid Python. A reviewer can flag suspicious changes. Regression tests can expose some functional mistakes. None of these alone proves that a patch resolves a real issue without introducing regressions. The appropriate outcome measure for code repair is verified resolution on held-out repository issues, not merely whether a generated diff looks plausible.
The repository contains a SWE-bench runner and analysis utilities, but the provided predictions.json includes empty model_patch fields. I therefore do not report a SWE-bench solve rate, measured latency improvement, cost reduction or quality parity with an all-large-model baseline. Those remain evaluation questions.
What a meaningful evaluation would compare
A fair study would hold the issue set, tool interfaces, retry budget and validation rules constant while comparing three configurations: all-large-model stages, the hybrid stage-specialized pipeline and an all-small-model control. Primary outcomes would include verified issue resolution, valid-patch rate, cost per resolved issue, latency and the rate of unnecessary escalation or retries.
Ablations should test whether AST-based context narrowing improves localization, whether deterministic patch formatting reduces malformed outputs, and whether the review loop adds enough verified success to justify its extra calls. A hybrid design is only better if the savings survive the costs of orchestration, verification and recovery.
The broader engineering question
The project is a concrete way to explore a question that matters beyond coding assistants: how should an AI solution allocate capability to subtasks? Some work may require stronger reasoning; other work benefits more from better structure, specialized tools and reliable verification.
The possibility is a task-aware system that uses expensive inference selectively rather than universally. The evidence required is not a model-size slogan, but reproducible measurements of end-to-end task outcomes.