CabrIA · Local AI for Software Testing, Built on Your Field Evidence
CabrIA (an AI Cabra R&D program) is a local, sovereign AI for software testing, under development: an open 35-billion parameter Mixture-of-Experts model served locally, paired with a verifiable local RAG (qa-brain) in which every answer cites its sources. Your conventions, screens and incidents provide context that generic models lack. Capture and refinement produced 1,892 accepted examples as of 9 October 2026; the first fine-tuning run remains pending. A cost-controlled laboratory setup on local infrastructure in France.
The Dead End of Generic LLMs in Enterprise QA
For enterprise IT leadership requiring compliance and high precision, delegating software quality to public cloud APIs presents three critical bottlenecks.
Ignorant of Your Custom Landscape
Public AI models know neither your custom SAP fields, nor your authorization workflows, nor your historical regression patterns. Their recommendations remain vague and unusable.
Exfiltration of Sensitive Test Data
Routing production screenshots, execution logs, or API payloads through external cloud APIs exposes confidential data to foreign extraterritorial regulations and security audits.
Unverifiable Recommendations
Standard LLMs invent non-existent UI locators and cannot cite authoritative sources. Relying on them for critical release gate approvals introduces dangerous silent regressions.
Generic cloud LLM or specialised local AI: what changes for QA
Two approaches compared without naming any vendor; the last row says where CabrIA stands.
| Criterion | Generic cloud LLM | CabrIA, local AI in R&D |
|---|---|---|
| Where test data goes | To the vendor's servers | It stays with you, on-premise or in a sovereign cloud (France, EU); secrets and personal data masked before storage |
| Knowledge of your conventions | None, beyond what the prompt contains | Served by the RAG (in service), learned by fine-tuning (planned) |
| Sources of answers | No source cited by default | Mandatory citations in qa-brain, abstention possible outside the corpus |
| Model version | Changed by the vendor | Frozen, chosen and measured on a 77-case benchmark |
| Availability today | Available product | R&D programme: RAG in service, first fine-tuning to come |
The CabrIA Continuous Learning Loop
Every test execution produces empirical experience: passively captured, verified by execution replay, and stored in a sovereign dual memory.
What runs, what is in progress, what remains to be done
Status read on October 6, 2026 in the programme roadmap. A step is declared done there only when its exit criterion is met.
- Flight recorder: a Robot Framework listener, a Playwright reporter, an rf-mcp plugin and Playwright MCP hooks capture the runs; 4,146,190 records consolidated on October 9, 2026.
- Refinery: 1,892 admitted examples on October 9, 2026, scored by a separate 30-billion-parameter judge.
- Evaluation benchmark: 77 cases on September 11, 2026, built before any training.
- qa-brain RAG memory: 4,119 passages indexed on October 9, 2026, answers with mandatory citations, queried by the test agents of SAPFX and rf-test-agents.
- Diversifying the dataset beyond SAP, towards the web and Odoo.
- Systematic validation of generated code by execution on offline fixtures.
- Preference pairs for the DPO pass.
- First LoRA bf16 fine-tuning of the open model, on a local workstation.
- Comparison of the trained model with the base model on the benchmark, with an exit criterion set in advance: a net gain on at least three axes, with no major regression.
- Local serving of the trained model and connection to the test agents as a local model.
No fine-tuning gain is claimed until this comparison is done.
The 4 Founding Pillars of the CabrIA Ecosystem
From passive collectors to locally served models, every component is built for universal portability, secret scrubbing, and systematic execution validation.
Mixture-of-Experts Architecture, On-Premise or Sovereign Cloud
Apache 2.0 MoE model with 35 billion parameters, including ~3 billion active per token, served locally in GGUF. The first LoRA bf16 training run is planned on a cost-controlled laboratory setup (128GB unified memory). Its gains will be compared with the base model on the 77-case benchmark.
Flight Recorder & Data Protection
4,146,190 records consolidated as of 9 October 2026, including 2,603,689 from SAP tests, 1,423,915 from web tests and 82,851 from Odoo tests. A Robot Framework listener, Playwright JSON report ingestion, an rf-mcp plugin and MCP hooks feed capture with secret masking. A captured record is not an accepted training example.
SFT/DPO Refinery & Verification by Real Execution
1,892 examples accepted as of 9 October 2026, including 917 test critiques, 526 diagnostics, 313 specification-suite pairs and 136 lessons learned; 52 duplicates are flagged. A separate 30-billion parameter judge scores candidates, with confirmation and traceability. Systematic offline-fixture validation of generated code and DPO pairs remain milestones to complete before training.
Dual Memory: Verifiable Local RAG via qa-brain
4,119 passages indexed as of 9 October 2026: versioned documentation and field lessons. Qdrant combines dense retrieval, supplementary BM25 and reranking. The qa-brain MCP server, in service, requires citations (an answer without citations is rejected and replaced by the raw passages) and can abstain; the paired comparison found neither gain nor degradation in supplementary mode on the 19 benchmark cases. To see it at work: the qa-brain chapter of the film “How AI agents drive SAP” (5:12).
How CabrIA connects to Robot Framework, Playwright and MCP agents
CabrIA replaces no test tool: it observes the runs and serves a memory to the agents.
It observes your runs
A Robot Framework listener, a Playwright reporter, a plugin for the rf-mcp server and hooks for Playwright MCP record runs, locator repairs and agent sessions, without changing the tests. Secrets and personal data are masked before any storage.
It answers with its sources
The local qa-brain MCP server exposes three tools: qa_search, qa_ask and qa_status. The test agents of SAPFX and rf-test-agents query it before their judgement calls (which anchor holds, which failure class); an answer without citation is rejected, and observation on the real system always decides.
It prepares a model that will know your conventions
After sorting and anonymisation, the captured runs become training examples: test critiques, diagnoses, specification and suite pairs, lessons. The trained model will be connected to the agents after its comparison with the base model.
See also: the rf-test-agents test agents, the CabrIA bricks in the SAPFX workbench and the qa-brain chapter of the film “How AI agents drive SAP”.
The 3 Ethical Pillars of CabrIA
Rigorous engineering safeguards ensuring complete traceability and human oversight.
Nothing heals silently
Opaque self-healing introduces silent technical debt into production. With AI Cabra, every locator update is explicitly logged with detailed warnings and a permanent patch recommendation.
Humans remain in command
AI accelerates root-cause investigation and proposes fixes. The domain expert or QA engineer retains complete authority to review and approve changes before test execution merges.
Absolute territorial sovereignty
Models are deployed on your internal servers or in a sovereign cloud, in France or the European Union. Your business workflows and test data remain completely shielded from third-party inspection.
Key Questions on CabrIA & AI for Quality Engineering
Everything you need to know about the MoE architecture, sovereign dual memory, evaluation testbed, and the FastMCP qa-brain server.
Why is a local 35-billion parameter MoE model better suited than a generic cloud LLM for QA?
The selected MoE model activates about 3 billion of its 35 billion parameters per token. It can be served locally on a cost-controlled laboratory setup with 128GB unified memory. This keeps data on private infrastructure; it does not establish general QA superiority. The first fine-tuning run and its comparison with the base model remain pending.
How does CabrIA's dual memory work (Fine-Tuning vs qa-brain RAG)?
CabrIA strictly separates engineering know-how from volatile facts (ADR 0002). Planned fine-tuning (LoRA SFT followed by DPO), not yet performed, is intended to teach durable methods through model weights: Robot Framework and Playwright conventions, root-cause diagnosis, and bounded self-healing. Local hybrid RAG (Qdrant + BM25 via FastMCP qa-brain) serves documentation and distilled field lessons, with an absolute requirement to cite sources under penalty of rejection.
Why does CabrIA mandate validation by execution on offline fixtures?
Plausible code can fail at runtime: a judge score alone cannot validate it. The refinery uses a separate 30-billion parameter judge and score confirmation. Systematic offline-fixture validation of generated code remains to be completed before training; the 1,892 examples accepted as of 9 October 2026 are not 1,892 scenarios executed under that protocol.
What on-premise hardware is used to train and serve CabrIA?
CabrIA uses a cost-controlled laboratory configuration: two workstations with 128GB unified memory, intended respectively for planned LoRA bf16 training and local GGUF serving. Historical throughput of about 100 tok/s depends on the model and measurement conditions; it is neither a performance commitment nor a post-fine-tuning result. The anti-forgetting mix remains to be defined.
Is CabrIA available today?
Not as a product. CabrIA is an AI Cabra R&D programme with no published release. What already runs: test run capture, the refinery, the evaluation benchmark and the qa-brain MCP server, which the test agents of SAPFX and rf-test-agents query. The first fine-tuning is planned; private demos and pilots are scoped on quote.
Can a local LLM be used to automate software testing?
Yes, that is the purpose of CabrIA: an open model served on-premise or in a sovereign cloud (France, EU), without sending test data to a third-party cloud. The part in service today is the qa-brain RAG memory, which answers with its sources; the model specialised by fine-tuning is in preparation and will be compared with the base model on a 77-case benchmark before any gain is announced.
RAG or fine-tuning: which one for an AI test assistant?
Both, for two uses (ADR 0002). RAG serves the facts that change, such as the documentation of installed versions and field lessons, and cites them; fine-tuning should learn durable methods, such as test conventions and failure diagnosis. In CabrIA, RAG is in service and fine-tuning is planned, its gain still to be measured.
Does test data leave the company with CabrIA?
No: model, RAG index and captured data stay with you, on-premise or in a sovereign cloud (France, EU), and secrets and personal data are masked before any storage. If an agent queries qa-brain from an AI assistant hosted outside that perimeter, the passages it receives go through that assistant; a fully sovereign loop requires an agent on a model served on-premise or in a sovereign cloud, which the rest of the programme plans.
Measured results and validation scope
Explore local QA AI combining an open model and hybrid RAG. Reviewed on 9 October 2026. Internal R&D status; no independent validation or customer outcome is claimed.
- 1,070 automated tests passed Run on 9 October 2026 in a single process: 1,070 passed out of 1,071 (capture 352, evaluation 130, orchestration 38, RAG 165, refinery 386). The remaining test queries the lab's live services and timed out. They verify CabrIA software, not the model's overall answer quality. Internal test run, 9 October 2026
- 1,892 accepted training examples Stock recorded on 9 October 2026, including 917 test critiques, 526 diagnostics, 313 specification-suite pairs and 136 lessons learned; 52 duplicates are flagged. The first LoRA training run and before/after comparison remain pending; no fine-tuning gain is claimed. Internal log, 9 October 2026
- 4,119 indexed passages State recorded on 9 October 2026 on the qa-brain server in service. Dense retrieval and supplementary BM25 are operational; the 11 September comparison on 19 cases found no gain or degradation in supplementary mode. Index size does not measure answer accuracy. Internal reading, 9 October 2026
- 77 evaluation cases Benchmark documented on 11 September 2026, including 19 RAG cases. On the 16 positive RAG cases run on 30 August: hit rate and groundedness were 0.875 before review and 1.0 after broadening two criteria; all 3 abstention controls passed. This result applies to the tested corpus, not to a general absence of hallucinations. Dated internal evaluations, without independent validation
Interested in an on-premise CabrIA testbench for your enterprise?
The evaluation bench and capture engine are live at AI Cabra. Contact our architects to schedule a private walkthrough or plan a local fine-tuning proof of concept.
- A technical exchange, not a sales pitch
- Your data stays in France