Best AI Model for RFP Responses: Our 2026 LLM Benchmark

We started running evals 1 month ago. It has become 1 of our most valuable product investments. The setup takes time, costs money and comes with a steep learning curve, but it shows us exactly where our AI succeeds and where it fails. So far, we have run more than 60 tests across 15 models, generated 3,500 pages and processed 458M tokens.

At Cobl, our customers use AI to prepare RFP responses: go/no-go dashboards, technical proposals, security questionnaires and more. These opportunities often exceed €1M. Buyers read and score the documents, and a fabricated claim, an unfinished section or a missed mandatory clause can disqualify a bid. Every week, we ask the same question: is this response ready to send? Our evals help us understand what needs to improve before the answer can be yes.

Key takeaways

Cobl tested 15 AI models on the same real public-sector RFP, with 7 scripted messages, 10 planted contradictions and a 100-point quality grid.
Astra 6 still scores highest at 87/100, but Opus 5.5 reaches 85/100 for $11.32, about a sixth of Astra's $68.09.
Opus 5.5 was also the quickest of the top scorers, delivering its response in 27 recorded minutes against Astra's 65-minute session.
GPT Sol 5.6 remains Cobl's default at 73/100 and $8.05, but the Opus 5.5 result is a reason to retest that choice.
Running the same model through Cobl scored 14 to 22 points higher than running it directly in ChatGPT or Claude.ai, across 3 comparisons.

Evals in 1 paragraph

An eval is a repeatable test. We give several AI models the same RFP, knowledge base and scripted user messages. We collect their documents, conversations and traces: the record of which sources they read and which skills and tools they used. We then assess them against a consistent set of rules. The result is a quality ranking, a cost and speed report, and a list of problems assigned to the people responsible for fixing them.

The test

We use a real public-sector RFP. It includes 3 lots, a 40-page specification, an evaluation grid and an annex with requirements that can disqualify a bid. We also use a real knowledge base with hundreds of documents in it.

Each model receives the same 7 messages in the same order. The conversation moves from the brief and architecture diagrams through corrections, technical and pricing documents, advice requests, permission to proceed and customer references. It should produce 3 deliverables: a technical memo, a financial proposal, and a references and governance document. We assess both the conversation and the final files.

The source material is deliberately messy. We planted 10 contradictions in the test files, covering inconsistent prices, conflicting quantities and gaps between the proposed infrastructure and the RFP's requirements. The knowledge base adds other challenges: conflicting certification claims, missing product information and standard contract wording that clashes with the RFP.

We want to see how the agent handles uncertainty. Does it compare sources, explain conflicts and ask for a decision when needed? Or does it quietly choose whichever answer seems most plausible? Finding a problem is only part of the job. The agent also needs to respect the user's authority and carry unresolved issues into the final documents.

The criteria

We score quality across 9 criteria, for a total of 100 points.

Heading	Points	What decides the score
Accuracy	15	Are factual claims and figures supported by the knowledge base, the RFP or the user messages? We also check for invented prices. Even a true fact from the model's training counts as fabrication in this test if the approved sources do not support it.
Context understanding	15	Did the agent read the RFP, diagrams and uploads before writing? Did it understand the scope, partner responsibilities and conflicting instructions, and retain that context across 7 turns and 3 documents? We check the traces to find out.
Completeness	15	Were all requested documents delivered, with the expected length, depth and supporting figures?
RFP requirements	15	Did the response follow the required structure and answer every mandatory commitment? This includes an explicit penalty table meeting the RFP's minimum requirements.
Quality of reasoning	10	How well did the agent explain its recommendations and answer the user's questions? Did it stay within the decisions the user had authorised it to make?
Issue spotting	10	Did it identify the 10 planted contradictions and any additional problems? An issue raised only in chat earns 50% credit; carrying it into the document earns full credit.
Writing quality	8	Is the writing clear, professional and in French throughout, as required for this RFP? Does it avoid unfinished text, internal notes and traces of the generation process?
Persuasion	7	Does the response make a convincing case using supported claims? Persuasion based on exaggerated or unsupported promises scores below the midpoint.
Final document quality	5	Are the structure and typography clean? Does the delivered file contain the work and corrections the agent said it had completed?

Cost and time sit alongside this quality score. We show them in separate charts: total response cost and recorded delivery time. We choose a model by looking at all 3 dimensions.

Where the models stand

The updated charts cover 15 models through Cobl and 3 direct runs in ChatGPT or Claude.ai. The 23 September update adds Opus 5.5, GPT Sol 6, GPT Luna 6 and a direct Opus 5.5 comparison. Opus 5 remains in the charts so we can follow the evolution alongside the newer version. These are selected runs across different test dates and application versions. White bars with grey outlines identify direct runs; separate charts compare each model across the 2 environments.

Each plotted result comes from 1 graded run. These results show how the models behaved in this scenario; they are not a definitive ranking across every RFP.

Quality

Which model produced the strongest response? The scores reveal different strengths, but also different ways of failing.

The scores show why we look beyond the overall ranking.

Astra 6 combines thorough analysis with strong source discipline. It reads extensively before drafting, checks the provided documents against the RFP and catches all 10 planted contradictions. It also identifies less obvious conflicts in schedules, responsibilities and reused content. Crucially, it explains the financial discrepancies in the final document and quantifies the impact of a technical shortfall. It provides an explicit penalty table and avoids importing unsupported facts, prices or certifications.

With an accuracy score of 14/15, it still has weaknesses: it repeatedly assigns a partner's responsibility to the bidder, and its cautious language weakens some commitments. Even so, its graded run was the 1st complete response this month to pass all our mandatory checks.

Opus 5.5 comes close to Astra at a much lower cost and shorter recorded time. It scores 85/100 for $11.32 and 27 minutes. Its strongest areas are context understanding, RFP requirements and issue spotting. The response is structured and the financial corrections are carried into the files. But consistency across documents remains a weakness: technical promises do not always match the infrastructure and services priced in the financial proposal. Its accuracy score of 11.5/15 leaves it below Astra on source discipline, and the response still needs review before submission.
Keeping Opus 5 shows how the results have evolved. Its retained run scores 79/100 for $17.10 and 93 active minutes, compared with 85/100, $11.32 and 27 recorded minutes for Opus 5.5. That is 6 additional quality points, about 34% lower cost and a shorter recorded duration. Opus 5 identified important problems but did not reliably apply its findings in the final files. Opus 5.5 improves requirements coverage and delivery, while inconsistencies between technical and financial commitments remain. Because these runs come from different dates and harness versions, the changes cannot be attributed to the model upgrade alone.
Sol 6 improves on the earlier GPT Sol result, with trade-offs. It scores 80/100 for $21.66 and 48 recorded minutes, compared with 73/100 for the retained GPT Sol 5.6 run. It scores well on accuracy, context and completeness, but remains weaker on writing and persuasion. This is a comparison of selected runs from different test dates, not an isolated measure of the model upgrade.
Fable 5.1 is comprehensive, but takes decisions beyond its authority. It earns full marks for completeness and RFP requirements, with detailed coverage, supporting figures and an explicit penalty table. Its accuracy score is only 2/15. It treats unapproved customer references as cleared, substitutes public prices for the figures in the provided documents and adds costs based on its own assumptions. It explains these choices in chat, then implements them anyway. The problem is that it treats identifying a possible improvement as permission to change the offer.
GPT Sol 5.6 offers the most balanced profile for our default use. It fabricates no facts, identifies conflicting source information and proposes clear, traceable financial corrections. It scores 73/100. Its main failures concern follow-through: a correction announced in chat does not reach every document, the references document is not exported, and the penalty commitment lacks the required table. Its reasoning is useful and grounded, but the delivery process needs stronger checks.
GPT Terra 5.6 and GPT Luna 5.6 stay accurate, but provide less depth. They score 15/15 and 14/15 on accuracy, with overall scores of 65/100 and 55/100. Their proposals reach roughly 1/3 of the target length, include few supporting figures and identify fewer issues. GPT Terra accepts the penalty requirements without supplying the table; GPT Luna defers the answer. Their restraint reduces unsupported claims, but leaves more analysis and completion work to the user.
GPT Luna 6 improves context understanding and issue spotting, but still misses requirements. It reaches 65/100, with 14/15 on accuracy, for an estimated $1.41 and 42 recorded minutes. Its 5/15 on RFP requirements and 9/15 on completeness show the remaining work: a low-cost response still needs to cover the commitments and depth the buyer expects.
Sonnet 5 does not challenge the source material enough. It carries financial errors into the response, treats a discrepancy as rounding and confuses certification wording. It raises few substantive reservations, misses the penalty clause and does not export to Word. The central weakness is insufficient verification before presenting the answer.
Gemini 3.8 Flash is fast and confident, including when the sources do not justify confidence. It catches 0 planted contradictions and describes conflicting financial files as fully aligned. It adds unsupported claims about qualifications, data-loss guarantees and legal protection, including a qualification error that automatically fails our checks. It does answer the governance requirements, explicitly accept the penalty minimums and deliver all 3 documents in 42 minutes. But its polished, assertive writing masks uncertainty the user needs to see.

What does Cobl add to ChatGPT and Claude.ai?

Choosing a model answers only part of the question. If that model is already available in ChatGPT or Claude.ai, what does using it through Cobl add? We compare Astra 6 in Cobl and ChatGPT, and Fable 5.1 and Opus 5.5 in Cobl and Claude.ai. The Astra and Fable comparisons use the same RFP, knowledge base, 7 scripted messages and 100-point scoring grid. For the new direct Opus run, the documents suggest equivalent source material, but no conversation trace is available to verify the exact sequence of messages. Its assessment therefore covers document evidence only.

These comparisons assess the whole working environment around the model: the instructions, skills and tools that guide how it reads sources, checks requirements and delivers documents. The practical question is whether Cobl produces a more complete, usable RFP response, with fewer omissions and less work left to the user. The charts below show the overall scores; the analysis explains where Cobl helps and where it still falls short.

RFP Response with Astra 6

Astra 6 scores 87/100 in Cobl and 65/100 in ChatGPT, a 22-point difference in this test. Both runs stay close to the approved sources, with accuracy scores of 14/15 and 13/15. The larger difference is how that knowledge becomes a finished response. In Cobl, Astra checks the requirements more thoroughly, carries identified problems into the documents and supplies the mandatory penalty table. In ChatGPT, it produces a shorter response, leaves that requirement unresolved and includes internal review language in the client document.

For the user, this means less completion and editing work after generation. The clearest gains are in RFP requirements, issue spotting and writing quality. The corrected Cobl run costs $68.09 and spans a 65-minute session window, compared with $25.37 and 67 recorded minutes in ChatGPT. Cobl produces the stronger response at a higher cost, with a similar recorded delivery time.

RFP Response with Fable 5.1

Fable 5.1 scores 74/100 in Cobl and 60/100 in Claude.ai, a 14-point difference. In Cobl, it delivers a more complete response, supplies the penalty table and identifies financial contradictions that the direct run misses. Its issue-spotting score rises from 3/10 to 8/10. The direct run also leaves more cleanup for the user, including spreadsheet references in the prose and a table announced but never written.

The improvement is uneven. Fable's accuracy falls from 8/15 in Claude.ai to 2/15 in Cobl because it introduces unsupported prices, assumes consent and changes the offer beyond its authority. Cobl improves coverage and follow-through in this run, but does not make those decisions reliable. The more extensive response also takes 126 active minutes versus 34 in Claude.ai.

RFP Response with Opus 5.5

Opus 5.5 scores 85/100 in Cobl and 68/100 in Claude.ai, a 17-point difference. The direct run produces a similarly structured response and handles several financial corrections. The larger gaps are in source discipline, explicit requirements and clean delivery. In Cobl, accuracy rises from 8.5/15 to 11.5/15, writing from 4.5/8 to 7.5/8, and final document quality from 1.5/5 to 4/5.

The harness does not resolve every problem. Both runs contain technical commitments that conflict with the financial proposal, so neither is ready to send as is. The Notion session table reports 27 minutes for both runs, with $11.32 in Cobl and an estimated $5 in Claude.ai. The attached document analysis does not independently measure those operational figures.

Across these 3 comparisons, Cobl scores 14 to 22 points higher. That is evidence that the working environment matters, while the individual criteria show where it still falls short. Each plotted result is 1 selected graded run, and the direct Opus assessment lacks conversation evidence. The gaps need repeated, fully matched testing before we can treat them as stable effects of the harness.

Quality against total response cost

The chart shows the total cost of each response, without dividing it by the number of pages produced. Among the Cobl runs, costs range from $0.38 for GPT Luna 5.6 to $68.09 for Astra. Opus 5.5 changes the trade-off: it scores 85/100 for $11.32, only 2 points below Astra at about 1/6 of the cost. GPT Sol 6 scores 80/100 for $21.66, while the retained GPT Sol 5.6 run scores 73/100 for $8.05. Quality remains central: the question is what it costs to reach the standard the user needs, including the review and completion work left after generation.

Why Astra costs more. The corrected graded run costs $68.09, about 8.5 times GPT Sol's $8.05. This is the cost of the plotted run, not a sum of multiple attempts.

A substantial workload at a higher total cost. Astra's corrected run used 21.9M prompt tokens and 118,000 completion tokens across 177 LLM calls. Sonnet used 18.2M input tokens and 144,000 output tokens for $3.99. Astra's total bill is about 17 times higher. Token volume alone does not explain that gap; pricing and caching also affect the bill.
Reading and checking are part of the workload. Astra repeatedly revisits the sources before and during drafting. That supports its strong context understanding, but also means processing context across many LLM calls. The 21.9M prompt tokens are cumulative input across the run, not the size of the knowledge base.
Writing more is not the main explanation. Fable generated 346,000 completion tokens, nearly 3 times Astra's 118,000, while its total recorded cost was $21.82 versus $68.09. A longer answer does not necessarily cost more.

The practical priorities are to preserve useful source checking, reduce repeated processing, keep the cache effective across turns and prevent avoidable restarts. Prompts alone do not change the model's price tier.

Quality against time to deliver

Among the Cobl runs, recorded delivery time now ranges from 27 minutes for Opus 5.5 to 126 minutes for Fable. Opus reaches 85/100 in that shorter window; GPT Sol 6 reaches 80/100 in 48 minutes. Astra retains the highest score, 87/100, with a 65-minute session window. Higher quality does not necessarily require a longer session. The direct runs record 67 minutes for Astra in ChatGPT, 34 for Fable in Claude.ai and 27 for Opus 5.5 in Claude.ai. Timing is not fully uniform: Astra uses a session window, earlier results retain active durations, and the new runs use the durations reported in Notion.

The workflow also matters. Waiting for unnecessary approval or failing to export a completed document can prevent a usable delivery, regardless of how well the model writes. We therefore look at speed alongside depth, completeness and successful export.

Conclusion

Astra 6 retains the highest quality score at 87/100. Opus 5.5 follows at 85/100, then GPT Sol 6 at 80/100. The Opus result changes the practical comparison: it comes close to Astra at much lower cost and a shorter recorded time, although it still makes commitments that are inconsistent across documents. Astra remains stronger on source discipline. Fable remains comprehensive, but its unsupported assumptions and unauthorised changes hold its score down.

Sol 5.6 has been our default because of its grounded reasoning and manageable cost. The new Opus 5.5 result gives us a strong reason to revisit that choice through repeated tests. A single selected run is not enough to establish that a new default will reliably perform better. The recurring failures, including incomplete exports and corrections not reaching every document, remain concrete targets for Cobl's skills, prompts and tools.

For a team prioritising speed, Opus 5.5 is now the strongest result in this selection: 85/100 in 27 recorded minutes. GPT Terra remains less expensive at $3.96, but its 65/100 response leaves more work to complete. Quality, cost and time need to be read together, and the remaining defects matter as much as the ranking.

These findings come from a specific RFP and selected runs across several test dates. We will rerun the evals every month as models, the harness and skills evolve, and extend them to other sales tasks. That lets us distinguish recurring behaviour from run-to-run variation and update our choices on evidence.

What this means for our users

Evals help us do more than choose a model. They show us where to improve the whole agent: how it reads sources, handles contradictions, respects the user's decisions and turns its analysis into finished documents. Each recurring failure becomes a concrete target for our skills, prompts and tools. The goal is to reduce the checking and rework our customers need to do, so they can spend more time winning deals.