Are open-source models good enough for analytics?
The test was never whether they can write SQL - it's whether they can reason through the analytics loop, and the good open ones now can.
Your default for real analytics work is a frontier model. You point Claude Opus or GPT5.x at a question, it writes the SQL, you move on. But every few weeks someone posts that some open model is now just as good, and you wonder whether that’s true - or whether the gap that matters is still there the moment you point it at your actual data, the messy tables nobody remembers the columns for.
For some of that data the question isn’t even optional. In healthcare, legal, or finance you legally can’t hand certain records to a frontier lab’s model, so “just use Claude” was never on the table for the work that matters most.
Not long ago that skepticism was fair - the open models really were a notch back, fine for a demo but not for work you’d put your name on.
That’s not what I found this year. We were running open models side by side with a frontier one on real analytics work, and they held up on the messy questions.
So I went back to what the job even is. In agentic analytics, you point a model at a question and it runs the whole loop - frames the question, explores the data, does the analysis, checks its own work, and tells you the story. The SQL is one step in the middle, and every model on this list can already do it well.
The hard steps are the human ones. Somebody asks “how has retention changed?” - that could mean cohort retention, rolling retention, or churn rate, and a good analyst asks which before touching the data. I’ve watched a model hand back rolling retention when the client meant cohort - a clean answer to a question nobody asked. “Checkout conversion” has five defensible readings that land anywhere from 3% to 90% depending on what you count. Framing the fuzzy question, picking the right definition among several honest ones, reading the numbers back into a recommendation - that’s the reasoning the whole thing rests on. So the real question isn’t whether an open model can code. It’s whether it can reason through that loop and follow the rules for doing analytics well.
With that as the bar, a quick tour of the field.
Qwen, from Alibaba, is the one everyone reaches for - Nathan Lambert’s read is that it’s overtaken Llama as the most-downloaded, most-fine-tuned open base, the default all-rounder, and it’s Apache 2.0, so you can build on it without a lawyer in the room.
DeepSeek is the reasoning-forward lab; its R1 release in early 2025 shook everyone’s assumption about how far ahead the frontier really was, and it’s MIT and cheap.
GLM, from z.ai, is the agentic, tool-native family that drops into Claude Code as a hosted substitute.
Kimi, from Moonshot AI, is the long-context, trillion-parameter one built for long multi-step runs.
Gemma, from Google, is the small, efficient one, now Apache 2.0.
And MiniMax is the cheap, efficient-MoE challenger.
Here’s what surprised me. The good open models have already cleared the steps of the agentic analytics loop - framing, exploring, analyzing, checking, telling the story. They can all reason through that much well enough now. The step that still breaks is context, and that one was never a model problem. The model guesses at “retention” instead of asking, settles on one reading of “conversion,” and hands you a clean answer to a question you didn’t ask - and no bigger model fixes that. It’s the analyst’s job and a context-engineering job: write the definition down once, and the guessing stops. So which model you run matters less than whether you scoped the question.
The numbers back this up. Anthropic gave an agent grep access to thousands of prior SQL files and accuracy moved less than one percent. In a separate study, handing the model the database schema moved accuracy from 53.9 to 93.3 percent. Context is the forty-point lever. A bigger model barely moves the needle next to it.
I wanted to see this on my own data, not in a paper. About a month ago I ran the open models through a benchmark - the messy, real-shaped analytical questions we get. What it shows is narrow and enough: they clear the good-enough bar, so the model stops being the thing you worry about.
None of this means open has caught up across the board. The more times a job has to run the loop, the more the reliability tax compounds - a step that’s right 90 percent of the time is right about 59 percent over five steps, and the frontier labs still edge ahead on holding a long plan together. But that gap is closing very quickly to a point where it’s indistinguishable depending on the complexity of the analysis.
If you’d like to learn more about what open source models are, how they stack up on doing analysis, and how to run analysis through them yourself, join us in this free workshop: https://maven.com/p/568b2d/run-your-data-analysis-on-open-source-ai-models