The usual complaints are context and scale. The model doesn’t know enough about the study. The dataset is too large for a prompt. Both are real, and both are solvable — context can be added through supporting documents, and data can be handled in chunks.
The harder problem is the shape of the data itself. There are two versions of this problem, one at each end of the spectrum.
Raw respondent files don’t tell the model what it’s looking at.
A variable labelled q4_1 could be any of the following:
An experienced analyst would open the questionnaire and check. An LLM won’t. It produces a plausible table and gives no indication of which interpretation it used. Bases get applied to the wrong universe. Multi-response questions get collapsed into single-response distributions. Ranked items get treated as categorical.
Crosstabs look safer because they already resemble tables.
But a crosstab’s meaning is carried by its layout. Merged headers get flattened into confusion. Base sizes sit in footers and disappear. The value in any cell depends on which column sits above it and which question sits to its left — and the model doesn’t reliably track either.
Quantitative work needs three properties from any workflow:
→ Verifiability — any number can be checked against its source → Validity — the computation is correct → Consistency — same input, same output
Pasting raw data or a crosstab into an LLM gives you none of them. This isn’t a general problem with LLMs — they handle summarisation and qualitative coding well. It’s specific to aggregate quantitative work, which is exactly where most teams are most eager to apply them.
The order matters. Plan and draft first, run the code second, hold the judgement at the end.
When this starts going wrong, the instinct is to pick one tool. Either hand the whole problem to the LLM and hope better prompting solves it, or work only in Python.
Both fail. The LLM on its own is inconsistent from run to run. Python on its own needs new code every time a crosstab template changes.
The better approach treats the two as complementary.
The LLM interprets. Python computes.
The LLM reads the file’s structure, drafts the transformation code, and helps articulate findings in plain language. Python runs the code, produces the numbers, and holds the data in a form that can be inspected.
The order matters: the LLM plans and drafts → Python executes → the analyst verifies.
Commercial tools are starting to package this pattern into platforms. Whether you build or buy, understand the mechanics first — it’s easier to evaluate a product when you know what it should be doing underneath.
Flat data is not the whole solution, but it is the shape that makes the rest of the workflow possible.
In the context of quantitative survey analysis, “flat” has a specific meaning:
Every row describes itself. There are no merged cells, no indented sub-rows, and no layout the reader has to interpret.
The contrast is easiest to see with an example.
Before — a typical crosstab fragment:
Total Male Female 18-24 25-34
Awareness of Brand
Top of mind 12 14 10 8 13
Unaided 34 36 32 28 35
After — flattened:
question response banner_group banner_value value
Awareness of Brand Top of mind Total Total 12
Awareness of Brand Top of mind Gender Male 14
Awareness of Brand Top of mind Gender Female 10
Awareness of Brand Top of mind Age 18-24 8
Awareness of Brand Top of mind Age 25-34 13
Awareness of Brand Unaided Total Total 34
The flattened version is longer, but every row is unambiguous — both to the LLM reading it and to Python computing on it. Everything downstream becomes more straightforward.
Three stages, in order. The LLM supports each stage but does not drive any of them.
Pick the path that matches the input. Crosstab in hand → A. Multivariate work → B. Need the tables yourself → C.
Preparation is where the craft lives, and it splits into three paths depending on what you’re starting with and what the analysis needs.
Path A → Flattened crosstabs.
This is the default for most quantitative work, assuming you are working with crosstabs to begin with. The crosstab arrives from the agency or the DP team. You convert it into the flat format shown above — either by running a script yourself or by specifying the format at source. From that point on, the flat file is the single source for the study: the LLM reads it, Python computes on it, and the analyst can trace any number back to a cell.
If you are working from raw respondent data instead of a crosstab, skip to Path C before coming back to this stage.
Path B → Block extraction from raw data.
This path is reserved for analyses that a crosstab cannot carry — driver analysis, regression, segmentation, clustering, factor analysis, and CFA. Anything multivariate. Anything that needs respondent-level correlations. Anything that builds a derived measure.
The important move here is not to load the full dataset. You extract the specific block of questions the analysis requires, at respondent level, with labels and codes intact. A minimum viable dataset, shaped precisely for the question being asked.
Path C → Cross-tab extraction from raw data.
This path applies when the only thing you receive is raw respondent data and you need to produce the standard cross-cut tables yourself. For any study of real size, this is the most challenging preparation work — and the one most prone to silent error.
It requires rigour at every step:
The principle across all three paths: shape the data for the question, not the question for the data.
Validation lives inside the prep code as automated checks. The specific checks vary by study, but common examples include:
If a check fails, the prep is wrong and the analysis does not start. Which checks matter for which study is a judgement that stays with humans.
Once the input is clean and validated, the LLM becomes genuinely useful in three modes:
The analyst sets the question and the frame. The LLM works inside it. Every number traces back to a cell, and every step can be re-run without producing different results the second time.
Four artefacts make this framework runnable end to end. They are designed to compose: prep upstream, analysis downstream, with the flat format as the contract between them. Each links through to its full detail page.
Converts agency crosstab deliverables (XLSX, XLS, CSV) into the flat format described above. The skill inspects the file, confirms the structure with you, then runs Python to flatten deterministically. Outputs long and wide flat files, a data dictionary, and a four-check validation report.
Use this when a crosstab has already arrived and you need to get it into a shape an LLM can read without losing meaning. Pairs with Path A.
A version-controlled markdown specification any data processing team can work from. It defines the flat output shape semantically — required columns, row-type taxonomy, value conventions, encoding — while leaving naming and internal workflow flexible. Includes worked examples and an FAQ covering common edge cases. Platform-agnostic and independently shareable.
Use this when you want the flat format produced at source, before the file ever reaches your hands. Pairs with Path A.
Generates flat-format crosstabs directly from raw respondent-level data (.sav, .csv, .xlsx). Supports all five question types, analyst-driven weighting, conditional bases, custom NETs, and optional significance testing. Outputs the flat file plus a formatted crosstab Excel — chainable into flatten-crosstab if you need both forms.
Use this when you only have raw data and need to produce the standard cross-cut tables yourself. Pairs with Path C.
Picks up where the prep stack ends. Works through research objectives interactively — proposes analytical moves, runs them deterministically, and helps you pin each finding to its supporting evidence. Auto-runs four sanity checks per finding. Exports findings together with the tables behind them, and sessions resume across sittings.
Use this when the data is clean and you need help going from a flat file to a defensible set of findings.
Some teams will prefer the flattening skill for speed and control. Others will prefer the DP spec for scale and repeatability. Teams working from raw data will need the extraction skill regardless — it’s the most demanding of the four and the one where rigour matters most. The analysis skill works on top of any of them, once the data is in shape.
All four artefacts, along with the supporting documentation and code, are available from the claude-skills directory of this site.