The Number After the Number: A Governance Check on Survey Weighting

When a UK survey result reaches the board, it often arrives as a single, confident figure: 62% of customers agree, 41% of buyers expect to increase spend, or 28% of SMEs have adopted a particular tool. That figure may well be robust. But it is rarely the raw count. Between the questionnaire and the headline sit a series of adjustments — quota targets, rim weights, propensity models and occasional post-stratification — that can move a number as materially as fieldwork execution. Weighting is a legitimate and necessary part of survey research. The governance question is whether the board can see what was done, why, and what difference it made.

Why weighting belongs in the board pack

Most survey samples are not miniature versions of the population they seek to represent. Younger people answer online panels more readily; certain regions respond at different rates; small firms are easier to reach than large ones; and some job titles are more willing to complete a B2B interview than others. Weighting corrects for known imbalances by giving more or less influence to particular respondents. Done well, it improves representativeness. Done opaquely, it can manufacture precision that the underlying sample does not support.

For decision-makers, the practical issue is not whether weighting exists. It is whether the adjustment has been documented, benchmarked and sensitivity-tested. A weighted result should be treated as an estimate with a method attached, not an incontrovertible fact. If the unweighted and weighted figures diverge sharply, that gap is itself a finding: it tells the board something about who responded and how hard the data had to work to become representative.

The three adjustment layers to examine

Quota design. Quotas set the shape of the achieved sample. Interlocked quotas — for example, age within gender within region — are more demanding than separate marginal quotas but reduce the residual skew left for weighting to fix. The board does not need the full quota grid, but it should know which variables were controlled, whether targets were met, and where shortfalls occurred.

Rim or raking weights. Rim weighting, also called raking or iterative proportional fitting, adjusts the sample to match known marginal totals. In UK consumer studies, those totals may come from the latest ONS census-based estimates, the Labour Force Survey or another authoritative source. The questions to ask are straightforward: which variables were used, what benchmarks were applied, how old are those benchmarks, and how many iterations were required? Weighting to many variables can improve marginal fit while distorting relationships between them. It can also produce highly variable weights, reducing the effective sample size.

Propensity weighting. Some online and B2B studies use propensity models to correct for differences in the likelihood of responding. This can be useful, particularly where no clean population frame exists. But the model is only as good as its inputs and validation. If a supplier cannot explain the target variable, the covariates and the diagnostic checks, the board should treat the adjustment as an assertion rather than evidence.

Where UK studies commonly come unstuck

One recurring problem is benchmark drift. A study weighted to an ageing profile of the population, or to business demographics that no longer reflect the current UK economy, may be technically weighted but substantively misaligned. Another is over-raking: adding variables until the margins look neat, at the cost of a small effective base and unstable subgroup results. A third is silence about weight variation. If the weight range is wide and the design effect is large, the published base overstates the precision of the result.

B2B studies present a further challenge. There is often no authoritative frame for the buying committee, so weighting may be limited to firm size, sector or region — useful correctives, but not a guarantee that the sample reflects decision-making authority. In these cases, the board should ask for the unweighted profile alongside the weighted one and consider whether the conclusions depend on a small number of heavily weighted respondents.

A one-page audit trail

A short weighting note can sit beside the methodology slide. It need not be technical, but it should be consistent. A workable template covers:

• study name, fieldwork dates and mode;
• unweighted and weighted bases;
• quota variables, targets and achieved counts;
• weighting variables and benchmark sources, including reference dates;
• weighting method and any trimming or capping rules;
• minimum, maximum and 95th percentile weight;
• estimated design effect and effective base;
• the largest movement between unweighted and weighted headline figures;
• the name of the person who approved the weighting approach.

This is not bureaucracy for its own sake. It creates a record that can be revisited when a result is challenged, reused in a future wave, or compared with another supplier’s study.

Questions for the research supplier

Before a weighted figure reaches the board, ask: Which variables were weighted, and why those? What benchmark data was used, and when was it published? What was the weight range, and what effect did weighting have on the headline? How many respondents sit in the smallest reported subgroup? Has the result been checked against an unweighted or alternative specification? What would change if the weighting were removed? And who signed off the approach?

Warning signs

Be cautious when the method note says only that data were “weighted to be representative”; when the weight range is undisclosed; when subgroup bases are small but presented without qualification; when the benchmark source is unspecified; or when a supplier cannot show the unweighted result. None of these automatically invalidates a study, but together they weaken the audit trail.

Weighting is a judgement embedded in the number. For UK boards making decisions in 2026, the sensible posture is neither suspicion nor blind acceptance. It is to ask for the adjustment record, inspect the sensitivity, and treat the gap between raw and weighted results as part of the evidence. The headline is the number after the number — and that second number deserves a place in the room.