Agree counting rules before naming metrics

Metric names such as “participant numbers”, “satisfaction” and “follow-up discussions” are easy to agree in a meeting. Once the records arrive, however, people disagree over whether a person was counted twice, whether gate records represent people, or whether a company that has not replied should count as zero. Metrics must be defined by counting rules, not names.

Author: EVENTODIT Editorial Team
Published: 2026.10.31
Updated: 2026-10-31T11:07:15+09:00
Original URL: https://eventodit.com/en/stories/event-metric-definition-dry-run-records.html

Practical summary

Key decision: Agree the decision, result, unit, calculation, cutoff and treatment of unknowns, and check that two people count the same records consistently.

Responsible roles / intended readers: Event planning leads, research and operations staff, and commissioners

Record / next task: Approved metric definitions, evidence and cutoff rules, fictional counting results, and open issues with owners and deadlines

Basis of information: Direct review of CDC’s general evaluation-design and evidence-collection guidance, applied to event work by the editorial team

Editorial review: 2026-10-31

Start by stating the decision in one sentence

Before defining a metric, ask what decision the number will support. There must be an actual choice, such as whether to retain a business-meeting programme at the next event or increase opportunities for a particular group. Agree when the result is needed too. A number due the day after an event cannot inform that decision if it will be established only through a survey a month later.

Check whether readily available data, such as registration numbers, are standing in for the main objective. If the aim is follow-up discussions, count meetings held, willingness to continue discussions and an actual confirmed appointment separately. They are three different states.

Answer six questions for every metric

For each metric, the research lead answers the following questions and checks that planners and operations staff understand them in the same way. The result being checked must specify which confirmed state, and for whom, counts as one. Population and unit specify whether people, companies, meetings or sessions are being counted and who is included or excluded. The calculation states the numerator and denominator for a proportion, or the rule for identifying duplicates in a count. Evidence and cutoff identify the supporting records and the latest point by which information must be confirmed to be included. The decision rule states the basis for the target and what distinguishes achieved, not achieved and judgment deferred. Finally, name the people responsible for collection, review and final approval.

Leaving only a label such as “participants” allows several interpretations. This is how teams end up counting gate passages when they need people, or adding every meeting by the same company when they need companies. Before collecting more names or email addresses to reduce duplicates, check whether existing operational identifiers can distinguish them.

Keep previous values separate from targets

A previous event’s figure is a comparison baseline; the current target is a criterion the organiser uses to make a decision. Putting both in the same field obscures their basis. Alongside the previous figure, record whether its population, period and definition match the current one. If the definitions differ, show the difference rather than calculating a growth rate.

For a first event without comparison data, record “no baseline”. Do not present an industry average without a source as a benchmark. If only the target is agreed and conditions for deferring judgment are not, the treatment of information still unconfirmed at the cutoff remains unresolved. Missing information must not automatically become zero.

Check first that the evidence actually exists

Check that registration forms, on-site records and follow-up procedures actually contain the fields required. Agree who enters the information and when, who reviews duplicates and gaps, and where corrections will be reflected. Work backwards from the date the result is needed to allow time for collection and review.

If a contractor or existing system cannot provide the required evidence, check the contracted scope and access rights first. Approving unavailable information as collectable causes the counting process to fail later. The purpose, notices, retention and deletion conditions for personal information must be reviewed separately against the requirements applicable to the event. This article does not replace that process.

Test duplicates and unknowns with fictional records

Enter fictional records before real collection begins. The following small business-meeting event was created for explanation; it is neither an actual result nor a recommended benchmark. The population consists of three companies, A, B and C, whose participation has been confirmed, and the unit is a company. The cutoff is 6 pm Korea time seven days after the event. Count companies whose follow-up discussion date and time have been confirmed by then. The illustrative target is at least two companies.

| Test record | Input | Treatment |
| --- | --- | --- |
| First record for A | Follow-up date and time confirmed | One confirmed company |
| Second record for A | The same confirmation entered again | Duplicate; do not add another company |
| Record for B | Response status not confirmed by the deadline | One unknown company |
| Record for C | Confirmed that no follow-up appointment is arranged | One company without a confirmed appointment |

There are four input rows but three companies. Do not add A’s duplicate and declare the target achieved. It is also wrong to treat B as zero and finalise the result as below target. Under these assumptions, defer judgment and record who will check B and by when. Do not add appointments newly arranged after the cutoff to this result.

Have the planner and the person responsible for counting calculate the same records independently, then compare their results. If the results differ, check the unit, duplicate handling and state definitions before changing the source records. After changing a rule, check again that both people reach the same result.

Begin collection after approval

The organiser or designated approver checks the metric definition, basis for the target, availability of evidence, fictional calculation results and unresolved issues. Do not mark the plan final if critical fields remain undefined or the two people’s results differ. Record who will resolve the issue and by when, then review it again.

After approval, tell recruitment, on-site operations and research staff who is included and excluded, the fields to record, cutoff, treatment of unknowns and contact person. If a definition changes after collection begins, record the date and reason and notify the people affected. If data gathered under the previous rule are recalculated under the new one, record what was recalculated.

Practitioner checklist

- Is the decision this number will support recorded, together with when the result is needed?
- Does every metric specify the result, unit, calculation, cutoff, decision rule and owner?
- Are the bases for the previous figure and current target recorded separately?
- Has it been confirmed that the evidence can actually be obtained and who will enter it and when?
- Have two people counted fictional records and reached the same result?
- Are the treatment of unknowns and conditions for deferring judgment defined?

Use the form for the next handover

Use the [Objectives, metric definitions and fictional counting tests](/en/templates/metric-definition-dry-run.html) for the records described here. The XLSX contains Blank, Example and Instructions sheets. Write your event in Blank; the fictional example is a guide to what to enter.

Record the decision supported and the objective, population and action or verified state. Define unit, numerator/denominator, duplicate key, exclusions, unknown treatment, source and cutoff.

Planning sends the approved definition, fields, cutoff and unknown treatment to recruitment, on-site and research teams. For changes, record the date, reason and recalculated scope and distribute the same version.

Conclusion

Metric definitions must be completed before collection. This reduces the chance of people counting the same records differently on site. Fictional records are the cheapest way to check whether a definition works in practice. Comparison and interpretation after actual evidence has accumulated can be addressed in a separate review stage.

*Reference: This is the editorial team’s application of the US CDC’s general programme-evaluation guidance, Step 3 and Step 4, to event work. Only the OECD event-impact publication’s official introduction and abstract were reviewed; this article’s procedure is not presented as an OECD requirement. The companies, times and target are illustrative, not industry averages or research findings.*

Application conditions

Use after event purpose and audiences are agreed and before registration, operations and survey fields are finalised. Confirm targets and access to the required evidence.

Unconfirmed information

The three companies, seven-day cutoff and two-company target are fictional examples, not benchmarks. Causal effects and the legality of personal-data handling were not verified. Only OECD’s official introduction and abstract were reviewed.

Sources

US CDC · Step 3: Focus the Evaluation Questions and Design, 18 August 2024
https://www.cdc.gov/evaluation/php/evaluation-framework-action-guide/step-3-focus-the-evaluation-questions-and-design.html

US CDC · Step 4: Gather Credible Evidence, 18 August 2024
https://www.cdc.gov/evaluation/php/evaluation-framework-action-guide/step-4-gather-credible-evidence.html

OECD · How to measure the impact of culture, sports and business events, Part I, 6 July 2023: official introduction and abstract reviewed
https://www.oecd.org/en/publications/2023/07/how-to-measure-the-impact-of-culture-sports-and-business-events_c8768aa2.html