Usability Test Reports: Separate Observation, Interpretation and Action

A useful issue report separates three kinds of statement

In a usability test, “the participant opened Billing while looking for an export” is an observation. “They expected export controls beside account settings” is an interpretation to check. “Move the export button” is a proposed design change. Keeping these statements distinct makes an issue report easier to assess and gives the next test a clear purpose.

The example below is hypothetical. It illustrates a reporting method, not a recorded ScribeFlash study or evidence that a particular redesign will improve task completion.

1. Agree the task and capture the session outside ScribeFlash

Define what the participant is trying to accomplish, the starting state, the product version, relevant device or assistive technology, and what counts as completing the task. “Prepare a report that you can send to a colleague” leaves the route open; “click Export” gives away the interface step you may need to examine.

The GOV.UK moderated usability testing guide recommends believable tasks that do not reveal the answer. Use that principle when writing the session guide and record any moderator assistance.

Obtain the appropriate permission for recording and handle personal information according to the research arrangement. Record with your own research or meeting tools, then upload an audio or video file using the interview upload workflow. ScribeFlash does not record the test screen or automatically observe mouse movements. A transcript captures spoken material; silent interface actions need researcher observation and the external recording as their source.

2. Write an evidence note before naming the problem

For an invented task T1, fictional participant U1 opens Billing, returns to the project and asks, “Where do I get a file I can send?” at 04:18. The moderator then points to Export.

Keep three fields:

• Observation: U1 opened Billing, returned, asked that question and received a hint. Attribute the navigation to the observer or recording; attribute the quotation to the transcript.
• Interpretation: the location or wording of the export control may not match U1's expectations. A permission issue or unfamiliar project state may be another explanation.
• Recommendation: test a clearer entry point or label. This remains a proposal until you examine alternatives.

Do not infer frustration, confusion or a mouse path from a sentence alone. If the external recording is unavailable, state which action you could not verify. The research analysis guidance offers a useful distinction between observations, findings and actions.

3. Give each issue a reproducible record

A handoff that a designer or engineer can assess includes:

1. Issue title: a specific obstacle, such as “U1 could not locate report export without help.”
2. Task and environment: T1, starting page, prototype or build, device and relevant account conditions.
3. Observed sequence: the steps actually seen, including moderator interventions.
4. Source references: participant code, recording timestamp, exact spoken quote and a separately attributed observation note.
5. Effect on the task: whether the participant stopped, completed with help or found a workaround.
6. Interpretation and alternatives: what may explain the sequence and what remains unknown.
7. Proposed follow-up: the owner, change or question to test, and evidence needed to revisit the issue.

Save spoken passages as evidence, with the observation context in the researcher note. The project research questions can connect separate moments to the same task without making every quotation a separate finding.

4. Prioritize task impact without inventing precision

Agree the meaning of your priority labels. A blocker without a workaround, repeated recovery work and a minor wording hesitation call for different conversations. Consider whether a moderator intervention concealed a failure, whether the issue blocks an essential task and whether a workaround is available.

Report what your sessions show. “U1 needed a hint” is more precise than “users cannot export.” If you report occurrences, include which participants attempted the same task and under what conditions. Small, selectively recruited studies do not establish a population failure rate.

Preserve the successful case as well. If fictional U2 finds Export immediately, compare the starting page, previous experience and visible permissions before treating U2 as irrelevant. The contrast may explain when the obstacle appears.

5. Assemble an issue report that supports the next test

Start with the decision being considered and the tested task. Then present the observed obstacle, sources, task effect, alternative explanations and a proposed next check. Include successful attempts and unresolved questions alongside problems.

For the hypothetical export issue, a next check could compare the existing control with a clearer label while keeping the task neutral. Define in advance what you will observe: independent discovery, help requested, the chosen route and whether the resulting file meets the handoff need. Do not present those future observations as an improvement already delivered.

In ScribeFlash, review findings against their evidence before confirmation. The report builder turns confirmed findings into an editable report. Human review still needs to distinguish spoken evidence, observer notes and proposed actions; a polished report cannot supply an observation that was never recorded.

Use a template, then inspect a complete example

Choose the usability test option in the research templates to start with task-focused questions. The sample project and downloads show the evidence-to-report structure with explicitly simulated data. For broader comparison across sessions, use the interview analysis workflow.

Before sharing

Check that every issue identifies a task, sources, assistance received and an observed consequence; every interpretation has a boundary; and every recommendation is labelled as a proposal. Remove unnecessary identifying details from quotations. Make sure the recipient can tell what happened, what you think it means and what you want to test next.