Conflicting User Interview Findings: Build an Evidence Wall

One interview participant asks you to remember their last settings. Another asks for a fresh start. Before choosing which request to follow, check whether they are doing the same task under the same conditions. Conflicting user interview findings can reveal different needs, missing context, or a question your research has not yet answered.

An evidence wall puts those differences where the team can inspect them. Use Boardmix’s shared online whiteboard to place source notes beside competing explanations, then work out what additional observation would distinguish them. The aim is not to make every participant agree; it is to avoid a conclusion that the evidence cannot support.

When an evidence wall is more useful than another theme

An affinity map groups related observations. An evidence wall for conflicting findings zooms in on one disputed question: which pieces support each explanation, which challenge it, and what context is missing? Use it when a broad theme such as “users want more control” hides opposing requests, or when a stakeholder selects a memorable quote as the reason to ship a feature.

You do not need to move the entire study onto a new board. Pull together the relevant source notes and keep their original identifiers. The output is a bounded statement about what you currently know, plus a specific next check—not a popularity contest between two piles of sticky notes.

1. Write the disagreement without choosing a winner

Consider an illustrative study of a business reporting app. The product team is deciding whether report filters should persist. “Should we always remember filters?” already makes one implementation the default. A more useful question is: When does reusing report filters help people, and when could it carry the wrong context into their next task?

Put that question above the evidence. Under it, write the two statements you are comparing:

• Reusing previous filters helps people repeat routine reports.
• Starting with empty filters helps people avoid carrying settings into an unrelated report.

These statements can both be true. Keep proposed solutions—an automatic default, a saved preset, or a “reuse last filters” action—in a separate area. Otherwise, discussion about a preferred interface can replace discussion about the actual need.

2. Build source cards that preserve the conditions

Each card needs one quote or observation, a participant code, a source reference, and the conditions that matter to this question. Use a recording timestamp or a precise note reference that an authorised teammate can find. Do not put personal details or unrestricted recording links on a widely shared board.

Distinguish what was said from what was observed. The GOV.UK guide to analysing research sessions recommends recording what researchers saw or heard separately from what they think it means. For this wall, add interpretations beside the source cards rather than rewriting the evidence to fit them.

The example below contains six evidence items from three participants, not six independent participants. The reporting app is the subject of the study; its filter behaviour is not a Boardmix feature.

Keep the quote intact even if it sounds absolute. E03 says “each time,” but the explanation mentions another client. That difference is a reason to ask about context, not permission to silently replace the quote with your interpretation.

3. Compare conditions before calling something a contradiction

Place the relevant cards side by side and inspect the task, role, company or account, product state, and evidence type. A short annotation beside each pair is more useful than a broad label such as “mixed feedback.”

Different contexts: E01/E02 versus E03/E04

P01 repeats a report within one company. P02 moves between companies and worries about carrying a previous client’s settings forward. These observations do not establish that one person is right and the other is wrong. They suggest that the boundary between repeated work and changed context matters.

Record the interpretation as: “Reusing filters may help repeated work within a company; a company change may require a visible reset or an explicit choice.” Keep “may” until the proposed interaction has been investigated. Two accounts of a need do not prove which interface will meet it.

Missing conditions: E05 versus E06

A recollection of Monday’s behaviour and an observation in a new tab are not yet a controlled comparison. The earlier application version, tab state, and exact report are unknown. Label the pair “unresolved—earlier conditions missing,” not “the app randomly forgets filters.”

If two observations really do use the same account, report, version, starting state, and steps but produce different results, preserve both and investigate the inconsistency. Do not explain it away simply because most other cards support one story.

Same words, different meanings

Check what a participant means by “reset,” “saved,” or “the same report.” Someone may expect the supplier filter to persist but the date range to change. Ask them to identify the specific fields or walk through the last occurrence. A short follow-up can be more informative than inventing a new segment from an ambiguous sentence.

4. Arrange the evidence wall in Boardmix

Open a Boardmix workspace, start with a blank board, and add four frames: Question, Evidence pairs, Possible explanations, and Next check. These are labels you create on the canvas, not a dedicated research-analysis mode.

1. Put the decision boundary in the Question frame. Add the report-filter question and the two competing statements. Leave room for the conditions that would make each statement useful.
2. Create one sticky note per evidence item. Begin each note with E01–E06 and its participant code. Add a short quote or observation and the source reference; keep longer contextual details in text beside it.
3. Arrange comparison pairs manually. Put E01/E02 opposite E03/E04, with “same company” and “company changed” visibly labelled. Place E05/E06 below in a separate unresolved pair. Do not sort all cards into “for” and “against” before checking their conditions.
4. Connect evidence to explanations. Use connection lines and nearby text such as “supports,” “limits,” or “missing context.” Connect E01 and E02 to the repeated-work explanation; E03 and E04 to the context-change explanation. Leave E05/E06 attached to the missing-conditions question.
5. Add a small follow-up table. Use columns for the unanswered question, the observation needed, the person responsible, and the review point. These are manually maintained research notes, not automatic assignments or reminders.
6. Invite a source review. Check board access before sharing. Ask a teammate who attended the sessions to comment on whether the cards accurately represent what happened. Put disagreements about interpretation beside the relevant explanation, without changing the source quote.

Use words as well as colour: a blue card must not be the only indication that something is an observation. When the wall becomes crowded, keep the disputed pairs in the main view and place the longer source index beside them. The reader should be able to follow a claim back to its evidence without guessing what a colour or line means.

5. Choose a follow-up that can change the interpretation

“Do more interviews” leaves the disagreement untouched. Write down what you need to observe and how the possible results would affect your explanation.

For the first check, the example board assigns Maya, the researcher, to prepare the two task prompts before the next research session. For the second, Leo, an engineer, records the reproduction steps for review with Maya. The named owners keep the next action concrete; they do not make the current interpretation more certain.

Avoid asking “Would you like smarter filters?” That introduces a solution and makes agreement easy. Ask participants to show what they did, what they expected to stay, and what they checked before using the report.

What to take into the product discussion

The wall can now support a narrower statement:

In this study, repeated reporting within one company and switching between client companies created different expectations about filter reuse. Explore an explicit way to reuse filters within the current company, while making a company change visible. The cause of P03’s date-filter behaviour remains unresolved.

This is not a recommendation to persist every field, nor evidence that all accounts assistants or consultants behave alike. Keep E01–E06 attached to the statement, along with the two planned checks. If a design decision must happen before those checks, record which uncertainty the team is accepting and choose a change that can be revisited.

In the next review, open the disputed pair first. Add the new observation beside it and amend the interpretation only if that observation warrants it. A useful evidence wall leaves room for the result that changes your mind.