Let’s be honest: AI has a processing power and speed that no human researcher can compete with. As a sparring partner in the initial phase and as an assistant during the analysis, the technology is worth its weight in gold.

During Sjoerd and Annelies' experiment, AI particularly excelled in the following areas:

• Breadth and Completeness: Within seconds, AI maps out a massive number of potential variables, influencing factors, and side paths. It forces you to look broader than your own initial thoughts or tunnel vision.

• The Perfect Preparation: Need quick background knowledge on a specific niche? AI helps you get up to speed fast, formulate hypotheses, and draft sharp, targeted interview questions.

• Analysis and Synthesis: Once the data is in, AI is fantastic at clustering, structuring, and providing clear recommendations based on the first raw insights.

In short: AI opens the funnel wide and brings structure to the chaos.

If you rely purely on AI output, you miss the essence of the human being. AI reasons fundamentally rationally and systematically. The computer assumes that a customer journey or a decision-making process is logical and linear.

But the reality? Human behavior is rarely logical.

People make choices based on emotion, intuition, gut feeling, and shifting context. We are guided by doubt, time pressure, a faltering attention span, or subtle aesthetic triggers like the 'vibe' of a brand. A human researcher catches these nuances in the real decision-making process flawlessly, because we can truly empathize with mental models and the customer's world.

Furthermore, AI has a major problem with prioritization. The models spit out a laundry list of potential factors or problems, but they are poorly equipped to weigh what is truly decisive for the end user in a specific, unique situation.

Our own practical experiences align seamlessly with independent scientific research from the renowned UX institute MeasuringU. They conducted a fascinating test comparing human UX researchers and advanced LLMs (such as ChatGPT and Gemini) analyzing the exact same video of a user test.

Those results confirm exactly what Sjoerd and Annelies saw in their experiment:

• Low mutual agreement: The overlap between what human researchers observed and what AI reported was surprisingly low (just 19% to 36% depending on the model). AI picks up fundamentally different signals than a human.

• Half is missed: AI managed to identify only about 44% to 55% of the real usability issues verified by humans.

• The 'Validation Burden' (AI slop): The AI models collectively generated more unique 'new' problems than the human researchers. Sounds great, but many of these points turned out to be noise, irrelevant details, or potential hallucinations. A human expert is always needed to filter all that AI output. If you have to spend that time manually validating noise, your time savings melt away like snow in the sun.

This experiment does not stand alone. It fits perfectly into our broader vision, which we previously shared in our article on [AI in digital strategy]. Too often, we see organizations deploy AI as a standalone innovation project on the sidelines. A tool is purchased or a pilot is started, but structural impact fails to materialize because the connection to business objectives and UX architecture is missing.

AI only becomes truly relevant when it answers strategic questions: Where do frictions arise in the customer journey? How do we mitigate risk in strategic choices?

Just as we use AI in our client cases to uncover patterns faster in vast amounts of qualitative data, we also apply it to user research. The goal is never to automate conclusions or replace the UX specialist. The goal is to increase data quality, reduce blind spots, and shorten the feedback loop. This shifts digital optimization from reactive hindsight to proactive steering.

At Concept7, we don’t believe in blindly replacing traditional research with AI. Anyone who does so builds products and strategies based on a clinical, rational illusion.

The real magic and the highest value for our clients' digital course, lies in the combination:

[ AI accelerates & broadens ] + [ Humans give meaning & nuance ] = The ultimate research synergy

We deliberately use AI as a powerful, broadening partner in the preparation and analysis phases. This allows us to prioritize more sharply and efficiently digest large volumes of data. But we keep the reins firmly in human hands. Because to understand the emotion behind the click and determine the right strategic direction, you need expertise, empathy, and human intuition.

AI gives us a broader horizon. We ensure we understand the context.

Curious about the role AI can play in your digital course? We would love to explore together where AI truly adds value, accelerates processes, or provides deeper strategic insight.