How AI Will Transform User Research

But Not In The Way You’re Thinking

AI will eventually commoditize user research. I don’t know when, but I have a decent intuition of how it would happen.

First, let me explain that when I say “commoditize,” I’m not saying that AI will suddenly and abruptly replace researchers.

As a matter of fact, that won’t happen to any knowledge workers anytime soon because there’s a big difference between the current flavor of generative AI and a true AGI.

Without real AGI, most AI innovation would happen as an effect of placing AI functions and agents in specific parts of a given stack. That will create giant multipliers, but it should be evident that those multipliers won’t be enough to extinguish an entire job family flat-out.

What I mean by commoditizing is that AI will enable the creation of tools and systems that can automate user research. This will reduce the time and effort required to establish research channels, extract qualitative data, analyze the data, and synthesize findings.

Current models are already good enough at generating research protocols and analyzing unstructured data. However, the challenge remains in automating specific workflow motions and strategically exposing AI to a firehose of customer feedback and data.

This would likely be accomplished once someone can create a system that can stack a few sub-systems in a clever way.

First, someone needs to figure out an effective way to aggregate a given company's existing institutional consumer and product knowledge, and embed it into a large language model.

Once that is accomplished, it would be a matter of creating automated workflows that can automatically generate research protocols, and agentic functions that can execute those protocols by using existing customer channels such as email, texting, or even machine-to-human recorded calls.

Other agentic functions in the stack can respond to data-gathering completion hooks and compile all the gathered insights into multiple analyses that can be more granularly indexed into the existing knowledge systems.

The research can evolve as the AI gathers more insights, so in that sense, AI would allow customer insights to become more akin to analyzing quantitive data.

Insights drip into the system’s context, as raw qualitative data is gathered in an automated fashion by AI research agents that follow the instructions of the high-level protocol.

As research evolves into becoming more programmatic, patterns of just-in-time feedback will emerge and become more common.

For example, capturing sentiment and other signals when the user is interacting with the product as a way of enriching the context available to AI components in the system. This is effectively how OpenAI does reinforcement learning on their models (thumbs up or thumbs down for a response).

In that world, I see a user research landscape that transforms researchers from tactical operators to AI supervisors. These systems will require a human in the loop that knows what type of adjustments need to be made to correct failures or enhance the system.

Ultimately this changed landscape creates an interesting situation: Researchers will either have to specialize in being good AI supervisors in a specialist market or compete in the commoditized low-end by offering prompt tune-ups or building scripts to be fed as research protocols into AI systems.

When and how fast would this happen? My intuition is that it will be slow, and it will happen through the emergence of specialized AI toolings, such as automated feedback collection and analysis systems.

But I’m sure that even in the current primitive state of generative AI, a motivated group of individuals could build a proof of concept that can:

1) Receive a product hypothesis.
2) In an agentic fashion, transform that hypothesis into research requirements, analyze existing customer data, produce a protocol, and dispatch an incentivized campaign to gather customer insights.
3) Analyze results, index results into a knowledge base, and produce a final report.

With such a system, any non-sophisticated company suddenly has access to the research capabilities that are often reserved for sophisticated companies that have the resources to employ or contract researchers.

And this is what I mean by commoditized user research: abundant and relatively low-cost (or at least cheaper than a full-time researcher) AI tools that can be tuned to analyze customer feedback, launch data-gathering campaigns, and build a robust context of user behavior that other human employees or other AIs can consume.

Wild but totally reasonable in the presence of an exploding offering of AI systems and solutions.