[Thoughts #95] User Research in Web3
Bringing Qualitative and Quantitative from 🤝 to 🫂
As I’ve spent time bouncing around Twitter and LinkedIn, I find it amusing that everyone is an ‘investor’ nowadays. So much so that it’s turning into a pet peeve of mine. I gathered that energy and turn it into a meme.
I also turned the same meme into a LinkedIn post.
Based on engagement, Twitter seems to have a better sense of humor. Makes sense.
Pretty sure my career is going down the drain after this.
PS: I am a hypocrite. Even though I’m actually investing beyond just buying NFTs, I’ll probably delete that word from my profile next time I update my LinkedIn. It needs a makeover anyway.
PPS: I came across this app earlier today
This is a cool concept but why do I feel like this app is going to turbocharge more people getting rekt?
Ya let’s trust social media with where we should invest our money. Like the last time your favorite influencer dumped NFTs on you, right? Senators though? There’s something there 😏
I can tell that today’s piece is going to be meme heavy. I feel the meme energy coursing through my veins.
That said, I promise — We’ll have some takeaways at the end of this meme-rific journey. 🙃
Anyway, we’re not here to talk about investing and hypocrisy. We’re here to talk about…
USER RESEARCH
This is one of those moments where the standup comedian goes, “Anyone from (insert obscure state or city)?” and a few people in the back whoop and holler while everyone else does this:
That’s right folks. Today’s piece is on User Research x Web3. Not the next hot NFT project, the next hot Web3 company, no sexy growth framework.
Also, to be clear: I’m not a User Researcher, Product Leader, or 3x YC Combinator grad. I’m a investor halfway decent growth guy that enjoys connecting dots and writing. 😉
So none of this please lol
This topic was a random thought that popped up into my head yesterday night, and now I’m writing this thought exercise out loud so…here we are 🤷♂️
And if you’re an expert in the space, let’s talk! Curious to see if I’m on or off point and how user research could benefit from Web3. Maybe there’s already been a lot of… research done on this topic already!
Let’s get some help from Google to define what user research is. I like this one:
Let me simplify it even more.
User research is knowing who your user is
Some simple implementations:
Talking to 10 of your customers
How Did You Hear About Us (HDYHAU) surveys during onboarding
Creating customer personas. For fintech as an example: The Spender, The Saver, The INVESTOOOR (sorry I had to 😂)
Some complex implementations:
Diary studies - Participants keep a log of their thoughts, experiences and activities over a longer period of time (days, weeks, months)
Triangulation - Combining multiple user research methods together
Quant usability testing - Surveying a larger sample set for statistical significance and ensuring that session setup is as consistent as possible
Good user research helps companies to receive feedback, determine their roadmap, and market to their customer more effectively.
Bad user research leads to convoluted conclusions, wasted time/resources, and counterproductive business decisions.
Did this Google design decision stem from good, bad, or no user research? I’ll let you be the judge of that.
So where does Web3 fit into all this?
This is where it gets fun.
From my personal experience in Growth and Marketing roles that focused more on the quantitative side of things. I struggled with the concept of conducting 20 user interviews and from that create 3 personas that defined the company’s marketing strategy for the year.
On the flip side, even though I leaned quant, I also struggled with the fact that you could pull trends from product analytics and make the decision that a particular step in the onboarding funnel should be the main focus of improving conversion rate for the year.
I distilled this discomfort into a simple question:
How do we know they walk the talk?
Conceptually, the answer is simple, but sometimes it’s harder than it sounds. Let’s say you run a user research study on the color of the buttons on your company page and you sell memes.
Most users say they like the blue button. Awesome! Change the button to blue.
What happens if you make the changes, and there are no improvements? It might be because of a different factor, lower quality users coming in, or macro changes (eg: memes aren’t cool anymore, your meme business is dead no matter what color the button is).
Or what happens if you make the changes and there are improvements? Is it because of that change or because of something else (eg: memes are even cooler than before, and your meme business goes parabolic no matter what color the button is).
Now the big companies of the world may have this all figured out, but I imagine most don’t.
So how can the 🤝 between Qual and Quant turn into 🫂?
Use wallet activity to complement your Qual + Quant user research
Honestly, I’ll be surprised if the user research behemoths like Nielson, Qualtrics, Kantar, Gartner, or IPSOS aren’t already thinking about this.
Blockchain data is public
This is additive to existing methodologies, it doesn’t disrupt them
It strengthens the rigor of user research practices
It can make studies have longer lasting impact
So what are some possible implementations of this?
New users
Build customer personas of new users who connect their wallet based on their historical data. Not just signals in the short amount of time that they enter your ecosystem.
Identify familiarity with a particular set of products based on the wallet’s previous activity with them. You can personalize content or products based on that wallet’s ‘skill or familiarity level’.
Note that this won’t be helpful if the activity is coming from a fresh wallet with no historical activity
Competitive analysis
Are your sales down? Wallet analytics can help tease additional insights to understand why that might be. Maybe users aren’t purchasing your product anymore, but they’re buying a competitor’s. You can map why that might be the case and timebox it to a competitor promo. Or maybe they’ve left the ecosystem entirely.
Back to the historical data point I made earlier. Maybe your user came from a competitor’s smart contract. You can see if they transacted with them.
Persona analysis
Instead of having your user research participants respond to where they buy their clothes from, you could deterministically identify where they purchase from.
Product, Data, Design, and Marketing teams can work together to create dynamic persona models not only based on first party data, but also public blockchain data for these wallets.
Eg: If a user’s wallet interacts with A, B, or X they go from a ‘Spender’ persona to a ‘Saver’ persona or from ‘Customer’ to ‘At Risk’
Mutually aligned incentives
I will note that even though this is pseudonymous (your wallet ID is a string of alphanumeric characters), it’s still creepy.
I’m not a decentralized ID expert, but there are solutions that will likely allow for more privacy measures be in place, which is overall good for consumers. Because of that, how can both users and the companies they do business with win in these wallet user research situations?
Run ‘dynamic diary studies’ where users can opt-in their wallets and transaction history with the companies. The companies pay the wallet periodically for the shared data they receive. Additional guidelines could be provided with the incentive structure
Users reveal their wallet transaction history (all or selective segments of their wallet) to the company
Few additional notes:
These ideas are complimentary to existing Qual and Quant user research methodologies. It can but IMO shouldn’t replace them.
I’m making an assumption that we’re already in a future where most logins and transactions are based on connecting your wallet. We’re not in that future state, but I think it’s closer than most think.
There are already several large blockchain analytics companies (Chainalysis, Nansen, Elliptic, etc.) but I’m not sure if there are any for specific consumer use cases outside of defi or other use cases
I wouldn’t be surprised if companies like Opensea have already built playbooks for this with the massive amounts of wallet data they have. But I also wouldn’t be surprised if they haven’t yet lol.
There’s a lot of nuances and caveats to these thoughts, but I think there can be a lot done here for companies that want better customer insights while also being thoughtful about privacy (eg: rewarding users for sharing data).
The playbook of best practices for user research will be rewritten with the dimensions of blockchain analytics and wallet analytics. I’m looking forward to reading the playbook in the future 🙂
See you tomorrow folks!