User research: My experience conducting guerrilla usability tests.

A couple of weeks ago I ran several guerrilla user tests during the redesign of the invite friends page of Bloomon. I approached people in the streets and asked them to go through different screens, and when they finished, I asked them some questions related to the referral program.

The main goal was to figure out if the referral program was well explained and people get it, and to spot pain points in the interface.

It was my first time that I followed this cycle: design some concepts, test them, iterate on those concepts based on the findings of the tests, test them again; until the final solution.

It was very helpful to get to a good solution for the invite friends page and also to understand how people navigate in these type of pages. Besides, it was also a very good exercise for me to learn how to approach strangers on the streets and get their help.

Overall, I think it’s a very good cycle, because in a very short time, a couple of hours, I can improve my concepts and come up with new ones.

As I mentioned before, I was a newbie at approaching people on streets (also I’m a bit shy), so this was my approach:

First, I didn’t prepare a lot my introduction or what I was going to say to people. I just followed some advice from other designers and I started asking people on the streets. I didn’t even focus a lot on a specific type of people, I just asked to everyone that was near me. Obviously, this led to a lot of: “No” and with the people that participated in the tests, I didn’t get the most out of it because I didn’t ask them a lot of valuable questions.

So, as the same with the concepts, I iterated the process and improved it along the way. In the end, I was approaching people that have similar characteristics as bloomon’s customers and asking the right questions.

The main takeaways:

• Do not use words like a test, evaluating, examining, or any similar one. When I said these types of words most people didn’t want to participate in the test. They thought it was a test for them or any kind of exam so they prefer not to take part in it. It seems like something difficult to do, so they just prefer not to do it at all. Instead of that, it’s better to use words like collecting feedback, opinions, and so on. Much more positive and less aggressive.
• Explain that I’m not the designer of that interface, I’m just collecting feedback: And yes, I was the designer of that interface, of course, but people were much more honest when I told them that I was collecting feedback for the design team of my organisation. Also, lucky me that I live in The Netherlands and it doesn’t take too much to get the truth from people ;).
• Avoid asking people with kids: They don’t want to focus the attention on other things and stop looking after their children.
• Tell people that I’ll only need 2 minutes of their time: Five minutes is too much for them.
• And related to the previous one, do not say: Can you spend 2 minutes of your time with me? I asked this question at the beginning, with the first user tests and most of the times, right after saying the word: “spend” most people answered: “No.” I couldn’t even finish the whole question. Instead of that, it worked much better when I said: “Can you help me for 2 minutes?”. The word “help” is much more positive for them than “spend” and they were more willing to participate.
• Give minimum information at the beginning about what I was testing: In order to know if they really understand the interface and is intuitive enough, it’s better not to give a lot about it. In this specific case, I only told them that it was about the invite friends section and the design team was redesign it. But I didn’t mention anything about the wizard or the fact that we have a promo code and a referral link. Check my previous post to know more about it.
• Ask specific questions at the end to make sure your design meets your goals: This one I improved it a lot with practice. At the beginning I didn’t ask any specific question, I just asked them: “what did you just learn about our referral program?” and “What do you think it will happen if you click on a social media icon?”. But then, when I was organizing all the feedback and thinking of how to improve the user interface I came up with new questions that I could ask as well in order to know if the interface was clear enough. For example, I asked about the email icon in particular because some people mentioned they thought they will send an SMS to their friend, and also I asked specific questions about the referral link and promo code to know if they understood the referral program. Sometimes when I asked them: “What did you understand?” They gave a very short answer but in reality, they understood much more than that, so it’s better to ask more questions.
• Try to speak slowly, especially when approaching people: They’re evaluating if you’re a thread for them or not, so they’re not listening to you. Sounds a bit silly but it’s true! Sometimes I have to repeat the first sentence again to them because they didn’t really listen to me.

People’s concerns:

• Are you selling something?: One person asked me what I was trying to sell and he specified that he didn’t want to buy anything. I was quite shocked about this question because I didn’t mention at all that I was selling anything. But it made me think more about my “introduction” and how to approach people in a better way. So they will understand from the beginning what I was doing.
• Will you need my details?: The same person asked me if I’ll need his email, name or phone number after the test. Again, this made me rephrase my “introduction” to them.
• What company do you work for?: Everyone asked me in what company I was working for and what do we sell. So people really need to know what you do and sell before deciding if they want to help you or not. Fair enough ;).
• Questions during the test: I tried to mention to everyone that they have to navigate through the different screens and I won’t be able to help them. So if they have any doubt they will have to figure out by themselves. Even though I mentioned this to them, some people asked me questions during the test about how to proceed or if they were doing the correct thing. Especially older people. I still didn’t help them, I only guide them a little on how to go to the next screen.

To sum-up:

In general, it’s a bit uncomfortable for me to conduct guerrilla user tests with strangers because I feel like I’m bothering people and they don’t have time for that. Maybe it’s because I don’t like to be approached by strangers at streets hahaha.

However, I think for some concepts or redesigns it’s an interesting technique, especially if I don’t have the opportunity to test it with our customers, and I can learn and improve the design a lot with it.

Besides, I think it could be a very good exercise when I’m stuck and I can’t come up with new concepts.

So I’ll definitely continue to do it from time to time.