Real user feedback is a vital part of any product development, and the importance of user testing cannot be overestimated here. Unfortunately, for most independent developers this process is literally not affordable and can easily run up to thousands of dollars. In this article, I would like to share with you my methodology for running mobile app user testing which will cost you less than $100 and a few hours of your time. This specific guide describes the testing of applications that are in the pre-release or production stage. Obviously, during the development of a new product, you should strive for feedback as early as possible, but the approaches and goals of this kind of testing will be noticeably different from what I am going to describe here. So let’s get started.

The roadmap

My secret sources for testing on a budget are testers I’m recruiting on Upwork. It's a really good place to find people for this kind of task as there are many who would like to get some ratings and feedback and these micro gigs are the perfect tool for them to do it. The only drawback of this approach for me is that you can't target your audience very accurately, as this process will take forever. If you need a niche audience, it is probably better to pay extra and use specialized services. In order to start testing, we need to have the following things:

• A client profile on Upwork. This company has a good reputation and the process should not be complicated. And of course, everything is official, including VAT invoices.
• Make your application easily accessible to testers, ideally so that they can download it from the App Store. You should not use complex schemes that require user registration or anything like that. This will be a huge imposition to ask of your testers, and can drastically reduce your test success rate.
• Define your test goals. In my case, I test my application's UX, but they may be different for you.
• Write a test plan and detailed instructions for your testers.
• Collect feedback and analyse.

Writing the test plan and instructions

The test plan that you hand over to your testers must be absolutely clear. There is no second chance, and you will not be there during the testing to explain your intentions right away. There are a few important recommendations on how to communicate with your testers to get the best results. Do not ask them to complete a task, you must communicate the real "user stories". This way, your participants will have more context and the actions will look more natural. Also, make sure that you do not give any hints on how to perform these steps. The "user stories" themselves are of course highly dependent on your application, I will provide you with an example later on.

Stepping away from the test plan, make sure that you start the document with a brief description of your application and give participants precise instructions on how to run the test. It is very important to see how users react and what their frustrations are, so I normally ask testers to make a video recording of their interaction with their phone instead of just capturing it onscreen.

Finally, be sure to have brief interviews with the participants to get more information about their background and general impressions.

For more tips, please refer to the ​​NN/g's article where this topic is covered in more detail.

An example of a test plan introduction:

The purpose of this test is to identify ​flaws​ in the application. ​We expect to find the user's pain points and get criticism of this application.​ During the test, ​please speak out loud all your thoughts​ about this application and what you see on the screen.

During the test, ​please record the video of your interaction with the application​ in a way that makes your mobile phone and your interaction with it ​clearly visible​; and please don’t do screen recording. Point a real camera at the phone. The easiest way to do this is to place another phone on the edge of the desk and simply hold your device with the app underneath it for recording.

Please ​don't download the app before you start​ recording as we want to make sure you see the app for the first time.

The purpose of the application is to keep track of your daily water intake. During this test, please imagine that you need to monitor your daily water consumption and you are looking for the right application to do so.

An example set of instructions from the same app:

Please follow these steps after starting the video recording:

1. Open the application page​, give us your opinion on it and tell us what could be improved?
   
2. Install the app on your device.
   
3. Open the app and finish the on-boarding process (the first 4 screens of the initial app setup), give us your opinion on this process.
   
4. You have drunk a glass of water 200ml (8oz) and want to track it. Please do so.
   
5. Imagine you have two cups from which you drink water regularly 300ml(10oz) and 400ml(13.5oz). ​Organise the quick actions ​in such a way as to enable access to these two actions from the main screen. What do you think about this process, what can be improved?
   
6. Imagine that you have drunk an unusual amount of water 100 ml (3oz). Track this intake without adding it to the quick actions.
   
7. Please delete your last tracked water intake.
   
8. Imagine that you get up at 8 am (8:00) and go to bed at 10 pm (22:00). You want to receive water intake reminders every hour when you are awake. How would you achieve that?
   
9. Please tell us your general impressions about the application. Have you used anything similar before?
   
10. What did you not like most about the app?
    
11. What is the most important change you would make to improve this application?
    

An additional recommendation that I would like to give in relation to this document, is that you conduct a “dry-run” and collect individual feedback before proceeding to the real test. If problems arise, correct any misunderstandings in the document to avoid failed tests and save your money later.

Inviting testers and collecting feedback

As I mentioned, we're going to hire our testers on Upwork. ​You need to post an advert for user testers and specify that you are going to hire several people. ​According to NN/g​, you only need 5 tests to find 85% of the usability problems. Usually, I set the price at $10-15 per test, this will be enough to find several testers. Be absolutely clear with the description, especially with requirements such as having a second device for video recording and an application platform.

Example of an advert:

I'm looking for testers for my iOS mobile app. You must have an iPhone and WebCam or another phone to record your interaction with the app. You will be asked to download and install the app from the AppStore and interact with the app according to the instructions while recording the video and saying out loud your impressions about the app. This is a short test and should not take more than 10 minutes.

Always take time to leave positive feedback, even if you are not completely satisfied, as this is the main reason why people do this kind of job. If you are not completely satisfied with their work, just order another test.

Now that you have the results, take a close look at all the videos that you have received. After watching some of these videos, you will definitely notice some patterns and challenges that your testers have come across. Draw your conclusions from it and improve your product.

Wrapping it Up

Proper user testing gives you valuable feedback on what real users expect from your application. However, most independent developers cannot afford it. In this article, I have described my suggestion for user testing on a budget that will help you make sure that you are on the right track and that your application does not have usability issues. In addition, it can save you a lot of development and marketing money if you identify any UX problems during user testing in the early stages.