Testing Mobile Applications Across Hundreds of Real Devices with Appium, Node.js & AWS Device Farm
How to test mobile applications on a fleet of real devices using Mocha, Appium & AWS Device Farm.
With Device Farm you can view videos, screenshots, logs, and performance data to pinpoint and fix issues and increase quality before shipping your app. AWS Device Farm lets you test your application on a shared fleet of over 2,500 real devices or on your own private device fleet in the cloud.
Device Farm recently added support for Node.js, meaning you can run Appium, Detox, & any JS testing framework.
In this blog post, we’re going to dive into how to use Device Farm to run & test a sample application using Node.js & Appium.
To get started, you’ll need 3 things:
- An AWS Account
- An Appium Node.js test
- An application
If you do not have a test already written or an application to test, I provide them for you in the following section.
If you do have a test already written and an .ipa or .apk file, skip to the section “Packaging the Test Files for Device Farm”. If not, continue to the next step of creating the demo test.
The first thing we’ll do is create a basic test for our application. To do this, we’ll need to create a test file (I’ll call mine
In this test we’re doing two things:
- We first test to see that the application loads
- Next, we’re running a loop that will check to see if a button exists, click on the button, & then go back. The loop runs 5 times.
The code provided is targeting the example
.apkwe will be providing you for the tutorial. If you bring you own
.apkyou will need to write your own test.
Next, we’ll create a
package.json file for the dependencies that we’ll need for this test:
Next, we’ll need to install the dependencies we’ll need for this test to run:
Packaging the Test Files for Device Farm
Now we need to package the files in this folder into a
.tgz file. To do so, we’ll first install npm bundle:
npm install -g npm-bundle
Next, we’ll run the npm bundle command from within root of our test:
Now, we’ll zip the
zip -r MyTest.zip *.tgz
Our test is now written, bundled & ready to go.
Finally, export your
.apk file if you have an existing application, or download the example Android
.apk file for the above test at this link.
Running your Test on AWS Device Farm
Now that we have a test written & an
.apk to run the test on, we can navigate to the Device Farm console at https://console.aws.amazon.com/devicefarm.
- Click Create New Project & give the project a name
- Click Create a New Run
- Click on the Android / iOS Button (to specify a Native application)
- Upload the
- Give the Run a name & click Next Step.
Next, in the dropdown menu for the type of test to run, you’ll see the option for Appium Node.js. Choose this option & upload the zipped folder we created in the previous steps:
Next, we’ll be given a default TestSpec YAML file that will describe some configuration for our test.
test: phase, we’ll need to update the
command: property to be the command we need to use to execute our mocha test.
Here, we’ll replace:
Then, click Save TestSpec.
In this YAML file you can also set other configuration for your test environment, including things like the node environment you’d like to be running in, post test commands, & ADB device commands (like turning animations off).
After the test has been saved, click Next step.
In the next screen, we’ll be given the option to choose a custom set of devices (a Device Pool) or choose a default Device Pool. Here, either choose the default Device Pool or create your own using the devices you’d like to test on.
After you’ve chosen your Device Pool, click Next step.
In the next screen, we’re given some options to specify device state. Here, you can set custom device settings that you’d like to configure on each device (things like geolocation, wifi status, etc..).
Here, either leave the defaults or configure your custom device state & click Next step.
Now, the test is ready to run & we can click Confirm & start run.
Once the test is complete, you should see the test results overview in the dashboard. To view the details of the test, click on the test name.
In this screen, you’ll be able to view a test overview for each device. You can click on an individual device to view details of the test.
After clicking on an individual device, you’ll be taken to the device details page. In this view, you can view details of the individual test including videos of the test, test logs, app performance metrics & more.
To learn more about AWS Device Farm, check out the documentation here.
To view all available devices in Device Farm, click here.
My Name is Nader Dabit .
I am a Developer Advocate at AWS Mobile working with projects like AWS AppSync and AWS Amplify, the author of React Native in Action, & the editor of React Native Training & OpenGraphQL.