Support free tutorials



vogella training Training Books

Android application testing with the Android test framework - Tutorial

Lars Vogel

Version 2.0

02.06.2015

Android Testing

This tutorial describes how to test Android applications with different Android testing frameworks.


Table of Contents

1. Why is testing important for Android applications?
1.1. Application under test
1.2. Android quality assurance
1.3. Android test strategy
1.4. History of testing Android applications?
1.5. Android and JUnit 4
2. Prerequisites
3. Android automated testing
3.1. Unit tests vs. integration tests on Android
3.2. Testing a single app or multiple apps
4. Important test ares of Android applications
4.1. What to test on Android applications
4.2. Testing preconditions
4.3. Running tests on a server without display
5. Which tests require an Android system to run?
5.1. Categories of Android tests
5.2. Unit tests also known as local tests
5.3. Instrumented tests for testing Java classes which use the Android API
6. Android project structure and test folder creation
6.1. Android project organization for tests
6.2. Test folder creation in Android Studio
7. Running tests and solving issues with running tests
7.1. Selecting which tests are compiled
7.2. Seeing all passed tests your test builds
7.3. Build error in your test builds
8. Unit testing
8.1. Unit testing in Android
8.2. Location of unit tests
8.3. Required dependencies in the Gradle build file
8.4. Run the unit tests from Gradle
8.5. Run the unit tests from Android Studio
8.6. Location of test reports
8.7. Activating default return values for mocked methods in android.jar
9. Exercise preparation: Create Android project to test project
10. Exercise: Create JUnit 4 test for your Android project
10.1. Exercise high level target
10.2. Create test folder and add dependency
10.3. Create test
10.4. Run unit tests
11. Instrumentation - The underlying Android testing API
11.1. Instrumentation
11.2. How the Android system executes tests
11.3. Usage of the instrumentation framework
12. Instrumented unit testing
12.1. Using instrumented unit tests in Android
12.2. Location of instrumentation tests
12.3. Required dependencies in the Gradle build file
12.4. Run the unit tests from Gradle
12.5. Run the unit tests from Android Studio
12.6. Location of test reports
13. Exercise: Create simple instrumented unit test
14. Mocking objects in Android
15. Using Mockito
16. Using Mockito on Android
17. Exercise: Mocking file access
17.1. Create Android application
17.2. Create a new unit test
18. Android Testing Support Library
19. More on Android testing
19.1. Android additional assertion
19.2. Test groups
19.3. Test filtering
19.4. Flaky tests
20. Activity testing
21. Cross-component user interface testing
21.1. Using UI automator for cross component testing
21.2. uiautomatorviewer
22. Monkey
23. monkeyrunner
23.1. Testing with monkeyrunner
23.2. monkeyrunner example
24. Common Android testing requires and solution approaches
24.1. Common logging on a server
24.2. Triggering system changes via tests
25. Other Open Source testing frameworks
26. Application testing
27. Exercise: Testing the Android application
27.1. Create project
27.2. Create unit test for application object
27.3. Create unit test for the application object
27.4. Create instrumented test for application object
28. Service testing
29. Content provider testing
30. Loader testing
31. About this website
32. Links and Literature
32.1. Android testing resources
32.2. vogella GmbH training and consulting support

1. Why is testing important for Android applications?

1.1. Application under test

The application which is tested is typically called the application under test.

1.2. Android quality assurance

Android applications run on devices with limited memory, CPU power and power supply. The also run on a variety of devices and depend on various factors, like network connectivity, general system lot, etc.

Hence it is very important to debug, test and optimize your Android application. Having a reasonable test coverage for your Android application helps you to enhance and maintain the Android application.

1.3. Android test strategy

As it is not possible to test Android application on all possible device configurations, it is common practice to run Android test on typical device configurations. You should test your application at least on one device with the lowest possible configuration and on one device with the highest available configuration, e.g., pixel density, screen resolution to ensure that it works fine on these devices.

1.4. History of testing Android applications?

Historically the support for testing of Android application was relatively bad, therefore lot of Android applications do not have a good test coverage. Fortunately the available Android test frameworks have been improved over time and today you you great framework support for testing on Android.

1.5. Android and JUnit 4

The Android test system used to be based on JUnit 3, but has been recently updated to be based on JUnit 4. The old test classes are still available but should be avoided for new tests.

2. Prerequisites

The following assumes that you have already a basic understanding of how Android applications are created. See the Android Tutorial for details. Also the following description assume that you know the basis of the Android build system based on Gradle. See Building Android applications with Gradle Tutorial for an introduction.

For more information about Gradle see the Gradle Tutorial.

3. Android automated testing

3.1. Unit tests vs. integration tests on Android

A unit test tests only the functionality of a certain component. Let's, for example, assume a button in an Android activity is used to start another activity. A unit test would determine if the corresponding intent was issued, not if the second activity was started.

An integration test would also check if the activity was correctly started.

3.2. Testing a single app or multiple apps

Another important criteria for testing is if you only test your own application or if you test the integration with other applications. If you test within your application you can use testing frameworks which require some knowledge about your application, e.g., the view IDs.

4. Important test ares of Android applications

4.1. What to test on Android applications

The following tables lists the important areas you should test in your Android applications in additional to the unit tests for the isolated tests.

Table 1. Areas to test

Test area Description
Activity life cycle events You should test if you activity handles the Android life cycle events correctly. You should also test if the configuration change events are handled well and if instance state of your user interface components is restored.
File system and database operations Write and read access from and to the file system should be tested including the handling of databases.
Different device configurations You should also test if your application behaves well on different device configurations.


4.2. Testing preconditions

It is good practice in Android testing to have one method called testPreconditions() which tests the pre-conditions for all other tests. If this method fails, you know immediately that the assumptions for the other tests have been violated.

4.3. Running tests on a server without display

To run tests without a display (headless), specify the adb -no-window parameter.

5. Which tests require an Android system to run?

5.1. Categories of Android tests

Android testing is based on JUnit. Testing for Android can be classified into:

  • Local tests - tests which can run on the JVM

  • Instrumented tests - tests which require the Android system

Android testing categories

If possible, you should prefer to use local tests as as the test execution is much faster compared to the time required to deploy and run the test on an Android device.

5.2. Unit tests also known as local tests

If your classes do not call the Android API or has only minimal dependencies to it, you can use the JUnit test framework (or any other Java testing framework) without any restrictions.

The advantages of the method is that you can use any Java unit testing framework and utility as well as that the execution speed of the unit test is very fast compared to tests which require the Android system.

If you have dependencies to the Android API you can replace these dependencies for these unit tests. This is typically done via a mocking framework like Mockito.

5.3. Instrumented tests for testing Java classes which use the Android API

If you want to test code which use the Android API, you need to run these tests on an Android device. Unfortunately, this makes the execution of tests take longer.

This is because android.jar JAR file does not contain the Android framework code, but only stubs for the type signatures, methods, types, etc. The android.jar JAR file is only used for the Java compiler before deployment on an Android device. It is not bundled with your application. Once your application is deployed on the device, it will use the android.jar JAR file on the Android device. Calling methods from the android.jar JAR file throw a new RuntimeException("Stub!").

This makes it impossible to test the Android framework classes directly on the JVM without additional libraries.

6. Android project structure and test folder creation

6.1. Android project organization for tests

The preferred way of organizing tests is based on a convention. In your application project you should use the following base folder structure for your code organization.

  • app/src/main/java - for your source code of your main application build

  • app/src/test/java - for any unit test which can run on the JVM

  • app/src/androidTest/java - for any test which should run on an Android device

If you follow this conversion than the Android build system can automatically run the unit tests on the JVM and the Android tests on the Android device.

6.2. Test folder creation in Android Studio

To create the test folder in Android Studio, switch to the Project view, this shows you the directory structure of your project.

Switching to the Project view in Android Studio

Select the src and use the context menu to create a new test folder.

Switching to the Project view in Android Studio

Adding a java folder

Adding a java folder

If not yet done, also add the JUnit dependency to your Gradle build file.

dependencies {
    // Unit testing dependencies
    testCompile 'junit:junit:4.12'
    // Set this dependency if you want to use the Hamcrest matcher library
    testCompile 'org.hamcrest:hamcrest-library:1.3'
    // more stuff, e.g., Mockito
} 

Warning

The creation of the Java folder might add the new test directory as source file to your build.gradle file. If you have the following entry in your app/build.gradle file you need to REMOVE it. test should NOT be treated as normale source folder.

sourceSets { main { java.srcDirs = ['src/main/java', 'src/test/java/'] } } 

Afterwards you can write your unit test.

7. Running tests and solving issues with running tests

7.1. Selecting which tests are compiled

Warning

To make Android Studio aware of your new dependency for unit or instrumentation tests you may have to select "Unit Tests" or "Android Instrumentation Tests" from the Build Variant view in Android Studio.

Making Android Studio aware of new dependencies

How you develop your tests depends if you are developing unit tests or integration tests.

7.2. Seeing all passed tests your test builds

To see all passed tests, click on the highlighted toolbar entry in the following screenshot.

Android Studio see all tests

7.3. Build error in your test builds

With your receive the error duplicate files in path. Path in archive: LICENSE.txt add the following to your app/gradle.build file.

android {
    packagingOptions {
    exclude 'LICENSE.txt'
    }
} 

8. Unit testing

8.1. Unit testing in Android

Android uses the term unit tests for tests which can run on a local JVM on the development machine instead of the Android Runtime.

The unit tests are executed against a modified version of the android.jar Android library where all final modifiers have been stripped off. This allow you to mocking libraries, like Mockito. All methods in the used android.jar file throw exceptions (by default). This is to make sure your unit tests only test your code and do not depend on any particular behavior of the Android platform. If you want specific behavior you can use a mocking framework to replace these call.

8.2. Location of unit tests

As described in androidtesting_projectstructure the unit tests of an Android project should be located in the app/src/test folder. See Section 6.2, “Test folder creation in Android Studio” for the creation of such a test folder.

8.3. Required dependencies in the Gradle build file

To use JUnit tests for your Android application you need to add the dependency to JUnit and you other test dependencies to your Gradle build file.

dependencies {
    // Unit testing dependencies
    testCompile 'junit:junit:4.12'
    // Set this dependency if you want to use the Hamcrest matcher library
    testCompile 'org.hamcrest:hamcrest-library:1.3'
    // more stuff, e.g., Mockito
} 

8.4. Run the unit tests from Gradle

Run your unit tests wit the gradlew test command.

8.5. Run the unit tests from Android Studio

Android Studio has two types of artifacts which can be selected in the build variant window. If you select Unit Tests the unit tests are executed on the JVM.

Running the Android tests from Android Studio

To run the unit, ensure that you have select Unit tests, right-click on your test class in the Project window and select Run.

Running Unit tests in Android Studio

8.6. Location of test reports

The rest reports are created in the app/build/reports/tests/debug/ directory. The index.html gives an overview and links to the individual test pages.

8.7. Activating default return values for mocked methods in android.jar

You can also instruct the Gradle build system to return default values for method calls in the android.jar with the following configuration in your Gradle build file.

android {
  // ...
  testOptions { 
    unitTests.returnDefaultValues = true
  }
} 

9. Exercise preparation: Create Android project to test project

Create the Android project as described in Android temperature converter.

10. Exercise: Create JUnit 4 test for your Android project

10.1. Exercise high level target

In this exercise you learn how to create a JUnit4 test for the Android project you created in Section 9, “Exercise preparation: Create Android project to test project”.

10.2. Create test folder and add dependency

Follow the process described in Section 6.2, “Test folder creation in Android Studio” to create a test folder for your unit tests.

Add the Junit dependency to your app/build.gradle file.

dependencies {
  // Unit testing dependencies
  testCompile 'junit:junit:4.12'
} 

10.3. Create test

In your app/src/test directory create the following two test methods of the ConverterUtil class.

package com.vogella.android.temperature.test;

import static org.junit.Assert.*;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import com.vogella.android.temperature.ConverterUtil;

public class ConverterUtilTest {

  @Test
  public void testConvertFahrenheitToCelsius() {
    float actual = ConverterUtil.convertCelsiusToFahrenheit(100);
    // expected value is 212
    float expected = 212;
    // use this method because float is not precise
    assertEquals("Conversion from celsius to fahrenheit failed", expected,
        actual, 0.001);
  }

  @Test
  public void testConvertCelsiusToFahrenheit() {
    float actual = ConverterUtil.convertFahrenheitToCelsius(212);
    // expected value is 100
    float expected = 100;
    // use this method because float is not precise
    assertEquals("Conversion from celsius to fahrenheit failed", expected,
        actual, 0.001);
  }

} 

10.4. Run unit tests

If everything is implemented correctly the tests should execute successfully.

11. Instrumentation - The underlying Android testing API

11.1. Instrumentation

The Android testing API provides hooks into the Android component and application life cycle. These hooks are called the instrumentation API and allow your tests to control the life cycle and user interaction events.

Under normal circumstances your application can only react to the life cycle and user interaction events. For example if Android creates your activity the onCreate() method is called on your activity. Or the user presses a button or a key and your corresponding code is called. Via instrumentation you can control these events via your tests.

Only an instrumentation-based test class allows you to send key events (or touch events) to the application under test.

For example, your test can call the getActivity() method which starts an activity and returns the activity under test. Afterwards, you can call the finish() method, followed by a getActivity() method call again and you can test if the application restored its state correctly.

11.2. How the Android system executes tests

The InstrumentationTestRunner is the base test runner for Android tests. This test runner starts and loads the test methods. Via the instrumentation API it communicates with the Android system. If you start a test for an Android application, the Android system kills any process of the application under test and then loads a new instance. It does not start the application, this is the responsibility of the test methods. The test method controls the life cycle of the components of the application.

The test runner also calls the onCreate() method of the application and activity under test during its initialization.

11.3. Usage of the instrumentation framework

With unit testing for Android which run their unit tests directly on the JVM and popular user interface testing framework like Espresso the developer rarely has to use the instrumentation API directly.

12. Instrumented unit testing

12.1. Using instrumented unit tests in Android

Instrumented unit tests are unit tests that run on Android devices and emulators instead of running on the Java virtual machine. These tests have access to the real device and its resources and are useful to unit test functionality which cannot be easily mocked by mocking frameworks. Also developer uses these tests if the framework functionality must be used to validate a functionality. Example are for example a test which validates a Parelable implementation.

Mockito as mocking framework can still be used to mock the parts of the Android system which are not interesting for the test.

12.2. Location of instrumentation tests

As described in androidtesting_projectstructure the unit tests of an Android project should be located in the app/src/androidTest/java folder. See Section 6.2, “Test folder creation in Android Studio” for the creation of such a test folder.

12.3. Required dependencies in the Gradle build file

To use JUnit tests for your Android application you need to add the dependency to JUnit and you other test dependencies to your Gradle build file. You needs also to specify the default testInstrumentationRunner runner as "android.support.test.runner.AndroidJUnitRunner".

defaultConfig {
       ..... more stuff
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }

dependencies {
    // Unit testing dependencies
    androidTestCompile 'junit:junit:4.12'
    // Set this dependency if you want to use the Hamcrest matcher library
    androidTestCompile 'org.hamcrest:hamcrest-library:1.3'
    // more stuff, e.g., Mockito
} 

12.4. Run the unit tests from Gradle

Run your unit tests wit the gradlew connectedCheck command.

12.5. Run the unit tests from Android Studio

Android Studio has two types of artifacts which can be selected in the build variant window. If you select Android Instrumentation Tests the unit tests are executed on the JVM.

Running the Android instrumentation tests from Android Studio

To run the unit, ensure that you have select Android Instrumentation Tests, right-click on your test class in the Project window and select Run.

12.6. Location of test reports

The rest reports are created in the app/build/reports/androidTests/connected/ directory. The index.html gives an overview and links to the individual test pages.

13. Exercise: Create simple instrumented unit test

You will later learn to create more complex tests but for now duplicate your test you created in ??? and also run it as instrumentation test.

14. Mocking objects in Android

For you unit tests you can use mocking a framework to mock Android objects. This works also for tests running the a Android runtime. The Android framework provided in the past specialized mocking classes but these are not necessary anymore.

15. Using Mockito

See Mockito tutorial for information how to use the Mockito framework.

16. Using Mockito on Android

Mockito can also be directly used in Android unit tests simply by adding the dependency to Mockito to the Gradle build file of the application. To use it in instrumented Android tests (since the release 1.9.5). Which requires that dexmaker and dexmaker-mockito are also added as dependency. in the Gradle build file.

dependencies {
    testCompile 'junit:junit:4.12'
    // required if you want to use Mockito for unit tests
    testCompile 'org.mockito:mockito-core:1.+'
    // required if you want to use Mockito for Android instrumentation tests
    androidTestCompile 'org.mockito:mockito-core:1.+'
    androidTestCompile "com.google.dexmaker:dexmaker:1.2"
    androidTestCompile "com.google.dexmaker:dexmaker-mockito:1.2"
} 

17. Exercise: Mocking file access

17.1. Create Android application

In an existing or new Android application with the package com.vogella.android.testing.mockitocontextmock add the following class.

public class Util {
  public static void writeConfiguration(Context ctx) {
    BufferedWriter writer = null;
    try {
      FileOutputStream openFileOutput = 
       ctx.openFileOutput("config.txt", Context.MODE_PRIVATE);
      openFileOutput.write("This is a test1.".getBytes());
      openFileOutput.write("This is a test2.".getBytes());
    } catch (Exception e) {
      throw new RuntimeException(e);
    } finally {
      if (writer != null) {
        try {
          writer.close();
        } catch (IOException e) {
          e.printStackTrace();
        }

      }
    }
  }
} 

Note

It is not required to use this class in your Android application as we are going to unit test it in isolation.

17.2. Create a new unit test

Using Mockito the mocking framework, write a unit test which validates that:

  • openFileOutput is called exactly once

  • the write() method is called at least twice.

package com.vogella.android.testing.mockitocontextmock;

import android.content.Context;

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;

import java.io.FileOutputStream;

import static org.junit.Assert.fail;
import static org.mockito.Matchers.any;
import static org.mockito.Matchers.anyInt;
import static org.mockito.Matchers.anyString;
import static org.mockito.Mockito.atLeast;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

public class TextContextOutputStream {

@Mock
Context context;

@Mock
FileOutputStream fileOutputStream;

@Before
public void init(){
    MockitoAnnotations.initMocks(this);
}

    @Test
public void writeShouldWriteTwiceToFileSystem() {
    try {
        when(context.openFileOutput(anyString(), anyInt())).thenReturn(fileOutputStream);
        Util.writeConfiguration(context);
        verify(context, times(1)).openFileOutput(anyString(), anyInt());
        verify(fileOutputStream, atLeast(2)).write(any(byte[].class));

    } catch (Exception e) {
        e.printStackTrace();
        fail();
    }
}
} 

18. Android Testing Support Library

The Android Testing Support Library provides functionality to create and run Android tests. The library contains the AndroidJUnitRunner, the Espresso test framework and the UI Automator.

AndroidJUnitRunner allows to create and run JUnit 4 tests, while the Espresso test framework can be used to test the User Interface of your application. UI Automator allows to write cross application functional tests.

AndroidJunitRunner provides access to the instrumentation API, via the InstrumentationRegistery.

  • InstrumentationRegistry.getInstrumentation(), returns the Instrumentation currently running.

  • InstrumentationRegistry.getContext(), returns the Context of this Instrumentation’s package.

  • InstrumentationRegistry.getTargetContext(), returns the application Context of the target application.

  • InstrumentationRegistry.getArguments(), returns a copy of arguments Bundle that was passed to this Instrumentation. This is useful when you want to access the command line arguments passed to Instrumentation for your test.

It also gives access to the life cycle via the ActivityLifecycleMonitorRegistry.

19. More on Android testing

19.1. Android additional assertion

The Android testing API provides the MoreAsserts and ViewAsserts classes in addition to the standard JUnit Assert class.

19.2. Test groups

The @SmallTest, @MediumTest and @LargeTest annotations allows to classify tests.

This allows you to run, for example, only tests which do not run very long. Long running tests could than run only in the continuous integration server.

To run only selected tests you can configure the InstrumentationTestRunner via your Gradle plug-in. The following listing is an example for your build.gradle file to run only the tests annotated with @SmallTests.

android {
  //....
  defaultConfig {
  //....
    testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    testInstrumentationRunnerArgument "size", "small"
  }
} 

Note

This works only as of the Android Plugin for Gradle 1.3.0. Also this is currently not yet supported in Android Studio, i.e., if you run a test from Android Studio the selection in the Gradle build file is ignored.

import android.test.suitebuilder.annotation.MediumTest;
import android.test.suitebuilder.annotation.SmallTest;

import org.junit.Test;

public class ExampleTest {

    @Test
    @SmallTest
    public void validateSecondActivity() {
        // Do something not so long...
    }

    @Test
    @MediumTest
    public void validateSecondActivityAgain() {
        // Do something which takes more time....
    }

} 

19.3. Test filtering

You can annotate tests with annotations. The Android test runner allows to filter these tests.

Table 2. Annotations for filtering tests

Annotation Description
@RequiresDevice: Specifies that the test should run only on physical devices, not on emulators.
@SdkSupress: @SDKSupress(minSdkVersion=18)


19.4. Flaky tests

Actions in Android are sometimes time dependent. To tell Android to repeat a test once it fails, use the @FlakyTest annotation. Via the tolerance attribute of this annotation you can define how often the Android test framework should try to repeat a test before marking it as failed.

20. Activity testing

See Android user interface testing with Espresso

21. Cross-component user interface testing

21.1. Using UI automator for cross component testing

Functional or black-box user interface testing does test the complete application and not single components of your application.

The Android SDK contains the uiautomator Java library for creating user interface tests and provides an engine to run these user interface tests. Both tools work only as of API 16.

The uiautomator test projects are standalone Java projects which the JUnit3 library and the uiautomator.jar and android.jar files from the android-sdk/platforms/api-version directory added to the build path.

Androids uiautomator provides the UiDevice class to communicate with the device, the UiSelector class to search for elements on the screen and the UiObject which presents user interface elements and is created based on the UiSelector class. The UiCollection class allows to select a number of user interface elements at the same time and UiScrollable allows to scroll in a view to find an element.

The following coding shows an example test from the official Android developer side. The URL for this is Testing UI example.

package com.uia.example.my;

// Import the uiautomator libraries
import com.android.uiautomator.core.UiObject;
import com.android.uiautomator.core.UiObjectNotFoundException;
import com.android.uiautomator.core.UiScrollable;
import com.android.uiautomator.core.UiSelector;
import com.android.uiautomator.testrunner.UiAutomatorTestCase;

public class LaunchSettings extends UiAutomatorTestCase {

  public void testDemo() throws UiObjectNotFoundException {

    // Simulate a short press on the HOME button.
    getUiDevice().pressHome();

    // We’re now in the home screen. Next, we want to simulate
    // a user bringing up the All Apps screen.
    // If you use the uiautomatorviewer tool to capture a snapshot
    // of the Home screen, notice that the All Apps button’s
    // content-description property has the value “Apps”. We can
    // use this property to create a UiSelector to find the button.
    UiObject allAppsButton = new UiObject(new UiSelector().description("Apps"));

    // Simulate a click to bring up the All Apps screen.
    allAppsButton.clickAndWaitForNewWindow();

    // In the All Apps screen, the Settings app is located in
    // the Apps tab. To simulate the user bringing up the Apps tab,
    // we create a UiSelector to find a tab with the text
    // label “Apps”.
    UiObject appsTab = new UiObject(new UiSelector().text("Apps"));

    // Simulate a click to enter the Apps tab.
    appsTab.click();

    // Next, in the apps tabs, we can simulate a user swiping until
    // they come to the Settings app icon. Since the container view
    // is scrollable, we can use a UiScrollable object.
    UiScrollable appViews = new UiScrollable(new UiSelector().scrollable(true));

    // Set the swiping mode to horizontal (the default is vertical)
    appViews.setAsHorizontalList();

    // create a UiSelector to find the Settings app and simulate
    // a user click to launch the app.
    UiObject settingsApp = appViews
        .getChildByText(new UiSelector()
            .className(android.widget.TextView.class.getName()),
            "Settings");
    settingsApp.clickAndWaitForNewWindow();

    // Validate that the package name is the expected one
    UiObject settingsValidation = new UiObject(new UiSelector()
        .packageName("com.android.settings"));
    assertTrue("Unable to detect Settings", settingsValidation.exists());
  }
} 

You need to use Apache Ant to build and deploy the corresponding project.

<android-sdk>/tools/android create uitest-project -n <name> -t 1 -p <path>

# build the test jar
ant build

# push JAR to device
adb push output.jar  /data/local/tmp/

# Run the test
adb shell uiautomator runtest LaunchSettings.jar -c com.uia.example.my.LaunchSettings 

21.2. uiautomatorviewer

Android provides the uiautomatorviewer tool, which allows you to analyze the user interface of an application. You can use this tool to find the index, text or attribute of the application.

This tool allows non-programmers to analyze an application and develop tests for it via the uiautomator library.

The tool is depicted in the following screenshot.

uiautomatorviewer in usage

22. Monkey

Monkey is a command line tool which sends pseudo random events to your device. You can restrict Monkey to run only for a certain package and therefore instruct Monkey to test only your application.

For example, the following will send 2000 random events to the application with the de.vogella.android.test.target package.

adb shell monkey -p de.vogella.android.test.target -v 2000 

Monkey sometimes causes problems with the adb server. Use the following commands to restart the adb server.

adb kill-server
adb start-server 

You can use the -s [seed] parameter to ensure that the generated sequence of events is always the same.

For more info on Monkey please see the Monkey description.

23. monkeyrunner

23.1. Testing with monkeyrunner

The monkeyrunner tool provides a Python API for writing programs that control an Android device or emulator from outside of Android code.

Via monkeyrunner you can completely script your test procedure. It runs via the adb debug bridge and allows you to install program, start them, control the flow and also take screenshots or your application.

To use monkeyrunner, ensure that you have Python installed on your machine and in your path.

In monkeyrunner you have primary the following classes:

  • MonkeyRunner: allows to connect to devices.

  • MonkeyDevice: allows to install and uninstall packages and to send keyboard and touch events to an application.

  • MonkeyImage: allows to create screenshots, compare screenshots and save them.

MonkeyImage can compare the screenshot with an existing image via the sameAs() method. A screenshot contains the Android notifcation bar, including time. You can enter a percentage as second parameter for sameAs() or use the getSubImage() method.

The API reference for monkeyrunner can be generated via the following command.

# outfile is the path qualified name
# of the output file
monkeyrunner help.py help <outfile> 

23.2. monkeyrunner example

Ensure Python is installed and in your path. Also ensure the [android-sdk]/tools folder is in your path. Create a file, for example, called testrunner.py

from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice
import commands
import sys
import os

# starting the application and test
print "Starting the monkeyrunner script"

if not os.path.exists("screenshots"):
    print "creating the screenshots directory"
    os.makedirs("screenshots")

# connection to the current device, and return a MonkeyDevice object
device = MonkeyRunner.waitForConnection()

apk_path = device.shell('pm path com.vogella.android.test.simpleactivity')
if apk_path.startswith('package:'):
    print "application installed."
else:
    print "not installed, install APK"
    device.installPackage('com.vogella.android.test.simpleactivity.apk')

print "starting application...."
device.startActivity(component='com.vogella.android.test.simpleactivity/[...CONTINUE]
com.vogella.android.test.simpleactivity.MainActivity')

#screenshot
MonkeyRunner.sleep(1)
result = device.takeSnapshot()
result.writeToFile('./screenshots/splash.png','png')
print "screenshot taken and stored on device"

#sending an event which simulate a click on the menu button
device.press('KEYCODE_MENU', MonkeyDevice.DOWN_AND_UP)

print "Finishing the test" 

You run this test via the monkeyrunner testrunner.py command on the console.

24. Common Android testing requires and solution approaches

24.1. Common logging on a server

Frequently the log files of the tests should be stored on a server and not on the device. A good practice is to provide a server backend and post the result via an HTTP request to this server. The server logs it centrally and provides central access to it.

24.2. Triggering system changes via tests

During tests you sometimes want to change the system status, e.g., turn WIFI of, for example. Typically, this cannot be done via the test directly, as the test only has the permissions of the application under test.

A good practice is to install another application on the device which has the required permission and trigger it via an intent from the test.

25. Other Open Source testing frameworks

Robotium is an Open Source framework on top of the Android testing framework which makes the testing API simpler. See Robotium for user interface testing with the Robotium framework.

Robolectric is an Open Source framework which allows you to run tests which use the Android API directly on the JVM. See the Robolectric tutorial for more information.

Roboguice allows you to use dependency injection in your Android components which simplifies testing. See Using Roboguice.

26. Application testing

The application class contains the logic, data and settings which are relevant for the whole application. Therefore you should test this object, to ensure it works correctly.

You can write JUnit 4 for the application object and test it on the JVM. In this case you would mock all dependencies to the application object.

To test an Android application object on the Andriod runtime you use the ApplicationTestCase class. It is expected that Google will soon provide a special JUnit4 rule for testing the application object but at the moment his is not yet available.

The test runner of of Android tests (InstrumentationTestRunner) creates automatically an instance of application during its initialization phase. If you do asynchronous processing in your onCreate method you should consider that.

27. Exercise: Testing the Android application

27.1. Create project

Create a new Android application with the com.vogella.android.testing.applicationtest package name based on the Blank Activity template.

Add the following application object to your application.

package com.vogella.android.testing.applicationtest;

import android.app.Application;

import java.util.ArrayList;
import java.util.List;

public class MyApplication extends Application {
    public static final List<String> list = new ArrayList<String>();
} 

Also declare the application it in your manifest file.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.vogella.android.testing.applicationtest" >

    <application
        android:name=".MyApplication"
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:theme="@style/AppTheme" >
        <activity
            android:name=".MainActivity"
            android:label="@string/app_name" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest> 

27.2. Create unit test for application object

In your app/src/test directory a new unit test and assert that the MyApplication.list field is not empty and has initially a size of zero.

27.3. Create unit test for the application object

Create the following unit test for your application object.

package com.vogella.android.testing.applicationtest;

import android.content.pm.PackageInfo;
import android.test.ApplicationTestCase;
import android.test.MoreAsserts;

public class ApplicationTest extends ApplicationTestCase<MyApplication> {

    private MyApplication application;

    public ApplicationTest() {
        super(MyApplication.class);
    }

    protected void setUp() throws Exception {
        super.setUp();
        createApplication();
        application = getApplication();

    }

    public void testCorrectVersion() throws Exception {
        PackageInfo info = application.getPackageManager().getPackageInfo(application.getPackageName(), 0);
        assertNotNull(info);
        MoreAsserts.assertMatchesRegex("\\d\\.\\d", info.versionName);
    }

} 

27.4. Create instrumented test for application object

Create the following unit test based on the JUnit 3 testing framework.

package com.vogella.android.testing.applicationtest;

import android.content.pm.PackageInfo;
import android.test.ApplicationTestCase;
import android.test.MoreAsserts;

public class ApplicationTest extends ApplicationTestCase<MyApplication> {

    private MyApplication application;

    public ApplicationTest() {
        super(MyApplication.class);
    }

    protected void setUp() throws Exception {
        super.setUp();
        createApplication();
        application = getApplication();

    }

    public void testCorrectVersion() throws Exception {
        PackageInfo info = application.getPackageManager().getPackageInfo(application.getPackageName(), 0);
        assertNotNull(info);
        MoreAsserts.assertMatchesRegex("\\d\\.\\d", info.versionName);
    }

} 

28. Service testing

To test a service you use the ServiceTestRule class from the Android Testing Support Library. The old ServiceTestCase is deprecated.

This rule provides a simplified mechanism to start and shutdown your service before and after your test. It guarantees that the service is successfully connected when starting (or binding to) a service. The service can be started (or bound) using one of the helper methods. It will automatically be stopped (or unbound) after the test completes and any methods annotated with @After are finished.

Note

Note: This rule doesn't support IntentService because it's automatically destroyed after the onHandleIntent method.

The following listing is an example of testing a service.

@RunWith(AndroidJUnit4.class)
@MediumTest
public class MyServiceTest {

    @Rule
    public final ServiceTestRule mServiceRule = new ServiceTestRule();

    // test for a service which is started with startService
    @Test
    public void testWithStartedService() {
        mServiceRule.
        startService(new Intent(InstrumentationRegistry.getTargetContext(), 
            MyService.class));
        // test code
    }

    @Test
 // test for a service which is started with bindService
    public void testWithBoundService() {
        IBinder binder = mServiceRule.
            bindService(new Intent(InstrumentationRegistry.getTargetContext(), 
                MyService.class));
        MyService service = ((MyService.LocalBinder) binder).getService();
        assertTrue("True wasn't returned", service.doSomethingToReturnTrue());
    }
} 

29. Content provider testing

To test a content provider, you use the ProviderTestCase2 class. ProviderTestCase2 automatically instantiates the provider under test and inserts an IsolatedContext object which is isolated from the Android system, but still allows file and database access.

The usage of the IsolatedContext object ensures that your provider test does not affect the real device.

ProviderTestCase2 also provides access to a MockContentResolver via the getMockCOnktentResolver() method.

You should test all operations of the provider and also what happens if the provider is called with an invalid URI or with an invalid projection.

30. Loader testing

To test a loader, you use the LoaderTestCase class.

31. About this website

32. Links and Literature

32.2. vogella GmbH training and consulting support

TRAINING SERVICE & SUPPORT
The vogella company provides comprehensive training and education services from experts in the areas of Eclipse RCP, Android, Git, Java, Gradle and Spring. We offer both public and inhouse training. Whichever course you decide to take, you are guaranteed to experience what many before you refer to as “The best IT class I have ever attended”. The vogella company offers expert consulting services, development support and coaching. Our customers range from Fortune 100 corporations to individual developers.