device_talosrunner.py: a first peek

[escapewindow]

[what is it?]

device_talosrunner.py is a work-in-progress mozharness script that sets up talos and runs it against a device using either the sut or adb protocols.

It's designed to:

• allow anyone with a [supported and rooted] Android device to run talos without building the tree;
• allow someone with access to a staging/production tegra to run a production-like talos run without setting up buildbot;
• at some point, become the official way we run talos on Android in production.

This is similar to, and parallel to, the work on make talos-remote. However, aiui, that is developer-oriented, whereas device_talosrunner.py is targeted towards someone who wants to test pre-built apks (as well as replacing our current production code).

I have used this script to run talos on both a tegra over sut, and an Asus Eee Pad Transformer using adb-over-ip, successfully.

[how do i use it?]

You need:

• python (2.5 - 2.7.x)
• virtualenv
• hg, to clone the various repos
• a copy of my github mozharness repo, on the branch 'talosrunner';
• a rooted, supported Android device that is either running the sutagent, or is attach{ed,able} via adb. The adb calls require busybox.

Once you have those things,

1. Create a config file. I've got a wip tablet config that I use for my Asus Eee Pad Transformer, and a wip tegra config that I use for a staging tegra.
   
   
2. If you're using adb-over-usb, the device needs to be connected to your desktop/laptop. If you're using adb-over-ip or sut, specifying the device_ip should be sufficient.
   
   
3. If this is the first time you're running device_talosrunner.py, you need a virtualenv. Run

   mozharness/scripts/device_talosrunner.py --cfg users/aki/tablet1.py --create-virtualenv

   (replacing the config file path with the path to your config file). This will create a python virtualenv in ./venv and install PyYAML, a Talos dependency, into it.
   
   
4. Run

   mozharness/scripts/device_talosrunner.py --cfg users/aki/tablet1.py

   If the start_python_webserver option is set, the script will start a webserver and point the device at it. Otherwise it'll point the device at whatever talos_webserver is set.
   
   When this script is polished and mature, I definitely want people to be comfortable using it on their personal phones and tablets, but it's still rough around the edges.
   
   This will uninstall Fennec and possibly reboot your device, so please be careful if you're trying this on a device/profile that you haven't backed up.

[what's left to do?]

• I've only tested a handful of talos suites against two devices; for this to be a production replacement, it needs to be able to run reliably against a large pool of devices.
• This relies on a talos patch in bug 688604 for the python webserver; I most likely need to use mozhttpd.
• I've hit issues with both m-c and birch builds, and will most likely need to make changes for the native UI.
• I have an idea which would allow us to find the next available and working tegra, which may reduce red and purple Tegra results; this needs writing and testing. More on that later.