search

openstf / stf

Control and manage Android devices from your browser.
https://openstf.io
Other
13.39k stars 2.79k forks source link
android android-development device-farm stf

readme

Warning

This project along with other ones in OpenSTF organisation is provided as is for community, without active development.

You can check any other forks that may be actively developed and offer new/different features here.

Active development has been moved to DeviceFarmer organisation. Version 3.4.2 is the last release of OpenSTF on DockerHub and 3.4.1 on npmjs.

STF

Build Status Docker Pulls NPM version Backers on Open Collective Sponsors on Open Collective

STF (or Smartphone Test Farm) is a web application for debugging smartphones, smartwatches and other gadgets remotely, from the comfort of your browser.

STF was originally developed at CyberAgent to control a growing collection of more than 160 devices. As of July 2016 development is mainly sponsored by HeadSpin and other individual contributors.

We welcome financial contributions in full transparency on our open collective.

Close-up of device shelf

Super short screencast showing usage

Credits

Sponsors

Thank you to all our sponsors! (please ask your company to also support this open source project by becoming a sponsor)

Gold Sponsor

HeadSpin

HeadSpin provides secure and scalable STF for iOS integrated with Appium/XCTest/Selenium/Espresso, High speed interaction Audio/Video/Game testing and AI based Root cause analysis for Performance Management. It's free to start using HeadSpin in 150+ locations worldwide! Try it out for free.

HeadSpin offers a generous monthly contribution towards STF development.

Contributors

Thank you to all the people who have already contributed to STF!

Backers

Thank you to all our backers! [Become a backer]

How to become a sponsor

Please use our open collective or contact us directly for sponsor arrangements. Both recurring and one-time contributions are most welcome. Contributions towards a specific issue or feature are also possible, and can be attributed to your company in our release notes and other related materials. Hardware-only contributions, whether new or used, are also extremely helpful and well received, especially if you find a device that doesn't work. Please see our donation transparency report for past hardware contributions.

Features

Status

STF is in continued, active development, but development is still largely funded by individual team members and their unpaid free time, leading to slow progress. While normal for many open source projects, STF is quite heavy on the hardware side, and is therefore somewhat of a money sink. See how to become a sponsor if you or your company would like to support future development.

We're also actively working to expand the team, don't be afraid to ask if you're interested.

Short term goals

Here are some things we are planning to address ASAP.

  1. Performance
  2. Properly expose the new VNC functionality in the UI
  3. Properly reset user data between uses (Android 4.0+)
  4. Automated scheduled restarts for devices
  5. More!

Consulting services

We highly encourage open participation in the community. However, if you're running a business that uses STF or would like to use STF, you may sometimes want to have an expert, i.e. one of the original developers or a skilled contributor, work with you to set up a prototype for evaluation purposes, add support for new or old hardware, figure out an issue, fix a bug or add some new feature. Our services are similar to FFmpeg's. Contact us with details and we'll see what we can do.

Availability is limited and tied to individual developer's schedules.

A quick note about security

As the product has evolved from an internal tool running in our internal network, we have made certain assumptions about the trustworthiness of our users. As such, there is little to no security or encryption between the different processes. Furthermore, devices do not get completely reset between uses, potentially leaving accounts logged in or exposing other sensitive data. This is not an issue for us, as all of our devices are test devices and are only used with test accounts, but it may be an issue for you if you plan on deploying STF to a multiuser environment. We welcome contributions in this area.

Requirements

Note that you need these dependencies even if you've installed STF directly from NPM, because they can't be included in the package.

On Mac OS, you can use homebrew to install most of the dependencies:

brew install rethinkdb graphicsmagick zeromq protobuf yasm pkg-config

On Windows you're on your own. In theory you might be able to get STF installed via Cygwin or similar, but we've never tried. In principle we will not provide any Windows installation support, but please do send a documentation pull request if you figure out what to do.

We also provide a Docker container in the Docker Hub as openstf/stf. You can use our Dockerfile as guidance if you'd prefer to do the installation yourself.

You should now be ready to build or run STF.

Note that while Mac OS can be used for development, it doesn't provide a very reliable experience in production due to (presumed) bugs in ADB's Mac OS implementation. We use CoreOS but any Linux or BSD distribution should do fine.

Installation

As mentioned earlier, you must have all of the requirements installed first. Then you can simply install via NPM:

npm install -g stf

Now you're ready to run. For development, though, you should build instead.

Building

After you've got all the requirements installed, it's time to fetch the rest of the dependencies.

First, fetch all NPM and Bower modules:

npm install

You may also wish to link the module so that you'll be able to access the stf command directly from the command line:

npm link

You should now have a working installation for local development.

Running

STF comprises of several independent processes that must normally be launched separately. In our own setup each one these processes is its own systemd unit. See DEPLOYMENT.md and Setup Examples if you're interested.

For development purposes, however, there's a helper command to quickly launch all required processes along with a mock login implementation. Note that you must have RethinkDB running first.

If you don't have RethinkDB set up yet, to start it up, go to the folder where you'd like RethinkDB to create a rethinkdb_data folder in (perhaps the folder where this repo is) and run the following command:

rethinkdb

Note: if it takes a long time for RethinkDB to start up, you may be running into rethinkdb/rethinkdb#4600 (or rethinkdb/rethinkdb#6047). This usually happens on macOS Sierra. To fix this on macOS, first run scutil --get HostName to check if the HostName variable is unset. RethinkDB needs it to generate a server name for your instance. If you find that it's empty, running sudo scutil --set HostName $(hostname) has been confirmed to fix the issue on at least one occasion. See the issues for more complete solutions.

You should now have RethinkDB running locally. Running the command again in the same folder will reuse the data from the previous session.

An administrator level is available in STF in addition of the native user one, with increased rights on some features (e.g. booking & partitioning systems, management of users & devices, ...). The corresponding built-in administrator user has the following default credentials:

Another built-in object exists, this is the root standard group to which the users and devices belong the first time they register to the STF database, its default name is Common

These built-in objects are created in the STF database if they do not already exist

Of course, you can override the default values of these built-in objects by settings the following environment variables before to initialize the STF database through stf local or stf migrate commands:

You're now ready to start up STF itself:

stf local

After the webpack build process has finished (which can take a small while) you should have your private STF running on http://localhost:7100. If you had devices connected before running the command, those devices should now be available for use. If not, you should see what went wrong from your console. Feel free to plug in or unplug any devices at any time.

Note that if you see your device ready to use but without a name or a proper image, we're probably missing the data for that model in our device database. Everything should work fine either way.

If you want to access STF from other machines, you can add the --public-ip option for quick testing.

stf local --public-ip <your_internal_network_ip_here>

Updating

To update your development version, simply pull the repo and run npm install again. You may occasionally have to remove the whole node_modules and res/bower_components folder to prevent NPM or Bower from complaining about version mismatches.

FAQ

Can I deploy STF to actual servers?

Yes, see DEPLOYMENT.md and Setup Examples.

Will I have to change battery packs all the time?

No, not all the time. Aside from a single early failure we had within only a few months, all of our devices were doing fine for about two years. However, having reached the 2-3 year mark, several devices have started to experience visibly expanded batteries. Expanded batteries should be replaced as soon as possible. Note that this issue isn't specific to STF, it's just what happens over time. You should be prepared to replace the batteries every now and then. In any case, we consider 2 years per battery pack to be fairly good value for a device lab.

You should set up your devices so that the display is allowed to turn off entirely after a short timeout. 30 seconds or so should do just fine, STF will wake it up when necessary. Otherwise you risk reducing the lifetime of your device.

Note that you may have a problem if your USB hubs are unable to both provide enough power for charging and support a data connection at the same time (data connections require power, too). This can cause a device to stop charging when being used, resulting in many charging cycles. If this happens you will just need to get a better USB hub.

Is the system secure?

It's possible to run the whole user-facing side behind HTTPS, but that's pretty much it. All internal communication between processes is insecure and unencrypted, which is a problem if you can eavesdrop on the network. See our quick note about security.

Can I just put the system online, put a few devices there and start selling it?

Yes and no. See "Is the system secure?". The system has been built in an environment where we are able to trust our users and be confident that they're not going to want to mess with others. In the current incarnation of the system a malicious user with knowledge of the inner workings will, for instance, be able to control any device at any time, whether it is being used by someone or not. Pull requests are welcome.

Once I've got the system running, can I pretty much leave it like that or is manual intervention required?

In our experience the system runs just fine most of the time, and any issues are mostly USB-related. You'll usually have to do something about once a week.

The most common issue is that a device will lose all of its active USB connections momentarily. You'll get errors in the logs but the worker process will either recover or get respawned, requiring no action on your side.

Below are the most common errors that do require manual intervention.

How do I uninstall STF from my device?

When you unplug your device, all STF utilities except STFService stop running automatically. It doesn't do any harm to force stop or uninstall it.

To uninstall the STFService, run the following command:

adb uninstall jp.co.cyberagent.stf

You may also wish to remove our support binaries, although as mentioned before they won't run unless the device is actually connected to STF. You can do this as follows:

adb shell rm /data/local/tmp/minicap \
  /data/local/tmp/minicap.so \
  /data/local/tmp/minitouch \
  /data/local/tmp/minirev

Your device is now clean.

Troubleshooting

I plugged in a new device but it's not showing up in the list.

There can be various reasons for this behavior. Some especially common reasons are:

A device that was previously connected no longer shows up in the list.

Again, there can be various reasons for this behavior as well. Some common reasons are:

Remote debugging (i.e. adb connect) disconnects while I'm working.

If you're using STF locally, the most common cause is that you're not filtering the devices STF is allowed to connect to. The problem is that once you do adb connect, STF sees a new device and tries to set it up. Unfortunately since it's already connected via USB, setting up the new device causes the worker process handling the original USB device to fail. This is not a problem in production, since the devices should be connected to an entirely different machine anyway. For development it's a bit inconvenient. What you can do is give stf local a list of serials you wish to use. For example, if your device's serial is 0123456789ABCDEF, use stf local 0123456789ABCDEF. Now you can use adb connect and STF will ignore the new device.

There's another likely cause if you're running STF locally. Even if you whitelist devices by serial in STF, your IDE (e.g. Android Studio) doesn't know anything about that. From the IDE's point of view, you have two devices connected. When you try to run or debug your application, Android Studio suddenly notices that two devices are now providing JDWP connections and tries to connect to them both. This doesn't really work since the debugger will only allow one simultaneous connection, which causes problems with ADB. It then decides to disconnect the device (or sometimes itself) entirely.

One more sad possibility is that your Android Studio likes to restart ADB behind the scenes. Even if you restart ADB, USB devices will soon reappear as they're still connected. The same is not true for remote devices, as ADB never stores the list anywhere. This can sometimes also happen with the Android Device Monitor (monitor).

Recommended hardware

This is a list of components we are currently using and are proven to work.

PC components

These components are for the PC where the USB devices are connected. Our operating system of choice is CoreOS, but any other Linux or BSD distribution should do fine. Be sure to use reasonably recent kernels, though, as they often include improvements for the USB subsystem.

Our currently favorite build is as follows. It will be able to provide 28 devices using powered USB hubs, and about 10 more if you're willing to use the motherboard's USB ports, which is usually not recommended for stability reasons. Note that our component selection is somewhat limited by their availability in Japan.

Component Recommendation How many
PC case XIGMATEK Nebula x1
Motherboard ASUS H97I-PLUS x1
Processor Intel® Core™ i5-4460 x1
PSU Corsair CX Series™ Modular CX430M ATX Power Supply x1
Memory Your favorite DDR3 1600 MHz 8GB stick x1
SSD A-DATA Premier Pro SP900 64GB SSD x1
USB extension card StarTech.com 4 Port PCI Express (PCIe) SuperSpeed USB 3.0 Card Adapter w/ 4 Dedicated 5Gbps Channels - UASP - SATA / LP4 Power x1
USB hub Plugable USB 2.0 7 Port Hub with 60W Power Adapter x4
MicroUSB cable Monoprice.com 1.5ft USB 2.0 A Male to Micro 5pin Male 28/24AWG Cable w/ Ferrite Core (Gold Plated) x28

You may also need extension cords for power.

Alternatively, if you find that some of your older devices do not support the recommended hub, you may wish to mix the hub selection as follows:

Component Recommendation How many
USB hub Plugable USB 2.0 7 Port Hub with 60W Power Adapter x2
USB hub for older devices System TALKS USB2-HUB4XA-BK x2-4

You can connect up to two of the older hubs (providing up to 8 devices total) directly to the motherboard without exhausting USB host controller resources.

We also have several "budget builds" with an MSI AM1I motherboard and an AMD Athlon 5350 4-core processor. These builds, while significantly cheaper, sometimes completely lose the USB PCIE extension cards, and even a reboot will not always fix it. This may normally be fixable via BIOS USB settings, but unfortunately the budget motherboard has a complete lack of any useful options. Fortunately the AMD processor does not share Intel's Haswell USB host control resource problem, so you can also just connect your hubs to the motherboard directly if you don't mind sharing the root bus.

Below is an incomplete list of some of the components we have tried so far, including unsuitable ones.

Tested equipment

Note that our hardware score ratings only reflect their use for the purposes of this project, and are not an overall statement about the quality of the product.

USB extension cards
Name Score Short explanation
StarTech.com 4 Port PCI Express (PCIe) SuperSpeed USB 3.0 Card Adapter w/ 4 Dedicated 5Gbps Channels - UASP - SATA / LP4 Power 9/10 Reliable, well supported chipset and good power connections
StarTech.com 4 Independent Port PCI Express USB 2.0 Adapter Card 8/10 Reliable
玄人志向 USB3.0RX4-P4-PCIE 4/10 Well supported chipset but breaks VERY easily

Our current recommendation is StarTech.com's PEXUSB3S44V. It provides an independent Renesas (allegedly Linux-friendliest) µPD720202 host controller for each port. Another option from the same maker is PEXUSB400, which also works great but may offer slightly less future proofing.

Our 玄人志向 USB3.0RX4-P4-PCIE cards have been nothing but trouble and we've mostly phased them out by now. Chipset-wise it's pretty much the same thing as StarTech's offering, but the SATA power connector is awfully flimsy and can actually physically break off. The card is also incredibly sensitive to static electricity and will permanently brick itself, which happened on numerous occasions.

USB hubs
Name Score Short explanation
Plugable USB 2.0 7 Port Hub with 60W Power Adapter 8/10 High power output, high reliability
Plugable USB 3.0 7-port Charging Hub with 60W Power Adapter 5/10 High power output, low reliability
System TALKS USB2-HUB4XA-BK USB 2.0 hub with power adapter 7/10 High power output on two ports which complicates device positioning, low port count
Anker USB 3.0 9-Port Hub + 5V 2.1A Charging Port 2/10 High port count, insufficient power
ORICO P10-U2 External ABS 10 Port 2.0 USB HUB for Laptop/Desktop-BLACK 3/10 High port count, insufficient power
ORICO BH4-U3-BK ABS 4 Port USB3.0 BC1.2 Charging HUB with 12V3A Power Adapter-BLACK 5/10 High power output, low reliability

The best hub we've found so far is Plugable's USB 2.0 7 Port Hub with 60W Power Adapter. It's able to provide 1.5A per port for Battery Charging spec compliant devices, which is enough to both charge and sync even tablets (although charging will not occur at maximum speed, but that's irrelevant to us). Note that even devices that are not compliant will usually charge and sync just fine, albeit slower. The more recent USB 3.0 version has proven unreliable with the rest of our components, causing the whole hub to disconnect at times. Annoyingly the ports face the opposite direction, too. Note that ORICO also provides hubs that are identical to Plugable's offerings, the latter of which seem to be rebrands.

Unfortunately Plugable's USB 2.0 hub is not perfect either, at least for our purposes. It includes a physical on/off switch which can be especially annoying if your devices are in a regular office with occasional scheduled power outages. This will shut down the PC too, of course, but the problem is that once power comes back online, the hubs will be unable to switch themselves on and the devices won't charge, leading you to find a bunch of dead devices the next Monday.

The System TALKS USB 2.0 hub is very reliable, but has a few annoying drawbacks. First, the power adapter only provides power to two of its four ports, while the other two are powered by the host PC. The problem with this approach is that you must figure out which devices are power hungry yourself and put them on the ports with higher current. This can complicate device setup/positioning quite a bit. Another drawback is that if the host PC is turned off, only the powered ports will keep charging the connected devices. However, the hub is amazingly compatible with pretty much anything, making it the top choice for older devices that do not support the Battery Charging hubs.

Most powered USB 3.0 hubs we've tested have had a serious problem: the whole hub occasionally disconnected. This may have been caused by the specific combination of our components and/or OS, but as of yet we don't really know. Disabling USB 3.0 may help if you run into the same problem.

Translating

Currently STF UI is available in English and Japanese.

If you would like translate to any other language, please contribute in the STF Transifex project.

For updating the source and all the translation files first you have to install the Transifex client.

Then just run:

gulp translate

It will do the following:

  1. Convert all the jade files to html.
  2. Extract with gettext all translatable strings to stf.pot.
  3. Push stf.pot to Transifex.
  4. Pull from Transifex all po translations.
  5. Compile all po files to json.

Then in order to add it officially (only needs to be done once):

  1. Add the language to res/common/lang/langs.json.
  2. Pull the specific language tx pull -l <lang>.
  3. Run gulp translate.

Testing

See TESTING.md.

Contributing

See CONTRIBUTING.md.

License

See LICENSE.

Copyright © 2017 The OpenSTF Project. All Rights Reserved.