How to write test that handles USB disconnect and reconnect properly

[sebkur]

Thanks for this library, it seems very useful!

I've got a lot of basic things working after going through the tutorials, but now I'm stuck with making my tests continue working even when disconnecting one of the devices.

Here's my code:

from mobly import base_test
from mobly import test_runner
from mobly.controllers import android_device
import time

class PublicMeshTest(base_test.BaseTestClass):

  def setup_class(self):
    # Registering android_device controller module declares the test's
    # dependency on Android device hardware. By default, we expect at least one
    # object is created from this, here we expect at least 2 devices.
    self.ads = self.register_controller(android_device, min_number=2)
    self.droid1 = self.ads[0]
    self.droid2 = self.ads[1]
    self.droid1.load_snippet('pm', 'org.briarproject.publicmesh')
    self.droid2.load_snippet('pm', 'org.briarproject.publicmesh')

  def test_basic_discovery(self):
    self.droid1.pm.startActivity()
    self.droid2.pm.startActivity()
    self.droid1.pm.startAdvertising()
    self.droid2.pm.startDiscovery()
    time.sleep(30)

if __name__ == '__main__':
  test_runner.main()

It loads a snippet on two devices, starts some activity and make it do something. Then I wait for 30 seconds in which I disconnect one of the devices from USB and reconnect it. Afterwards, Mobly succeeds to finish the activity on the device that was connected all along but fails to do so on the one that had been temporarily disconnected.

I get some errors on stdout:

$ python3 public_mesh_test.py -c public_mesh.yml
[PublicMeshTestBed] 11-25 15:00:26.414 INFO ==========> PublicMeshTest <==========
[PublicMeshTestBed] 11-25 15:00:26.781 INFO [AndroidDevice|PT99651AA1AC1803610] Initializing the snippet package org.briarproject.publicmesh.
[PublicMeshTestBed] 11-25 15:00:37.867 INFO [AndroidDevice|ZY32BSN89S] Initializing the snippet package org.briarproject.publicmesh.
[PublicMeshTestBed] 11-25 15:00:39.376 INFO [Test] test_basic_discovery
[PublicMeshTestBed] 11-25 15:00:49.618 INFO [Test] test_basic_discovery PASS
tearing down
[PublicMeshTestBed] 11-25 15:00:57.101 ERROR Failed to stop service "snippets".: <AndroidDevice|PT99651AA1AC1803610> Failed to stop existing apk. Unexpected output: .
Traceback (most recent call last):
  File "~/.local/lib/python3.8/site-packages/mobly/expects.py", line 152, in expect_no_raises
    yield
  File "~/.local/lib/python3.8/site-packages/mobly/controllers/android_device_lib/service_manager.py", line 204, in stop_all
    service.stop()
  File "~/.local/lib/python3.8/site-packages/mobly/controllers/android_device_lib/services/snippet_management_service.py", line 116, in stop
    client.stop()
  File "~/.local/lib/python3.8/site-packages/mobly/controllers/android_device_lib/snippet_client_v2.py", line 575, in stop
    self._stop_server()
  File "~/.local/lib/python3.8/site-packages/mobly/controllers/android_device_lib/snippet_client_v2.py", line 623, in _stop_server
    raise android_device_lib_errors.DeviceError(
mobly.controllers.android_device_lib.errors.DeviceError: <AndroidDevice|PT99651AA1AC1803610> Failed to stop existing apk. Unexpected output: .
[PublicMeshTestBed] 11-25 15:00:59.141 INFO Summary for test class PublicMeshTest: Error 1, Executed 1, Failed 0, Passed 1, Requested 1, Skipped 0
[PublicMeshTestBed] 11-25 15:00:59.142 INFO Summary for test run PublicMeshTestBed@11-25-2022_15-00-26-412:
Total time elapsed 32.72900233200926s
Artifacts are saved in "~/gitlab/briar/public-mesh-testbed/mobly/mesh-run/logs/PublicMeshTestBed/11-25-2022_15-00-26-412"
Test results: Error 1, Executed 1, Failed 0, Passed 1, Requested 1, Skipped 0

I also tried to wait for the devices at the end of the test, but that didn't help, like this:

    …
    time.sleep(30)
    print("wait for device 1")
    self.droid1.adb.wait_for_device()
    print("wait for device 2")
    self.droid2.adb.wait_for_device()

I also tried various things with handle_usb_disconnect() but also couldn't make it fix anything.

Do you have any idea how to solve this? I have the feeling Mobly is designed to handle this kind of situation and I'm missing something obvious. I think an example on this would be beneficial for the community as disconnecting devices during a test and reconnecting it for logs collection seems to me like something that could be a very common situation. Thanks!

[xpconanfan]

Use handle_usb_disconnect as a context, and perform the disconnect/reconnect within the context.

[sebkur]

Thanks! I'm not sure I've understood this correctly, but this is what I figured out now and which seems to be working:

  def test_basic_discovery(self):                                                                                                                                      
    self.droid1.pm.startActivity()                                                                                                                                     
    self.droid2.pm.startActivity()                                                                                                                                     
    self.droid1.pm.startAdvertising()                                                                                                                                  
    self.droid2.pm.startDiscovery()                                                                                                                                    
    with self.droid1.handle_usb_disconnect():                                                                                                                          
      with self.droid2.handle_usb_disconnect():                                                                                                                        
        maxWaitDisconnect = 10                                                                                                                                         
        print("waiting for disconnect for", maxWaitDisconnect, "seconds")                                                                                              
        time.sleep(maxWaitDisconnect)                                                                                                                                  
        print("wait for device 1")                                                                                                                                     
        self.droid1.adb.wait_for_device()                                                                                                                              
        print("wait for device 2")                                                                                                                                     
        self.droid2.adb.wait_for_device()

I'm giving the test user 10 seconds time to disconnect one or more of the devices running the tests and the tests will continue running on-device until the user reconnects all previously disconnected devices. This fits our scenario of what we're trying to do, i.e. let some app run for a while on the devices until they're all plugged back into the host where logs are collected.

[sebkur]

I'm not sure how this would scale to a variable number of devices, i.e. if one wanted to write this code so that it would not operate on exactly two devices but for example on a list of devices. Putting the with self.droid1.handle_usb_disconnect(): into a loop probably won't work... It's not crucial for me, just curious if this is generally possible and also if the way I'm doing it now seems reasonable.

I think it would probably be great to add an example to the Mobly repository that does something like this. It doesn't seem straightforward to figure this out and there also doesn't seem to be any real-world code that uses handle_usb_disconnect(). Try a Github code search: https://github.com/search?q=handle_usb_disconnect&type=code it only turns up occurrences in mobly itself where the method is defined or in forks.