Test-tools improvement

[tsampazk]

Attempting to manipulate github actions to install the toolkit once and then use this single installation to run all the tool tests in parallel, instead of reinstalling the whole toolkit in each separate VM for every tool.

Update: caching the venv partially works speeding up the whole procedure. It seems that some system dependencies are missing, will investigate further.

Update: Progressing well, need to reinstall some system dependencies for each tool which is a really fast process. Many tools also require retinaface's make to run or else they get some Cython errors. End to end planning and multi object search still get some errors, presumably because of ROS dependencies etc, investigating further.

Update: Latest issues were solved, now new speech transcription tools are broken, attempting fixes.

Update: All tests pass successfully, now trying out various scenarios to see if caching behaves as expected for normal use.

[tsampazk]

Hey @sycz00, i've been having some issues with multi object search in this PR. When running the tests, after downloading some stuff i get:

2023-08-29 09:40:21 (42.6 MB/s) - ‘/tmp/Rs.tar.gz’ saved [19887264/19887264]

INFO: Probing, EGL cannot run on this device
BDDL could not be imported - object taxonomy / abilities will be unavailable.
E
======================================================================
ERROR: setUpClass (test_multi_object_search.MultiObjectSearchTest)

From these lines, "INFO: Probing, EGL cannot run on this device" does not appear on the regular tests and then the tests run fine. On the updated tests of this PR where installation is split from running the tests, it fails afterwards with the following error:

Traceback (most recent call last):
  File "/home/runner/work/opendr/opendr/tests/sources/tools/control/multi_object_search/test_multi_object_search.py", line 64, in setUpClass
    cls.env = MultiObjectEnv(config_file=EVAL_CONFIG_FILE, scene_id="Rs")
  File "/home/runner/work/opendr/opendr/src/opendr/control/multi_object_search/algorithm/igibson/multi_object_env.py", line 65, in __init__
    super(MultiObjectEnv, self).__init__(
  File "/home/runner/work/opendr/opendr/src/opendr/control/multi_object_search/algorithm/igibson/env_functions.py", line 78, in __init__
    self.simulator = Sim(
  File "/home/runner/work/opendr/opendr/src/opendr/control/multi_object_search/algorithm/igibson/sim.py", line 51, in __init__
    super(Sim, self).__init__(
  File "/home/runner/work/opendr/opendr/venv/lib/python3.8/site-packages/igibson/simulator.py", line 161, in __init__
    self.load()
  File "/home/runner/work/opendr/opendr/venv/lib/python3.8/site-packages/igibson/simulator.py", line 231, in load
    self.renderer = MeshRenderer(
  File "/home/runner/work/opendr/opendr/venv/lib/python3.8/site-packages/igibson/render/mesh_renderer/mesh_renderer_cpu.py", line 107, in __init__
    available_devices, _ = get_available_devices()
ValueError: not enough values to unpack (expected 2, got 1)

Do you have any ideas on why this is caused? I think it might be because of some missing system installation or something very minor that i am not seeing.

Edit: here you can find the full logs of one of the failed runs.

[tsampazk]

Hey @sycz00, i've been having some issues with multi object search in this PR. When running the tests, after downloading some stuff i get:

2023-08-29 09:40:21 (42.6 MB/s) - ‘/tmp/Rs.tar.gz’ saved [19887264/19887264]

INFO: Probing, EGL cannot run on this device
BDDL could not be imported - object taxonomy / abilities will be unavailable.
E
======================================================================
ERROR: setUpClass (test_multi_object_search.MultiObjectSearchTest)

From these lines, "INFO: Probing, EGL cannot run on this device" does not appear on the regular tests and then the tests run fine. On the updated tests of this PR where installation is split from running the tests, it fails afterwards with the following error:

Traceback (most recent call last):
  File "/home/runner/work/opendr/opendr/tests/sources/tools/control/multi_object_search/test_multi_object_search.py", line 64, in setUpClass
    cls.env = MultiObjectEnv(config_file=EVAL_CONFIG_FILE, scene_id="Rs")
  File "/home/runner/work/opendr/opendr/src/opendr/control/multi_object_search/algorithm/igibson/multi_object_env.py", line 65, in __init__
    super(MultiObjectEnv, self).__init__(
  File "/home/runner/work/opendr/opendr/src/opendr/control/multi_object_search/algorithm/igibson/env_functions.py", line 78, in __init__
    self.simulator = Sim(
  File "/home/runner/work/opendr/opendr/src/opendr/control/multi_object_search/algorithm/igibson/sim.py", line 51, in __init__
    super(Sim, self).__init__(
  File "/home/runner/work/opendr/opendr/venv/lib/python3.8/site-packages/igibson/simulator.py", line 161, in __init__
    self.load()
  File "/home/runner/work/opendr/opendr/venv/lib/python3.8/site-packages/igibson/simulator.py", line 231, in load
    self.renderer = MeshRenderer(
  File "/home/runner/work/opendr/opendr/venv/lib/python3.8/site-packages/igibson/render/mesh_renderer/mesh_renderer_cpu.py", line 107, in __init__
    available_devices, _ = get_available_devices()
ValueError: not enough values to unpack (expected 2, got 1)

Do you have any ideas on why this is caused? I think it might be because of some missing system installation or something very minor that i am not seeing.

Edit: here you can find the full logs of one of the failed runs.

I managed to solve it by running sudo apt install libegl1-mesa-dev. It seems that this package is installed as a dependency as i was not able to find it being installed explicitly anywhere in the toolkit.

[tsampazk]

New workflow should be good to go for normal usage, from my end it seems to be stable.

Toolkit gets installed once and saved to github cache and then loaded for each separate tool. If test tools had been run recently (within last week) and no changes have been made to dependencies, requirements or install.sh, the previous installation is loaded and installation is skipped altogether. If anything goes wrong during cache loading, e.g. timeout on download which is a github issue, toolkit is reinstalled and tests run normally.

Github cache (Actions->cache on github top menu) doesn't seem to require any intervention in regular tests use cases and should be managed automatically.