upkie / ppo_balancer

Train a balancing policy for Upkie by reinforcement learning
Apache License 2.0
5 stars 3 forks source link

Unable to run `make train_and_show` or `make train` #6

Open itazap opened 4 days ago

itazap commented 4 days ago

After errors leading me to learn that XCode is also a dependency for this project, I was finally able to compile the cpp files/actions when running make train_and_show or make train. However, I am running into spine error: TimeoutError: Spine did not process request within 100.0 ms, is it stopped?

EDIT: I was able to get a few seconds passed the error by hardcoding the timeout timeout_ns = 300000000, and it appears to run for a few seconds, detect a fall, and then reach the error again. Also, there are always 6 simulations that start when running train_and_show (even after force quitting all of them) (image attached). Thanks!

For ref: macos 15.1.1, m3

Error traceback:

Process ForkProcess-6:
Traceback (most recent call last):
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 29, in _worker
    env = _patch_env(env_fn_wrapper.var())
                     ^^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_itazaporozhets/498dbf992c37892e7013f5ba2c923d0f/execroot/ppo_balancer/bazel-out/darwin_arm64-opt/bin/ppo_balancer/train.runfiles/ppo_balancer/ppo_balancer/train.py", line 192, in _init
    velocity_env.reset(seed=seed)
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 75, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 61, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/wrappers/env_checker.py", line 57, in reset
    return env_reset_passive_checker(self.env, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/utils/passive_env_checker.py", line 186, in env_reset_passive_checker
    result = env.reset(**kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/envs/upkie_ground_velocity.py", line 227, in reset
    return super().reset(seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/envs/upkie_base_env.py", line 188, in reset
    spine_observation = self._spine.start(self._spine_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/spine/spine_interface.py", line 94, in start
    self._wait_for_spine()
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/spine/spine_interface.py", line 146, in _wait_for_spine
    raise TimeoutError(
TimeoutError: Spine did not process request within 100.0 ms, is it stopped?
[2024-11-23 14:14:04.641] [warning] CPU temperature observation disabled: file not found
Process ForkProcess-5:
Traceback (most recent call last):
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 29, in _worker
    env = _patch_env(env_fn_wrapper.var())
                     ^^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_itazaporozhets/498dbf992c37892e7013f5ba2c923d0f/execroot/ppo_balancer/bazel-out/darwin_arm64-opt/bin/ppo_balancer/train.runfiles/ppo_balancer/ppo_balancer/train.py", line 192, in _init
    velocity_env.reset(seed=seed)
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 75, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 61, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/wrappers/env_checker.py", line 57, in reset
    return env_reset_passive_checker(self.env, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/gymnasium/utils/passive_env_checker.py", line 186, in env_reset_passive_checker
    result = env.reset(**kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/envs/upkie_ground_velocity.py", line 227, in reset
    return super().reset(seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/envs/upkie_base_env.py", line 188, in reset
    spine_observation = self._spine.st[2024-11-23 14:14:04.654] [warning] CPU temperature observation disabled: file not found
art(self._spine_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/spine/spine_interface.py", line 94, in start
    self._wait_for_spine()
  File "/opt/anaconda3/envs/ppo_balancer/lib/python3.12/site-packages/upkie/spine/spine_interface.py", line 146, in _wait_for_spine
    raise TimeoutError(
TimeoutError: Spine did not process request within 100.0 ms, is it stopped?
Screenshot 2024-11-23 at 2 19 08 PM
stephane-caron commented 2 days ago

The default value of six parallel environments is probably too large here. You can adjust this number from the command line:

make train_and_show NB_TRAINING_ENVS=2

You can also save the change permanently by editing the value of NB_TRAINING_ENVS in the Makefile.