SAC - can't pickle _thread.RLock objects

[cambel]

I created a Gym environment for a robot in ROS-Gazebo. Now, I want to train it using SAC.

I am trying to implement the SAC agent based on the example but I get the following error

Traceback (most recent call last): File "/home/cambel/ws_ur5/src/ur3/ur3e_openai/scripts/sac.py", line 135, in experiment(variant) File "/home/cambel/ws_ur5/src/ur3/ur3e_openai/scripts/sac.py", line 102, in experiment algorithm.train() File "/home/cambel/dev/rlkit/rlkit/core/rl_algorithm.py", line 46, in train self._train() File "/home/cambel/dev/rlkit/rlkit/core/batch_rl_algorithm.py", line 84, in _train self._end_epoch(epoch) File "/home/cambel/dev/rlkit/rlkit/core/rl_algorithm.py", line 56, in _end_epoch logger.save_itr_params(epoch, snapshot) File "/home/cambel/dev/rlkit/rlkit/core/logging.py", line 288, in save_itr_params torch.save(params, file_name) File "/home/cambel/.pyenv/versions/py3/lib/python3.6/site-packages/torch/serialization.py", line 224, in save return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol)) File "/home/cambel/.pyenv/versions/py3/lib/python3.6/site-packages/torch/serialization.py", line 149, in _with_file_like return body(f) File "/home/cambel/.pyenv/versions/py3/lib/python3.6/site-packages/torch/serialization.py", line 224, in return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol)) File "/home/cambel/.pyenv/versions/py3/lib/python3.6/site-packages/torch/serialization.py", line 296, in _save pickler.dump(obj) TypeError: can't pickle _thread.RLock objects

I'm working on Ubuntu 16.04. I was using pytorch v0.4.1, then I also tried with the lasted version of pytorch but got the same results.

I am not using images, the state of the robot is just an array of the position of it's end-effector.

Has anyone had this same problem?

[vitchyr]

It looks like your environment can't be pickled. I recommend removing the env from the snapshot that is being saved.

See rlkit/core/rl_algorithm.py, line 56

[cambel]

@vitchyr Thanks for your reply.

I removed the env from the snapshot as you suggest but then I got a different error

ephoc... 0 2019-09-27 14:23:38.024797 JST | [name-of-experiment_2019_09_27_14_22_13_0000--s-0] Epoch 0 finished [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {'TimeLimit.truncated': True}] Traceback (most recent call last): File "/home/cambel/ws_ur5/src/ur3/ur3e_openai/scripts/sac.py", line 129, in experiment(variant) File "/home/cambel/ws_ur5/src/ur3/ur3e_openai/scripts/sac.py", line 96, in experiment algorithm.train() File "/home/cambel/dev/rlkit/rlkit/core/rl_algorithm.py", line 46, in train self._train() File "/home/cambel/dev/rlkit/rlkit/core/batch_rl_algorithm.py", line 84, in _train self._end_epoch(epoch) File "/home/cambel/dev/rlkit/rlkit/core/rl_algorithm.py", line 62, in _end_epoch self._log_stats(epoch) File "/home/cambel/dev/rlkit/rlkit/core/rl_algorithm.py", line 114, in _log_stats eval_util.get_generic_path_information(expl_paths), File "/home/cambel/dev/rlkit/rlkit/core/eval_util.py", line 40, in get_generic_path_information for p in paths File "/home/cambel/dev/rlkit/rlkit/core/eval_util.py", line 40, in for p in paths File "/home/cambel/dev/rlkit/rlkit/pythonplusplus.py", line 166, in list_of_dictstodict_of_lists assert set(d.keys()) == set(keys) AssertionError

The epoch is empty, do you know why that might be?

[vitchyr]

You need to make sure the environment adheres to the MultitaskEnv interface. See https://github.com/vitchyr/multiworld/