search

mfinzi / residual-pathway-priors

BSD 2-Clause "Simplified" License
17 stars 0 forks source link

Humanoid-v2 task of RL experiment doesn't work. #7

Open kim-hyunsu opened 2 years ago

kim-hyunsu commented 2 years ago

Hi, I tried the following command to test the Humanoid-v2 task python train_rpp.py --env_name=Humanoid-v2 --save_dir=./tmp/rpp --rpp_value=False However, it didn't work and raised those error messages

Warning: Flow failed to import. Set the environment variable D4RL_SUPPRESS_IMPORT_ERROR=1 to suppress this message.
No module named 'flow'
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/glfw/__init__.py:906: GLFWError: (65544) b'X11: The DISPLAY environment variable is missing'
  warnings.warn(message, GLFWError)
Warning: CARLA failed to import. Set the environment variable D4RL_SUPPRESS_IMPORT_ERROR=1 to suppress this message.
No module named 'carla'
pybullet build time: Apr 26 2022 03:12:14
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/jax/_src/numpy/lax_numpy.py:5406: UserWarning: 'kind' argument to argsort is ignored; only 'stable' sorts are supported.
  warnings.warn("'kind' argument to argsort is ignored; only 'stable' sorts "
CWD =  /workspace/siml/hyunsu/temp/RL
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/gym/envs/registration.py:505: UserWarning: WARN: The environment Humanoid-v2 is out of date. You should consider upgrading to version `v3` with the environment ID `Humanoid-v3`.
  logger.warn(
/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/gym/core.py:172: DeprecationWarning: WARN: Function `env.seed(seed)` is marked as deprecated and will be removed in the future. Please use `env.reset(seed=seed) instead.
  deprecation(
tau =  0.005
rpp policy is True
Traceback (most recent call last):
  File "train_rpp.py", line 187, in <module>
    app.run(main)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "train_rpp.py", line 105, in main
    agent = SACLearner(FLAGS.seed,
  File "/workspace/siml/hyunsu/temp/RL/jax_rl/agents/sac/sac_learner.py", line 126, in __init__
    actor_def = policies.RPPNormalTanhPolicy(state_rep,action_rep,action_std_rep,symmetry_group,ch,
  File "/workspace/siml/hyunsu/temp/RL/jax_rl/networks/policies.py", line 25, in RPPNormalTanhPolicy
    state_rep = state_rep(G)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/emlp/reps/product_sum_reps.py", line 69, in __call__
    return SumRepFromCollection({rep(G):c for rep,c in self.reps.items()},perm=self.perm)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/emlp/reps/product_sum_reps.py", line 69, in <dictcomp>
    return SumRepFromCollection({rep(G):c for rep,c in self.reps.items()},perm=self.perm)
  File "/mnt/gsai/hyunsu/residual-pathway-priors/.venv/lib/python3.8/site-packages/emlp/reps/representation.py", line 49, in __call__
    raise NotImplementedError
NotImplementedError

The error doesn't look an issue of wrong package versions. Could you help me to resolve it? For reference, the only two things I modified in the code were

  1. Converting every "@jax.partial" to "@partial" of functools.
  2. Revising the PseudoScalar class from
class PseudoScalar(Rep):
    is_regular=False
    def __init__(self,G=None):
        self.G=G
        self.concrete = (self.G is not None)

to

class PseudoScalar(Rep):
    is_regular=False
    def __init__(self,G=None):
        self.G=G
        # self.concrete = (self.G is not None)
    @property
    def concrete(self):
        return self.G is not None

as in the Rep class, and here are the versions of related packages

mfinzi commented 2 years ago

Hi @kim-hyunsu, Thanks for bringing this to our attention, and I'm very sorry you had this issue! I suspect we may have been relying on an undocumented feature of emlp 1.0.0 that was removed. I'll investigate tomorrow (and look into making an update to the RPP repo not to do it this way) but until then try setting emlp==1.0.0 using the original code.

mfinzi commented 2 years ago

Actually come to think of it, it must have been 0.9 or earlier. I'll investigate tomorrow