search

ml-jku / L2M

Learning to Modulate pre-trained Models in RL (Decision Transformer, LoRA, Fine-tuning)
MIT License
50 stars 5 forks source link

env_param config for debugging and any tips #5

Closed Abc11c closed 1 month ago

Abc11c commented 3 months ago

Hi @thomasschmied,

Thank you for releasing the code! I want to quickly benchmark some of the metaworld v2 environments but stuck at mujuco setup I have the following doubts and would be grateful for any advice

  1. The data paths, I assume the links on the readme point to these names in the config dm_control_1M_v3_full and metaworld_v2_cwnet_2M in the config, so just renaming the data subfolder should be good?
  2. Can you provide a config for pretraining runs, 1 seed, 1 env to debug since loading all the trajectories tend to take time?
  3. [Resolved] What version of mujoco to use? I had some issues with installing mujoco-py==2.0.2.13 as seen in the requirements.txt so had to downgrade to 2.0,2.8, and use mujoco version.

I get the following error when I try to run the pretrain configs

python main.py -m experiment_name=pretrain seed=42 env_params=multi_domain_mtdmc run_params=pretrain eval_params=pretrain_disc agent_params=cdt_pretrain_d
isc agent_params.kind=MDDT agent_params/model_kwargs=multi_domain_mtdmc agent_params/data_paths=mt40v2_dmc10 +agent_params/replay_buffer_kwargs=multi_domain_mtdmc +agent_params.accumulation_steps=2
  File "/home/a9karthi/.conda/envs/mddt/lib/python3.9/site-packages/dm_control/mujoco/index.py", line 623, in struct_indexer
    attr = getattr(struct, field_name)
AttributeError: 'MjData' object has no attribute 'qacc_unc'

It was caused by mujoco version issue downgrading it to mujoco==2.1.5 fixed it.

Thanks!

thomasschmied commented 2 months ago

Hi @Abc11c,

Regarding you questions:

  1. Yes, renaming the data subfolders should do it. Make sure that the data dirs are set correctly, as described here: https://github.com/ml-jku/L2M?tab=readme-ov-file#experiment-configuration
  2. Do I understand it correctly, that you want to pre-train and evaluate on a single env only? In this case, you could for example set env_params.eval_env_names=reach-v2 and use the following agent_params/data_paths config: 
    mt40_v2: 
base: ${DATA_DIR}/your_metaworld_dir
names: 
- reach-v2.pkl
  3. Thanks for posting your solution here.

I hope this helps!