Installation about `og_marl`

[fyqqyf]

When executing pip install -r ./third_party/og_marl/requirements.txt, an error occurs indicating 'No such file or directory: './third_party/og_marl/requirements.txt'.

[zbzhu99]

Try the updated command (https://github.com/zbzhu99/madiff/commit/ddef3ab57a6e149f97b9dcb4136042dbf290516c):

pip install -r ./third_party/og-marl/install_environments/requirements/smacv1.txt

[fyqqyf]

@zbzhu99 Thank you for your prompt response! However, I encountered an issue when executing the command:

python scripts/transform_og_marl_dataset.py --env_name mamujoco --map_name 2halfcheetah --quality Good

The error message I received is as follows:

File "scripts/transform_og_marl_dataset.py", line 8, in <module>
    from og_marl.environments import smac, mamujoco
ImportError: cannot import name 'smac' from 'og_marl.environments'

Besides, it appears that there might be some problems with the transform_og_marl_dataset.py script.

[zbzhu99]

Sorry for the mismatch between og-marl package and transformation script. Please pull the latest commit and reinstall the local og-marl package, which has the version of 0.0.1. This should work fine with current script and data.

(and I will provide links for downloading transformed dataset in the next week 😊

[fyqqyf]

Thank you for your continued attention to this matter. Unfortunately, the error persists. Perhaps it would be advisable for me to consider waiting until next week's datasets are available ;)

Traceback (most recent call last):
  File "scripts/transform_og_marl_dataset.py", line 217, in <module>
    main(args.env_name, args.map_name, args.quality)
  File "scripts/transform_og_marl_dataset.py", line 28, in main
    filenames = sorted(filenames, key=get_fname_idx)
  File "scripts/transform_og_marl_dataset.py", line 24, in get_fname_idx
    dir_idx = sub_dir_to_idx[file_name.split("/")[-5]] * 1000
KeyError: 'mamujoco'

[zbzhu99]

Oops, I only tested the transformation of smac datasets. The problem on mamujoco datasets can be fixed with a minor change: https://github.com/zbzhu99/madiff/commit/bd0c5b7ab834b7b8c1912653ae06f9c7dc989ccf

[fyqqyf]

@zbzhu99 Maybe it is below? (I use file_name.split("/")[-3] and file_name.split("/")[-5] in the latest version but the error persists)

if env_name == "smac":
    dir_idx = sub_dir_to_idx[file_name.split("/")[-2]] * 1000
    return dir_idx + int(file_name.split("log_")[-1].split(".")[0])
elif env_name == "mamujoco":
    dir_idx = sub_dir_to_idx[file_name.split("/")[-2]] * 1000
    return dir_idx + int(file_name.split("/")[-1].split("log_")[-1].split(".")[0]) 
else:
    raise ValueError(f"Unknown environment {env_name}")

This is my tree of data folder:

• data
  • mamujoco
    • 2 halfcheetah
      • Good
        • 1
        • 2
        • 3
        • 4

[zbzhu99]

It seems the developers of og-marl updated the data in their download links. In my local folder downloaded before, the files look like this:

I am not sure whether they just rearrange the folders or provide a different dataset. I will take a look and upload my local copy if necessary.

[zbzhu99]

@fyqqyf , I have checked og-marl's new dataset and found they just renamed and rearranged the log folders from different tasks to a unified structure.

Actually, I heard from the maintainer of og-marl dataset that they are also implementing the transformation (concatenate segments to complete trajectories) I have done here. So, the change of file structure may be to make it easier to process all datasets with the same script.

I have modified the script scripts/transform_og_marl_dataset.py and it should work fine with all new datasets now. Still, I will provide links for downloading transformed datasets by the end of this week.

Happy Women's Day!