[Question] ValueError: Loaded data does not have the required field "init_obs" for episodic data

[akifumi-wachi-4]

Required prerequisites

• [X] I have read the documentation https://omnisafe.readthedocs.io.
• [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• [X] Consider asking first in a Discussion.

Questions

I tried to follow the README in https://github.com/PKU-Alignment/omnisafe/tree/main/benchmarks/offline, but encountered an issue saying that ValueError: Loaded data does not have the required field "init_obs" for episodic data. when executing the command for https://github.com/PKU-Alignment/omnisafe/tree/main/benchmarks/offline#run-the-benchmark. Could you tell me how to resolve this issue?

[Gaiejj]

Are you using an existing dataset or collecting a new dataset using an actor to run the offline algorithm? I tried collecting an offline dataset locally and then running run_experiment_grid.py, and the training was successful. Please provide more detailed error information and descriptions so that we can help resolve your issue.

[akifumi-wachi-4]

Thank you for the response. I tried exactly the same process that is written in https://github.com/PKU-Alignment/omnisafe/tree/main/benchmarks/offline.

1. Training agents used to generate data
   • The command I used is omnisafe train --env-id SafetyAntVelocity-v1 --algo PPOLag
2. Collect offline data
   • The Python script I used is (almost) the same as in https://github.com/PKU-Alignment/omnisafe/tree/main/benchmarks/offline#collect-offline-data.
3. Run the Benchmark

   
   import warnings

import torch

from omnisafe.common.experiment_grid import ExperimentGrid from omnisafe.utils.exp_grid_tools import train

if name == 'main': eg = ExperimentGrid(exp_name='Offline-Benchmarks')

# set up the algorithms.
offline_policy = ['VAEBC', 'BCQ', 'BCQLag', 'CCR', 'CCRR', 'COptiDICE']

eg.add('algo', offline_policy)

# you can use wandb to monitor the experiment.
eg.add('logger_cfgs:use_wandb', [False])
# you can use tensorboard to monitor the experiment.
eg.add('logger_cfgs:use_tensorboard', [True])
# add dataset path
eg.add('train_cfgs:dataset', [dataset_path])

# set up the environment.
eg.add('env_id', [
    'SafetyAntVelocity-v1',
    ])
eg.add('seed', [0, 5, 10, 15, 20])

# total experiment num must can be divided by num_pool
# meanwhile, users should decide this value according to their machine
eg.run(train, num_pool=5)

In Step 3, I encountered the error saying `ValueError: Loaded data does not have the required field "init_obs" for episodic data.` I looked into the dataset (i.e., npz file), but there is no field of "init_obs".

OmniSafe version is the latest (0.5.0).

[akifumi-wachi-4]

We looked into this error and found that this error is specific to COptDICE implementations. In the COptDICE implementation, init_obs is used as an input of NNs. For example,

• https://github.com/PKU-Alignment/omnisafe/blob/080e6b828c65902611e86e7436dc47c674cdafd7/omnisafe/algorithms/offline/coptidice.py#L169

However, the init_obs field is not in the offline dataset, which leads to the error. Could you tell me how to resolve this?

[Gaiejj]

Thank you very much for raising this issue. We have found that this occurs because the import of offline datasets does not support assigning episode_length for user-collected data. Currently, you can manually modify the code to accomplish this. We will fix this in future updates.

For example, if you collected 2 million steps of data on SafetyPointCircle1-v0, and since this task corresponds to 1,000 steps per episode, you have a total of 2,000 episodes of data. Therefore, please set episode_length to 2,000.

episode_length = 2000 # line 328 of omnisafe/common/offline/dataset.py

[akifumi-wachi-4]

Thank you!! I could successfully run the whole process for online-to-offline RL training!