Loading custom local dataset: DatasetNotFoundError

[emfebert]

Hi,

I created my own custom dataset by using the rlds dataset builder repo, and I'm trying to finetune on this data using python scripts/finetune.py --config.pretrained_path=hf://rail-berkeley/octo-small

I modified the finetuning kwargs as follows:

FINETUNING_KWARGS = {
        "name": "act_dataset",
        "data_dir": "/home/febert/tensorflow_datasets/act_dataset/1.0.0",
        "image_obs_keys": {"primary": "image_0", "wrist": None},
        "state_obs_keys": ["state", None],
        "language_key": "language_instruction",
        "action_proprio_normalization_type": "normal",
        "absolute_action_mask": [False, False, False, True, True, True, True, True, True, True],
        "standardize_fn": "octo/data/oxe/oxe_standardization_transforms.py:act_dataset_transform",
    }

I'm getting the following error:

tensorflow_datasets.core.registered.DatasetNotFoundError: Dataset act_dataset not found.
Available datasets:
    - abstract_reasoning
    - accentdb

The dataset is located on my local machine here: "/home/febert/tensorflow_datasets/act_dataset/1.0.0"

[kpertsch]

Try setting the data_dir to /home/febert/tensorflow_datasets

[emfebert]

Thanks that helped!