search

octo-models / octo

Octo is a transformer-based robot policy trained on a diverse mix of 800k robot trajectories.
https://octo-models.github.io/
MIT License
858 stars 162 forks source link

Secondary image Tokenizer #115

Open HM102 opened 4 months ago

HM102 commented 4 months ago

Is the image tokenizer for the secondary image already created, or should we create it in the finetune file:

config["model"]["observation_tokenizers"]["top"] = ModuleSpec.create(
        ImageTokenizer,
        obs_stack_keys=["secondary"],
        task_stack_keys=["secondary"],
        encoder=ModuleSpec.create(SmallStem16),
    )
gaokai15 commented 2 months ago

Not sure the specific model that you download, I think you need to create it by default. You can search for "secondary" in the config.json in the model folder. They are there if the model have them created.