If you want to make it work for new objects/task categories you will need to build your own OSTOG_physical_experiments.json as the one-shot learning models use it for references. Depending on how comfortable you are with transforming masks you can use MakeSense.AI or CVAT for labeling your own images, which are both free to use.
CVAT can export your object/affordance annotations as binary mask images but has a bit more setup
MakeSense.AI requires no setup but you may need to convert its exported polygon annotations into binary masks yourself as done when training the segmentation-based models.
If you want to make it work for new objects/task categories you will need to build your own
OSTOG_physical_experiments.jsonas the one-shot learning models use it for references. Depending on how comfortable you are with transforming masks you can use MakeSense.AI or CVAT for labeling your own images, which are both free to use.