Manual training errors on missing data directory

[peterklipfel]

I am trying to run the training of the model manually (to identify OOM issues for custom training), but when I follow the steps in the readme, I get an error when looking for images in a folder that doesn't exist. There are several possible issues here, so I'll try to detail them independently.

Steps to reproduce:

1. clone repository and install conda environment>
2. Update src/configs/train_config.yml and src/configs/eval_config.yml with my pytorch data directory
3. run python download_datasets.py
4. cd $PYTORCH_DATA_DIR
5. unzip all downloads (cocostuff.zip, cityscapes.zip, potsdam.zip, potsdamraw.zip)
6. cd $STEGO_DIR/src
7. python precompute_knns.py
8. Error

Since no pretrained weights have been provided, we load the reference pretrained DINO weights.
/home/peter/datasets/nns/nns_vit_small_cocostuff27_val_five_224.npz not found, computing
Error executing job with overrides: []
Traceback (most recent call last):
  File "precompute_knns.py", line 78, in my_app
    cfg=cfg,
  File "/home/peter/code/vendor/STEGO/src/data.py", line 495, in __init__
    target_transform=target_transform, **extra_args)
  File "/home/peter/code/vendor/STEGO/src/data.py", line 380, in __init__
    self.num_images = len(os.listdir(self.img_dir))
FileNotFoundError: [Errno 2] No such file or directory: '/home/peter/datasets/cropped/cocostuff27_five_crop_0.5/img/val'

I get no results when I run:

find $PYTORCH_DATA_DIR -type d -name "*crop*"

Other things that I don't understand that may be related:

• In the training config, I see a couple references to "cocostuff27". I am unable to find this directory anywhere in the provided data.
• I would expect a dataset prefix in the dataset directory. That is, the code wants /home/peter/datasets/cropped/cocostuff27_five_crop_0.5/img/val, but I would expect /home/peter/datasets/SOME_DATASET_NAME/cropped/cocostuff27_five_crop_0.5/img/val
• I see no references to anything containing 27, but there are several references to 2017. Perhaps there is a typo here and the code just needs to be updated to 2017 (i.e. train_2017 and val_2017?

[KennyChen880127]

Befor you execute the precompute_knn.py, You should execute the crop_datasets.py and then it will auto create the floder. Path is cropped\cocostuff27_five_crop_0.5.

[peterklipfel]

Thank you very much @KennyChen880127! I've submitted a pull request to document this in the readme. Feel free to close after accepting/rejecting the PR

[mhamilton723]

Thank you for submitting the PR! I have just merged it in (Also sorry for the delay in getting back to all the issues!). Closing this out for now