Colab training example

[SkalskiP]

Hello 👋🏻! As requested in https://github.com/SHI-Labs/OneFormer/pull/9 I created dedicated issue. I hope you will get email notifications now.

I started to build Google Colab but I did run into some problems:

ImportError: /usr/local/lib/python3.7/dist-packages/MultiScaleDeformableAttention-1.0-py3.7-linux-x86_64.egg/MultiScaleDeformableAttention.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZNK2at10TensorBase8data_ptrIdEEPT_v

Here is my current version of the notebook: https://colab.research.google.com/drive/1ugQqod5zZLTh9bibOEaflI6QHwkcAM41#scrollTo=Rerjwwk_ZEY_

Do you have any idea how to solve that?

[praeclarumjj3]

Hi @SkalskiP, did you ensure your detectron2 version is compatible with the torch version? You followed the installation instructions as in the official colab demo, right?

[SkalskiP]

Hi @praeclarumjj3 I managed to start training script (looks like all imports are in order) but it fails with error bellow. Btw I'm using this config configs/ade20k/oneformer_R50_bs16_160k.yaml.

FileNotFoundError: [Errno 2] No such file or directory: 'datasets/ADEChallengeData2016/ade20k_panoptic_train.json'

Of course I understand that I don't have ADEChallengeData2016 dataset. I go to Prepare Datasets for OneFormer but it's not super clear for me what should I do now.

I run: !python datasets/prepare_ade20k_sem_seg.py but I get:

FileNotFoundError: [Errno 2] No such file or directory: 'datasets/ADEChallengeData2016/annotations/training'

There is this part Expected dataset structure for [ADE20K](http://sceneparsing.csail.mit.edu/):

ADEChallengeData2016/
  images/
  annotations/
  objectInfo150.txt
  # download instance annotation
  annotations_instance/
  # generated by prepare_ade20k_sem_seg.py
  annotations_detectron2/
  # below are generated by prepare_ade20k_pan_seg.py
  ade20k_panoptic_{train,val}.json
  ade20k_panoptic_{train,val}/
  # below are generated by prepare_ade20k_ins_seg.py
  ade20k_instance_{train,val}.json

But I'm not really sure how should I create that structure. I'd appreciate any guidelines on how to create the dataset. I don't even have ADEChallengeData2016 directory in my datasets, not to mention any content in it. I'm sorry if that seems like a obvious questions.

[praeclarumjj3]

Hi @SkalskiP, you need to download the ADE20K dataset to prepare the given file structure.

• Download ADEChallengedData2016 and unzip it into the $DETECTRON2_DATASETS (by default, it's datasets) directory. You must run export DETECTRON2_DATASETS = /path/to/datasets. If you have timeout issues on colab, it's better to mount google drive and download the dataset there.

  wget http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip

• Follow the instructions in Prepare Datasets for OneFormer to prepare the complete file structure.

[SkalskiP]

Hi, @praeclarumjj3 👋🏻! I'll try to follow your instructions and let you know if I managed to succeed :) Thank you for being so responsive and patient ;)

[praeclarumjj3]

Hi @SkalskiP, I just wanted to check in to see if you were able to resolve the issues :)

[SkalskiP]

Hi, @praeclarumjj3! This week I have a rather busy schedule - I'm working on an extensive video on tracking. I will return to the OneFormer topic next week. I apologize for the delay.

[praeclarumjj3]

Sure, @SkalskiP, no worries. Good luck with the video on tracking ✌🏻

[SkalskiP]

Thank you it is online relight now! I just need to get a bit of rest and I’ll be bank here fighting with custom dataset training tutorial 🔥

[cloverGithub]

ImportError: /usr/local/lib/python3.7/dist-packages/MultiScaleDeformableAttention-1.0-py3.7-linux-x86_64.egg/MultiScaleDeformableAttention.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZNK2at10TensorBase8data_ptrIdEEPT_v

Here is my current version of the notebook:

Hi, i encountered the same issue too. May i know how it was resolved? Thanks.

ImportError: /home/xxx/anaconda3/envs/oneformer/lib/python3.8/site-packages/MultiScaleDeformableAttention-1.0-py3.8-linux-x86_64.egg/MultiScaleDeformableAttention.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNK2at6Tensor7optionsEv

[praeclarumjj3]

Hi @cloverGithub, are you running the colab demo?

[cloverGithub]

Hi @praeclarumjj3 Thanks for reply, apologies for delayed response. Yes, i am trying out the demo.py script.

[praeclarumjj3]

Hi @cloverGithub, this error usually pops up when there are incompatibility issues between your torch cuda build and local cuda versions. Could you confirm if both are compatible?

[praeclarumjj3]

Hi @SkalskiP, this issue has been open for a while. If you are not facing any more issues, I would like to close this.

[SkalskiP]

I understand. I just have so much on my plate. Really sorry for that. I'll reopen it when I'll be back playing with OneFormer.

It is shocking to me that this repo is not getting more recognition. Such a cool model... We need to work on it!

[praeclarumjj3]

Thanks, @SkalskiP. Eager to watch your next video on OneFormer! Again, let me know if you face any issues.

[praeclarumjj3]

I pushed some instructions for training with custom datasets. You may take a look if you face any more issues: https://github.com/SHI-Labs/OneFormer/tree/main/datasets/custom_datasets.

[SkalskiP]

Thank you 🙏🏻 ! I'll take a look for sure :)