search

SHI-Labs / OneFormer

OneFormer: One Transformer to Rule Universal Image Segmentation, arxiv 2022 / CVPR 2023
https://praeclarumjj3.github.io/oneformer
MIT License
1.44k stars 130 forks source link

Adding "OneFormer - One Segmentation Model To Rule Them ALL" YouTube video to README.md. #9

Closed SkalskiP closed 1 year ago

SkalskiP commented 1 year ago

Hello 👋!

I absolutely love your project! We released a YouTube video covering OneFormer today. We walk viewers through your demo notebook, and then use OneFormer to build a simple script that calculates the size of objects in real life. I thought it might be a good idea to add it to README.md.

We are considering recording another video showing how to do the training on custom dataset, but it seems we would need your help to do it.

Let me know what do you think.

praeclarumjj3 commented 1 year ago

Hi @SkalskiP, thanks for covering OneFormer on youtube! I loved the video.

We are considering recording another video showing how to do the training on custom dataset, but it seems we would need your help to do it.

Sure, let me know how I can help you.

SkalskiP commented 1 year ago

Hi @praeclarumjj3 👋🏻! I really had a lot of fun while recording. I'm blown away by predictions quality.

Sure, let me know how I can help you.

That would be awesome. :tada: We already see in the comments bellow the video that people would be interested in that. What we would love to do is create Jupyter Notebook covering OneFormer training on custom dataset and add it to our repository -> Roboflow Notebooks + some blog post on our blog.

Do you think that creating such a Notebook is even possible given the fact that you need multiple GPUs to do that?

praeclarumjj3 commented 1 year ago

Hi @SkalskiP, I think training on a single GPU for demo purposes should be possible. Which GPU would you be using?

I believe the best way to find out would be to train a ResNet-50 OneFormer on any existing dataset (maybe 512x512 ADE20K?) to see if your GPU can accommodate the training. I have not tested the training code on a single GPU, but it should work fine. Let me know if you face any issues.

SkalskiP commented 1 year ago

Hi @praeclarumjj3! The OneFormer video was one of our best recently. 💪 Do you have any existing config file in the repository that we could use for that? I took a quick look, and I don't see any with ResNet-50 backbone. Could I count on your help, maybe some guidelines on constructing such a config?

As for the GPU, it all depends. The standard Google Colab is 1x Tesla T4. GoogleColab Pro (I have it) is A100-SXM4-40GB. And, if that will not be enough, I'll just spin the AWS machine.

praeclarumjj3 commented 1 year ago

Hi @SkalskiP, glad to know about the video's success! Could you provide a link to this repo and the arxiv preprint in your youtube description, if possible? That would be great for the repo's traffic.

Could I count on your help, maybe some guidelines on constructing such a config?

Sure, you can find the R50 configs for each dataset on the following links:

As for the GPU, it all depends. The standard Google Colab is 1x Tesla T4. GoogleColab Pro (I have it) is A100-SXM4-40GB. And, if that will not be enough, I'll just spin the AWS machine.

40 GB might not be enough. AWS should do the trick.

For some reason, I am not getting notifications regarding your comments here. Why don't you create an issue for this?

SkalskiP commented 1 year ago

Hi @praeclarumjj3! Links are added to the video description. Sure! I'll start the work on Google Colab and create a dedicated issue.