search

ucbdrive / few-shot-object-detection

Implementations of few-shot object detection benchmarks
Apache License 2.0
1.08k stars 225 forks source link

Steps to register new dataset #120

Closed pranay-ar closed 3 years ago

pranay-ar commented 3 years ago

Hi, I have gone through #29 which discusses the same issue but I am exactly not sure as to what to change in these files. After just modifying the names of the train path and val path and commenting out the rest of them since I did not have separate annotations for these, I executed the builtin.py file:

sample

After running the builtin.py file, I get this error: python -m data.builtin.py /usr/bin/python3: Error while finding module specification for 'data.builtin.py' (ModuleNotFoundError: No module named 'data')

Could someone please guide me as to how to train Fsdet on a custom dataset. Apologies, if it sounds like a beginner's issue, I am still a beginner and hence I've trying to run this code without errors for a few days.

davesie commented 3 years ago

What kind of dataset do you want to use? Is it available in the COCO format?

pranay-ar commented 3 years ago

Hi, yes, I have annotated the data in COCO format. For example, this is the link to my training data: https://drive.google.com/file/d/1jow7tiPZDgbs_OhY4Ii4sIwheA5C50Kl/view?usp=sharing

PS: I am initially trying out this model. As a whole, I have to train around 100 base classes with 60 images each and around 500 novel classes. Is there any base/novel class ratio that is to be followed for better results?

davesie commented 3 years ago

Did you already prepare your data for the few-shot training with this?

There are several parts that have to be modified. Out of memory this is what I did:

davesie commented 3 years ago

Hi, yes, I have annotated the data in COCO format. For example, this is the link to my training data: https://drive.google.com/file/d/1jow7tiPZDgbs_OhY4Ii4sIwheA5C50Kl/view?usp=sharing

PS: I am initially trying out this model. As a whole, I have to train around 100 base classes with 60 images each and around 500 novel classes. Is there any base/novel class ratio that is to be followed for better results?

Did you read the paper? There the ratio is about 3:1 base:novel And did you read the response in #86 ?

pranay-ar commented 3 years ago

Hi, yes, I have annotated the data in COCO format. For example, this is the link to my training data: https://drive.google.com/file/d/1jow7tiPZDgbs_OhY4Ii4sIwheA5C50Kl/view?usp=sharing PS: I am initially trying out this model. As a whole, I have to train around 100 base classes with 60 images each and around 500 novel classes. Is there any base/novel class ratio that is to be followed for better results?

Did you read the paper? There the ratio is about 3:1 base:novel

Yes, I went through the algorithm part and skimmed through the rest. Might have missed it. My bad. Thank you for your help. I will try it as you have suggested.

pranay-ar commented 3 years ago

Hi @davesie, thank you for the help previously. I was able to register and train the model until the last step. While fine-tuning it on the balanced novel set (the very last step of the training process), after the training is completed, while evaluating, I am running into Assertion error i.e. error

I have trained the model on 12 classes (9 base and 3 novel) and have changed the code accordingly in here, here, here, here and here.

I have googled and found on detectron2's repo but was not able to fix it. Can you please guide me in what way can I fix it?

selimlouis commented 3 years ago

@pranay-ar hi, I had the same problem but it stopped occuring after I used a test set that had all the categories the model was trained on, not just my novel category. I am not sure if that was the actual cause for the problem though. Hope that helps

pranay-ar commented 3 years ago

@pranay-ar hi, I had the same problem but it stopped occuring after I used a test set that had all the categories the model was trained on, not just my novel category. I am not sure if that was the actual cause for the problem though. Hope that helps

Hi @selimlouis, thank you for the reply. I actually solved this error by using an additional argument --coco that had to be used when training on a custom set. I forgot to close the issue. Thank you for the suggestion by the way!