search

airctic / icevision

An Agnostic Computer Vision Framework - Pluggable to any Training Library: Fastai, Pytorch-Lightning with more to come
https://airctic.github.io/icevision/
Apache License 2.0
849 stars 150 forks source link

detr #977

Closed nikky4D closed 2 years ago

nikky4D commented 2 years ago

📓 New <Tutorial/Example>

Is this a request for a tutorial or for an example?
Yes, I want to train detr using IceVision.

What is the task? Object detection with detr, showing 1) example to replace the backbone with my own pretrained resnet50, 2) use detr on icevision custom parser training

Is this example for a specific model? Detr

Don't remove Main issue for examples: #39

nikky4D commented 2 years ago

any update on this?

FraPochetti commented 2 years ago

@ai-fast-track we don't support DETR, do we?

FraPochetti commented 2 years ago

@ai-fast-track wdyt?

hectorLop commented 2 years ago

@FraPochetti I don't know if someone is trying to implement detr into icevision, but I would like to help. I have taken a look at the official implementation, but it isn't a python package. Considering it isn't a python package, how would you approach using all the functionalities they provide? I have thought about cloning the repository and adding its folder to the path, allowing to do import statements such as detr.models ..., but I don't know if it is a suitable solution to icevision.

FraPochetti commented 2 years ago

Hey @hectorLop, thanks a lot for the initiative. The fact that detr is not a python package complicates a lot the integration. We have the same problem for Yolov5 and we ended up creating a package ourselves. We have a script which parses the yolov5 repo, structures it as a proper package and then we manually have to push to pipy. A real pain tbh. Would you mind moving the discussion to the #ice-dev channel on our Discord. It would be a lot easier to have this kind of conversation over there.

fstroth commented 2 years ago

There is a detr implementation in mmdetection this might be the easiest way to integrate detr.

FraPochetti commented 2 years ago

Plus 1 for mmdetection! I didn't know it was there

hectorLop commented 2 years ago

I decided to use the mmdetection approach. I have already implemented it and I'm trying to train a detr model on a custom dataset to see if it works properly. I discovered some cases where the backbone returns nan features, so I'm debugging it.

FraPochetti commented 2 years ago

Let us know if you need help and feel free to already open a PR in case you need assistance

hectorLop commented 2 years ago

I just opened a PR with the implementation #1026. Regarding the problem in which the backbone returned some nan outputs, I spend several days debugging the mmdet library to finally discover that a high learning rate was the source of the problem.