Roadmap of MMDetection

[hellock]

We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan will be updated in different issues.

In this issue, you can either:

1. Suggest a new feature by leaving a comment.
2. Vote for a feature request with :+1: or be against with :-1:. (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!)
3. Tell us that you would like to help implement one of the features in the list or review the PRs. (This is the greatest things to hear about!)

We also released our TODO list on the project page. Most of the TODO items are described in their corresponding issues (those labeled by Dev-RD) with detailed requirement documentation. Feel free to leave a message in the issue of any item and create a PR if you are interested in any of the item.

[jin-s13]

Thanks for the amazing project! Glad to see that you are working on YOLOv4. Could you also support ScaledYOLOv4? It is a recently released project, where the best model achieves 55+mAP on COCO dataset.

official code: https://github.com/WongKinYiu/ScaledYOLOv4/tree/yolov4-large

[OnurKyn]

Would you consider adding post processing methods for video object detection like Seq-Nms, REPP etc.? It would be great.

[Tiberius131]

Motivation There is a recent paper https://arxiv.org/abs/2003.10152, which is very helpful for instance segmentation.

Related resources There is an official implementation: https://github.com/WXinlong/SOLO but many people have issues getting it running. Since mmdetection is easy to install, it would be a great addition to this project.

[sandylaker]

I would like to see Deformable DETR: official repo, paper. Compared to the DETR, the novel Deformable DETR has much faster convergence rate, it also shows better performance on the small objects.

It would be great if the Deformable DETR can be added into mmdetection.

[dimimal]

This would be a nice feature https://arxiv.org/abs/2012.07177 Copy-Paste data augmentation method for instance segmentation!

[dimimal]

How about adding the support for yolov4?

Basically YOLOv4 requires mosaic data augmentation and CSP backbone network

[lfydegithub]

As we all know, each method has a benchmark in README.md, But how about to make a big benchmark which includes all method? In my opinion, it need no much time while useful. Thanks .

[zhongqiu1245]

How about support Copy-Paste data augmentation? paper: https://arxiv.org/abs/2012.07177 Copy-Paste data augmentation method can improve mAP very efficiently, simple and effective

[mathmanu]

Looks like EfficientDet and YOLOv4 has got the maximum number of likes in this conversation.

I don't see any recent PR or activity towards EfficientDet in this repository though. Hope that this will be supported soon.

It will be great to see support of EfficientDetLite which is the embedded friendly variant of EfficientDet as supported here: https://github.com/rwightman/efficientdet-pytorch

[abcxs]

any plan for EfficientDet?

[steven0129]

How about support self-adversarial training in YOLOv4 (https://arxiv.org/abs/2004.10934)?

[zhiqwang]

Thanks and glad to know you are willing to help! We can have further discussion in that PR and may expect YOLOv3 in V2.3. The copyright is ok as if it is licensed under Apache-2.0.

Hi mmdetection team and all here,

I've refactored the yolov5 structure to make it more readable in my own repo, and I'm also interested to add it to mmdetection.

I have one doubt here, as we all know that ultralytics releases their yolov5 under the GPL-3 Licenses. But I've totally rewritten the DarkNet, PAN, BackboneWithPAN, and PostProcess modules, the codes don't rely on ultralytics's and it's totally different now, you can check the detail implementation of the modules as mentioned above if you are interest and convenient. I'm not a legal expert, I'm not sure whether we could relicense it to Apache-2.0?

[ayushjaiswal]

Hi, is there any plan to support TFRecord data input format?

[chriss2401]

Hi, is there any plan to support TFRecord data input format?

I asked them recently if they plan to add other annotation formats other than JSON, and it seems that it isn't in their plans at the moment.

https://github.com/open-mmlab/mmdetection/issues/4437

[MySuperSoul]

any plan for CenterNet2? https://arxiv.org/abs/2103.07461 and Swin Transformer? https://arxiv.org/abs/2103.14030

[yarkable]

It would be interesting to be able to visualize the features like CAM or GradCAM， maybe modify or add some hooks

[coollwd]

Hi, thanks for your amazing project! Is there any plan to support few shot object detection?

[fcakyon]

QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection paper: https://arxiv.org/abs/2103.09136 detectron2 code: https://github.com/ChenhongyiYang/QueryDet-PyTorch

[zhongqiu1245]

Hi, thanks for your amazing job！ Is there any plan to support Panoptic-DeepLab？ Thank you！

[cskarthik7]

I have never contributed to mmdetection but would definitely like to contribute. I would like to add support for boundary aware mask RCNN https://arxiv.org/pdf/2007.08921.pdf .

[ananzeng]

SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization paper：https://arxiv.org/pdf/1912.05027 implementation on mmdetection 1.1：https://github.com/yan-roo/SpineNet-Pytorch implementation on tf ：https://github.com/tensorflow/tpu/tree/master/models/official/detection

[mmoghadam11]

Hi authors Can you add DOTA dataset support? ex:more bbox parameters! https://captain-whu.github.io/DOTA/dataset.html Think you so much!!

For now we do not have plan for that, but there have been mmdet-based project doing that. You might be interested in https://github.com/dingjiansw101/AerialDetection.

thats not Enough its just 2 net how can change hbb 2 rbb in other nets????

the second tip is DOTA has hbb ann how can i use them train hbb networks in mmdet???

[rhiver]

CBNetV2 has a very good performance (much better than DetectoRS, king of 2020) in Object Detection as well as in Instance Segmentation. Do you guys plan to integrate it into mmdetection? maybe it will become the king of 2021

[realeve]

https://github.com/JDAI-CV/CoTNet

https://arxiv.org/pdf/2107.12292.pdf

[realeve]

Cross-Covariance Image Transformer

https://github.com/facebookresearch/xcit

[hasuoshenyun]

It seems that there are not a good way to transform the .pth model to .pb model, this may be a major reason why the project is not used by more engineering practitioners at present. If there is a script, that could parse the parameters of .pth to the rewriten tensorflow parameters and save it as .pb, and further more the .tflite, it could be more attractive, it may make the project have more potential users and contributors.

I would like to do part of this, maybe i'd like to finish the transfomation of series of 'yolox'. do you think whether it's worth doing this.

[zhenglongjiepheonix]

any plans to support efficientdet series

[bhyun-kim]

Is there any plan to add

VisDrone Dataset and AI-TOD dataset??

[Keiku]

@hellock Do you manage the roadmap summary? I have to look for it when I want to look back on it later.

• Iteration Plan of v2.13.0 - May 2021 #5064
• Iteration Plan of v2.14.0 - June 2021 #5279
• Iteration Plan of v2.15.0 - July 2021 #5527
• Iteration Plan of v2.17.0 - September 2021 #6000
• Iteration Plan of v2.18.0 - October 2021 #6281
• Iteration Plan of v2.19.0 - November 2021 #6421

I couldn't find anything about v2.12.0 and v2.16.0.

[SEHUN0124]

Is there any reason, develop with pytorch? not tensorflow pls let me know about the reason. thanks

[fcakyon]

A high performing zero shot object detector with adjustable vocabulary: https://github.com/facebookresearch/Detic

[ZwwWayne]

A high performing zero shot object detector with adjustable vocabulary: https://github.com/facebookresearch/Detic

They are in our plan! Feel free to create a PR if you want to contribute it!

[luzibuye]

Any plan to support DAB-DETR, DN-DETR and DINO? :)

[wenmengzhou]

Is there any reason, develop with pytorch? not tensorflow pls let me know about the reason. thanks

papers releasing code with pytorch take more much percent

[ZwwWayne]

In our plan.

[zhongqiu1245]

Any plan to support yolov7? :) code: https://github.com/WongKinYiu/yolov7 paper: https://arxiv.org/abs/2207.02696

[ZwwWayne]

Any plan to support yolov7? :) code: https://github.com/WongKinYiu/yolov7 paper: https://arxiv.org/abs/2207.02696

Will do that in MMYOLO

[kingstarcraft]

how about sliding windows for inference

[2793145003]

Any plan about DiffusionDet? code: https://github.com/ShoufaChen/DiffusionDet paper: https://arxiv.org/abs/2211.09788

[Zency-Sun]

有计划增加DiffusionDet的计划嘛？

[Min-Sheng]

Any plan to support PyTorch 2.0 ?

[Li-Qingyun]

Any plan to support DAB-DETR, DN-DETR and DINO? :)

@luzibuye DAB-DETR: https://github.com/open-mmlab/mmdetection/pull/9252 DINO: https://github.com/open-mmlab/mmdetection/pull/9149

[Li-Qingyun]

@2793145003 @Zency-Sun DiffusionDet #9639

[xlim1996]

How about Polarmask?

[MohitBurkule]

Are there any plans for cutler https://arxiv.org/abs/2301.11320 . More of an augmentation and loss procedure rather than model, but improves segmentation performance even in supervised setting

[Iranb]

Do you have the plan to create a discord channel to exchange skills in mmdetection or other related methods to openmmlab?

[abdksyed]

Validation Loss Plotting in MMdet 3.x It was supported before 3.x, but was removed from 3.

[wqh17101]

@hellock When will you fix this bug? It is blocking our usage of mmdet 3.x https://github.com/open-mmlab/mmdetection/issues/10112

[akashAD98]

when we should expect proper documentation for custom trainig in mmdetection 3x?

[yt7589]

Do you have the plan to support multi task learning in MMDetection? My application need to detect vehicles in the road used object detection. At the same time it need to get the lane areas by using instance segmentation. It is OK that I used two different network such as RTMDet object detection and RTMDet instance segmentation. But it is better if I can use a common backbone and two seperate header: one is object detection and the other is instance segmentation. It will save computation and reduce latency and may have even better generalization. I had implemented this logic in Yolov5 net. It worked just fine. But I found that it is very difficult to implement it in MMDetection. Because the network define is fixed. It seemed that it did not support share the common backbone and have seperate neck and head. Am I right? I think if MMDetection can support multi task learning natively it will be great. By the way in mmclassification project there has some navieve multi task learning feature. But it reside in this project only and it is in its very early stage.