Add more DL models for semi-automatic annotation

[bhack]

Do you plan to integrate some new models from Davis 2019 unsupervised/interactive challenges https://davischallenge.org/challenge2019/publications.html?

[nmanovic]

@bhack , it can be interesting. We always want to integrate as many semi-automatic methods as necessary to improve the annotation experience. Do you recommend some specific models from the list?

[bhack]

For the interactive one you can take a look at https://davischallenge.org/challenge2019/papers/DAVIS-Interactive-Challenge-1st-Team.pdf Check also the semi-supervised: https://davischallenge.org/challenge2019/papers/DAVIS-Semisupervised-Challenge-1st-Team.pdf

[bhack]

Also if it is not strictly related to Davis: https://github.com/fidler-lab/curve-gcn

And: http://openaccess.thecvf.com/content_CVPR_2019/html/Wang_Object_Instance_Annotation_With_Deep_Extreme_Level_Set_Evolution_CVPR_2019_paper.html

[nmanovic]

Need to check but some models (e.g. curve-gcn) have GPL license and I believe we cannot use such code in our project under MIT license.

[bhack]

Model license vs code license is an hot topic.

[nmanovic]

@bhack, I remember these guys. I talk with them on CVPR 2019 a week ago. They are going to publish own tool (they had demo as well) in the nearest future. But I agree that need to invest time to add more automated methods into CVAT and make it easy to add such methods in the future.

P.S. If you see any methods which should work well (as you send us above) just contact us. Really appreciate your help.

[nmanovic]

One more idea which is worth to capture. Right now we have one built-in method for interactive annotation. Probably it is better to have an interface for such models like we have for auto annotation using OpenVINO. I mean we have .xml, .bin, .json, .py files + a file which describe interactive input for the model (e.g. bounding box, a couple of points, segment, etc..). Thus it will be easy to integrate new such models in the future.

[bhack]

Ideally there could be a sort of standardizzation input->model->post-processing phases. But i.e. curve-gcn requires splines as it was an evolution over polyrnn and polyrnn-pp. So in that case need we to maintain splines for input and post-processing?

Check also this old thread https://github.com/fidler-lab/polyrnn-pp/issues/1

[bhack]

There are also other type of interaction models like with the "Collaborative Assistant" https://arxiv.org/pdf/1906.06798.pdf

[davodogster]

@nmanovic This seems to be the latest state of the art auto-segmentation. Better than Dextr and curve-gcn https://openaccess.thecvf.com/content_CVPR_2020/html/Zhang_Interactive_Object_Segmentation_With_Inside-Outside_Guidance_CVPR_2020_paper.html

[nmanovic]

@davodogster , thanks for the advice. Created a separate issue for the model: https://github.com/openvinotoolkit/cvat/issues/2227

[nmanovic]

I will close the issue. IoG model was added. Need to specify exact models to add and describe the reason.