Models for the Google Summer of Code 2023 prerequisite task

adrianboguszewski commented 1 year ago

The models listed below are for the GSoC 2023 prerequisite task only.

We provide several potential candidates. Please select only one which hasn't already been selected (look at the checkboxes and comments below). When you decide, assign a model to you by adding a comment with the model name. Then we will tick it to mark reserved.

If you struggle, you can reassign yourself to another non-taken model. However, we can do it only once.

When you create a PR, please follow the self-checklist below:

each function is described by docstrings and type hints
notebook contains explicit descriptions and explanatory diagrams
the notebook doesn't use any data (image, video, etc.) that is not CC4.0 licensed
there is a README.md file in consistent style (look at other notebooks)
the notebook is added to the main README
there are no grammar, punctuation or typo issues (use any free tool for that e.g. Grammarly)
there are no committed files besides notebook and readme (please use images or videos from data dir)
your PR doesn't change any other notebooks
all CI checks passed

Object detection:

[x] Yolov6 - https://github.com/meituan/YOLOv6 (@ahmd-nish)
[x] DAMO YOLO - https://github.com/tinyvision/DAMO-YOLO (@Muskan33)
[x] YoloX - https://github.com/Megvii-BaseDetection/YOLOX (@sawradip)
[x] RTMDet - https://github.com/open-mmlab/mmyolo/tree/main/configs/rtmdet (@AnuragMaiti)
[x] EfficientDet - https://github.com/google/automl/tree/master/efficientdet (@ashish-2005)
[x] CenterNet - https://github.com/xingyizhou/CenterNet/ (@rajuptvs)
[x] SSD MobileNet V2 http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md (@AlexFierro9)
[x] FasterRCNN Inception ResNet v2 http://download.tensorflow.org/models/object_detection/tf2/20200711/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8.tar.gz https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md (@Paulooh007)
[x] YOLOS https://huggingface.co/hustvl/yolos-tiny (@SandeepaDevin)
[x] DETR https://huggingface.co/facebook/detr-resnet-50 (@Tatwansh)
[x] YoloR https://github.com/WongKinYiu/yolor (@18yz153)
[x] YoloF https://github.com/megvii-model/YOLOF (@thegeek13242)
[x] NanoDet https://github.com/RangiLyu/nanodet (@sahilpmehra)
[x] UltraFace https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB (@JacketChenlll)
[x] YoloV7 Face https://github.com/derronqi/yolov7-face (@lucifertrj)
[x] yolov5-blazeface https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/205-vision-background-removal (@AnuragTimilsina)
[x] RetinaFace https://github.com/biubug6/Pytorch_Retinaface (@VaillaRohit)

Rotated object detection:

[ ] Rotated FCOS - https://github.com/open-mmlab/mmrotate/blob/main/configs/rotated_fcos/README.md
[x] ReDet - https://github.com/open-mmlab/mmrotate/blob/main/configs/redet/README.md (@nischay7)
[ ] Roi_trans - https://github.com/open-mmlab/mmrotate/blob/main/configs/roi_trans/README.md

Semantic Segmentation:

[x] SegFormer - https://huggingface.co/nvidia/segformer-b0-finetuned-ade-512-512 (@Kasliwal17)
[x] ClipSeg - https://huggingface.co/CIDAS/clipseg-rd64-refined (@RishithaR-388)
[x] SETR - https://github.com/fudan-zvg/SETR (@AniketARS)
[x] BeIT - https://huggingface.co/microsoft/beit-base-finetuned-ade-640-640 (@hadyy17)
[x] Segmenter - https://github.com/open-mmlab/mmsegmentation/tree/master/configs/segmenter (@blaz-r)
[x] DeepLab V3 - https://github.com/tensorflow/models/tree/master/research/deeplab (@chaitravi-ce)
[x] FaceParsing |& MakeUp - https://github.com/zllrunning/face-parsing.PyTorch https://github.com/zllrunning/face-makeup.PyTorch (@Lj1ang)
[x] ESPNet https://github.com/sacmehta/ESPNet (@Nouran-Muhammad)
[ ] YoloP https://github.com/hustvl/YOLOP

Instance Segmentation:

[x] YOLACT https://github.com/dbolya/yolact.git (@Abdullah-Elkasaby)
[x] Mask RCNN Inception ResNet V2 http://download.tensorflow.org/models/object_detection/tf2/20200711/mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8.tar.gz https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md (@mr-rajashekhar)

Action/Gesture recognition:

[x] TSM - https://github.com/mit-han-lab/temporal-shift-module (@ntombi)
[x] Timesformer - https://huggingface.co/facebook/timesformer-base-finetuned-k400 (@BrennoMello)
[x] SlowFast - https://github.com/open-mmlab/mmaction2/blob/master/configs/recognition/slowfast/README.md (@rajatkrishna)
[x] YOWOv2 - https://github.com/yjh0410/YOWOv2 (@Matrixmang0)
[x] movinet - https://github.com/tensorflow/models/tree/master/official/projects/movinet (@sharvesh642)
[ ] xclip https://huggingface.co/microsoft/xclip-base-patch32

Background matting:

[ ] ModNet - https://github.com/ZHKKKe/MODNet
[x] Robust Video Background matting - https://github.com/PeterL1n/RobustVideoMatting (@wulongjian)
[ ] MGMatting https://github.com/yucornetto/MGMatting
[ ] PortraitNet https://github.com/dong-x16/PortraitNet

Old Photos Restoration/Image colorization/Image denoising/super resolution:

[x] Bringing Old Photos Back to Life - https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life (@Om-Doiphode )
[x] DeOldify - https://github.com/jantic/DeOldify (@Dhruvanshu-Joshi)
[x] Coltran - https://github.com/google-research/google-research/tree/master/coltran (@weronikazak)
[x] Colorizer https://github.com/richzhang/colorization (@pyther-hub)
[x] SwinIR - https://github.com/JingyunLiang/SwinIR (@Z-Fran)
[x] style-swapping https://github.com/irasin/Pytorch_Style_Swap (@m-gopichand)
[x] Real-ESRGAN (for real images) - https://github.com/xinntao/Real-ESRGAN/blob/master/docs/model_zoo.md (@aadhamm)
[ ] Real-ESRGAN (for animation video) - https://github.com/xinntao/Real-ESRGAN/blob/master/docs/model_zoo.md
[ ] RCAN https://github.com/yulunzhang/RCAN
[ ] Super-SlowMo https://github.com/rmalav15/Super-SloMo
[x] Photo2Cartoon https://github.com/minivision-ai/photo2cartoon (@sususama)

Depth estimation:

[ ] lite-mono https://github.com/noahzn/lite-monoc
[x] MiDaS 3.1 https://github.com/isl-org/MiDaS (@nsk126)
[x] Vi-Depth https://github.com/isl-org/VI-Depth (@pronoym99)

Text classification:

[x] Roberta - https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment @ABHIJATSARARI)
[x] XLM-Roberta - https://huggingface.co/papluca/xlm-roberta-base-language-detection (@hazrulakmal)
[x] DepRoBerta - https://huggingface.co/rafalposwiata/deproberta-large-depression (@SpyzzVVarun)
[x] CodeBerta - https://huggingface.co/huggingface/CodeBERTa-language-id (@zilto)
[x] Albert V2 - https://huggingface.co/textattack/albert-base-v2-MRPC (@dwipddalal)
[x] DistilRoberta - https://huggingface.co/j-hartmann/emotion-english-distilroberta-base (@MR-ENVYR)
[x] FinBERT https://huggingface.co/yiyanghkust/finbert-tone (@shrey-2803)
[x] Deberta https://huggingface.co/microsoft/deberta-base-mnli (@mhy-666)

Token classification:

[x] Part of speech tagging - https://huggingface.co/flair/pos-english (@harish2773)
[x] Punctuation restoring (bert-restore-punctuation) - https://huggingface.co/felflare/bert-restore-punctuation (@seanjyu)
[x] Punctuation restoring (punctuate-all) - https://huggingface.co/kredor/punctuate-all (@theNobody-12)
[x] Typo detection - https://huggingface.co/m3hrdadfi/typo-detector-distilbert-en (@Ravindu987)
[x] Named entity recognition - https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english (@Aditya-vardhan13)

Text generation:

[x] BioGPT https://huggingface.co/microsoft/biogpt (@sidyakinian)
[x] gpt-neo https://huggingface.co/EleutherAI/gpt-neo-125M (@Warlord-K)
[x] OPT https://huggingface.co/facebook/opt-350m (@zhumakhan)

Text Summarization

[x] DistilBART https://huggingface.co/sshleifer/distilbart-cnn-12-6 (@samycolen)

Question Answering

[x] MiniLM https://huggingface.co/deepset/minilm-uncased-squad2 (@Akshit17)
[x] ELECTRA https://huggingface.co/deepset/electra-base-squad2 (@sanjayk0508)
[x] DistilBert https://huggingface.co/distilbert-base-cased-distilled-squad (@fajemila)

Sound classification:

[x] speech emotions recognition (wav2vec) https://huggingface.co/harshit345/xlsr-wav2vec-speech-emotion-recognition (@paxF3E)
[x] Hubert key words spotting https://huggingface.co/superb/hubert-base-superb-ks (@100-87)

sawradip commented 1 year ago

Hello, firstly I want to thank you for the awesome list. I had a question:

Is there any existing instance/example of using open-mmlab models, or such config based model libraries(Ex: detectron2), with Openvino?

sawradip commented 1 year ago

I would like to assign YoloX to myself.

adrianboguszewski commented 1 year ago

@sawradip, assigned to you. We don't have such examples in this repository and I'm not aware of any existing examples of open-mmlab or detectron2 with OpenVINO.

eaidova commented 1 year ago

@sawradip please check https://github.com/open-mmlab/mmdeploy this library support export mmlab pretrained models to OpenVINO format

sawradip commented 1 year ago

Thank you for assigning me that task. I will submit a PR on that, in a few days.

And I am interested on the open-mm projects is, such a good number of models, are out of openvino accessibility. Though It is not GSOC requirement, neither on planned projects, I was thinking, If I could contribute some tutorial, on integrating open-mm or similar config based models.

akashAD98 commented 1 year ago

@eaidova should we need to implement any of the repo in openvino ir format? this is only for GSoC people or anyone from open source community can contribute

eaidova commented 1 year ago

@akashAD98, this activity is only for GSoC participants for now (we use this models list for candidates selection), but we are open for community contributions too :)

If you have any ideas or great models for openvino notebooks, feel free to contribute :)

Kasliwal17 commented 1 year ago

Hi, I would like to assign SegFormer to myself.

ahmd-nish commented 1 year ago

Hi, i would like to assign YOLOv6 to myslef and contribute. Can you please help me with that ?

adrianboguszewski commented 1 year ago

Hi @ahmd-nish, please first clone the repo locally and make it run. Then please look at the 226-yolo7 notebook and create a notebook with the similar flow.

chathuranga-jayanath-99 commented 1 year ago

Hi, I would like to assign Roberta to myself.

ahmd-nish commented 1 year ago

Hi @ahmd-nish, please first clone the repo locally and make it run. Then please look at the 226-yolo7 notebook and create a notebook with the similar flow.

Sure I will go through and get backto you

hazrulakmal commented 1 year ago

Hello, I would like to work on XLM-RoBERTa.

raymondlo84 commented 1 year ago

This is really cool @adrianboguszewski -- Also, would be good to put tags in Discussion just on GSOC2023 topics to keep it organized.

raymondlo84 commented 1 year ago

https://github.com/openvinotoolkit/openvino_notebooks/discussions/categories/gsoc-2023 :) Here we go.

ashish-2005 commented 1 year ago

Hi , I would like to assign EfficientDet to myself

SpyzzVVarun commented 1 year ago

Hi, I would like to assign XLM-RoBERTa to myself.

adrianboguszewski commented 1 year ago

@SpyzzVVarun XLM-Roberta has already been assigned four comments above. Please select another model :)

SpyzzVVarun commented 1 year ago

Then I would like to work with DepRoBerta.

pyther-hub commented 1 year ago

I would like to work with DAMO YOLO, My proficiency lies in computer vision.

pyther-hub commented 1 year ago

sir/ ma'am by what time shall we put a pr for the Prerequisite task notebook

adrianboguszewski commented 1 year ago

The PR must be created (doesn't have to be merged) by April 4, as the link must be included in your GSoC application.

Om-Doiphode commented 1 year ago

Hi, I would like to assign Bringing Old Photos Back to Life to myself.

AlexFierro9 commented 1 year ago

Hi, Would like to assign Timesformer to myself!

Z-Fran commented 1 year ago

Hi, I would like to assign SwinIR to myself.

AnuragMaiti commented 1 year ago

Hi, I would like to work with RTMDet and contribute! Could you please help me out with that?

muskan-k commented 1 year ago

Hello @adrianboguszewski ! I'm Muskan Kothari and I'm here to enhance my contribution to OpenVINO through GSOC'23. I'd like to assign CodeBerta to myself.