mindspore-lab / mindcv

A toolbox of vision models and algorithms based on MindSpore
https://mindspore-lab.github.io/mindcv/
Apache License 2.0
231 stars 140 forks source link

feat: add mindspore-version CLIP (rewritten based on openAI-CLIP) #740

Closed XixinYang closed 10 months ago

XixinYang commented 11 months ago

Thank you for your contribution to the MindCV repo. Before submitting this PR, please make sure:

Motivation

Add mindspore-version CLIP which is totally rewritten based on openAI-CLIP.

Test Plan

These code have been tested on the first example in ./README.md with both ViT and ResNet ckpts, which has used all the main APIS, and the result differences are as below: ViT: image ResNet: image After discussion, all the diffences are considered within an acceptable range.

Related Issues and PRs

(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

SamitHuang commented 11 months ago

As a text-image alignment foundation mode, CLIP is not appropriate to place in examples. We can put it under mindone

SamitHuang commented 11 months ago

As a text-image alignment foundation mode, CLIP is not appropriate to place in examples. We can put it under mindone

Take back by words. Let MindCV be its root.

XixinYang commented 11 months ago

TODO: add an API check list in a susquent PR.

XixinYang commented 10 months ago

TODO:Compare with the original CLIP paper on the zero-shot and linear-probe ACC (imagenet)

XixinYang commented 10 months ago

TODO: add an API check list in a susquent PR.

done