search

baaivision / EVA

EVA Series: Visual Representation Fantasies from BAAI
MIT License
2.29k stars 167 forks source link

EVA 2.0-CLIP #41

Closed vateye closed 1 year ago

vateye commented 1 year ago

Hi, nice work to EVA-2.0, is there any plan to try EVA 2.0 training on CLIP or scaling EVA 2.0 to giant size?

Yuxin-CV commented 1 year ago

Hi, nice work to EVA-2.0, is there any plan to try EVA 2.0 training on CLIP or scaling EVA 2.0 to giant size?

Hi @vateye, all EVA-CLIP models (from 150M to 4.7B params) will be released in one week, please stay tuned.

vateye commented 1 year ago

Is there any information for 4.7B model? I am excited about it.

Yuxin-CV commented 1 year ago

Hi @vateye, we have released all EVA-CLIP models and inference code at https://github.com/baaivision/EVA/tree/master/EVA-CLIP#model-card.

We offer performant CLIP models at all scales, range from 150M to 4.7B. Try them out if you are interested.

vateye commented 1 year ago

Thanks for your response. Is there any technical report for it?

Yuxin-CV commented 1 year ago

Thanks for your response. Is there any technical report for it?

The technical report are coming very soon.

vateye commented 1 year ago

Hi, can you explain why Rope and subln is not used for EVA-2-e (CLIP)?

Yuxin-CV commented 1 year ago

Hi, can you explain why Rope and subln is not used for EVA-2-e (CLIP)?

Because the 4.7B EVA-CLIP-E were trained before we started to explore RoPE, SwiGLU and sub-LN, etc.

We still call it EVA-02-CLIP-E because of the MIM teacher is EVA-01-CLIP-g.

vateye commented 1 year ago

Thanks for your response. Is there any technical report for it?

The technical report are coming very soon.

Hi, is there any update for EVA-CLIP report?

Yuxin-CV commented 1 year ago

Thanks for your response. Is there any technical report for it?

The technical report are coming very soon.

Hi, is there any update for EVA-CLIP report?

Coming in 24h.

Yuxin-CV commented 1 year ago

The report is available at https://arxiv.org/abs/2303.15389