baaivision / EVA

EVA Series: Visual Representation Fantasies from BAAI
MIT License
2.24k stars 165 forks source link

Pooling strategy #130

Open mactavish91 opened 10 months ago

mactavish91 commented 10 months ago

Hello Author,

Firstly, I'd like to express my admiration for the excellent work you have been doing. It truly stands out in the research community.

Recently, some studies have suggested that replacing average pooling and CLS features with max pooling when calculating InfoNCE loss can result in a better representation of image semantic features. This prospect was quite captivating to me.

I was wondering whether you have ventured into this area of experimentation or not? Any insights from your experiences in this regard could provide significant contribution towards improving our understanding.

Thank you so much for your time and consideration.