-
Thanks for your excellent work about the topic of contrastive learning. I'm interested in the **Contrastive Learning with Multi-level Views**, and looked through your code.
But I feel frustrated f…
-
## 一言でいうと
対照学習の構成要素を分解し最適な選択を探索した研究。対照(ペア)にするData Augmentation、loss前ヘッドの有無/変換方法、バッチサイズ等のパラメーターの3つを調査。Random Crop/Color、非線形変換ヘッド、大きめバッチ/長めの学習stepがキーで、ImageNetにおいて1%のラベルで85.8% top-5を達成。
![image](h…
-
> Liu, Yixin, and Pengfei Liu. 2021. “SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization.” arXiv [cs.CL]. arXiv. http://arxiv.org/abs/2106.01890.
## 解決したい課題
学習時の目的関数(…
-
Hello, thanks for your excellent work.
I try to use contrastive learning in revelant object detection tasks (e.g. semi-supervised object detection), and I write my contrastive loss code by referering…
-
### TODO for Contrastive Learning in TFJS
- faster training with less data?
- [ ] Optional in settings "training mode": a) Supervised with Scores, b) Contrastive with Pairs, c) Mixed Supervised Contra…
-
## 内容
用mixup为对比学习做数据增强,可以在SimCLR,MoCo,BYOL上用,在图像、语音和表格数据(为啥这类文章都不做文本?)上都有提升。也验证了方法的正则化效果,在(1)数据不够(2)数据增强的领域知识不够,两种情况下明显改善了对比学习。
具体就是为batch中每个instance分配virtual label(其实就是它的序号对应的one-hot),然后再mixup生成新…
-
### Title
Rep2Vec: Repository Embedding via Heterogeneous Graph Adversarial Contrastive Learning
### Link
https://dl.acm.org/doi/abs/10.1145/3534678.3539324
### Year
2022
### Author …
l1tok updated
10 months ago
-
## DirectCLR: Understanding Dimensional Collapse in Contrastive Self-supervised Learning
18.10.2021
https://arxiv.org/abs/2110.09348
https://github.com/facebookresearch/directclr
Similar model…
-
## Problem statement
- 최근 제안된 방법론들이 Contrastive learning에 미치는 각각의 영향 실험
- data augmentation: crop, cutout, color, sobel filtering, noise, blur, rotate
- projection layer: linear vs. non…
-
https://twitter.com/arxiv_org/status/1324123219612246018
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning. https://t.co/WIYOoid5Rb pic.twitter.com/kUCcuYvZMP— arxiv …