rflperry
commented
2 years ago Would recommend FTE/BTE experiments like in the supcon loss PR. I think the gain of unsupcon over supcon is being able to leverage large amounts of unlabeled data to approach the supervised accuracy levels.
But the SimCLR paper examines transfer learning performance, learning an unsupervised representation and then given labeled data a 1-layer sigmoid classifier. ProgLearn does the same thing, but learns a kNN classifier.
Background
A loss function suitable for unsupervised learning would contribute to extending ProgLearn's training domain to unsupervised settings in which labeled data are not available.
The unsupervised contrastive loss function is described in the paper SimCLR - A Simple Framework for Contrastive Learning of Visual Representations and implemented at this repository. It leverages data augmentations such as cropping and color shifts to enable unsupervised image classification.
Proposed Feature: demonstrate a working prototype of the unsupervised contrastive loss function
Implementation
Addition of folder
unsupcon. This will store the unsupervised contrastive loss function along with the associated demos and data. The goal is to provide a proof of concept to motivate integration of the unsupervised contrastive loss function with the ProgLearn network. For demonstration purposes, the ResNet50 architecture will be trained on the CIFAR-10 dataset in an experiment analogous to the one described in SimCLR.