search

tensorflow / similarity

TensorFlow Similarity is a python package focused on making similarity learning quick and easy.
Apache License 2.0
1.01k stars 104 forks source link

Model Reproducibility #278

Open yonigottesman opened 2 years ago

yonigottesman commented 2 years ago

Hi, I think its important to provide scripts to reproduce paper results for the models the package implements. To be some kind of proof everything is working correctly. What do you think? Im not sure who can fund these experiments ¯_(ツ)_/¯

owenvallis commented 2 years ago

Hi @yonigottesman, we're actually working on this right now and plan to provide benchmarks for all the losses this quarter. Additionally, I've personally reproduced (or as close as I could) the experiments for Multi-Sim, Circle Loss, SimSiam, and Barlow Twins. I've also compared our loss outputs for Multi-Sim and SimSiam against the implementations from the authors.

I have a notebook in the cars196 branch that contains my attempt to repro the multi-sim results on the caltech bird dataset. I even tried to emulate the pytorch version of the random crop and resize. We have an optimized variant of this in KerasCV now, and we plan to switch to use that instead as part of the new benchmarks.

yonigottesman commented 2 years ago

sounds amazing

ebursztein commented 2 years ago

@yonigottesman if you are interested in helping reproduce the experiments (e.g adding losses or more datasets, visualization or other idea you might have - we would love your help. We (Google) are going to train the models and make them available but we could use help to have the best reproduction pipeline possible :)