search

huggingface / pytorch-image-models

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
https://huggingface.co/docs/timm
Apache License 2.0
31.51k stars 4.7k forks source link

Questions about self-trained results #41

Closed TPFRL closed 4 years ago

TPFRL commented 4 years ago

Hi, thanks for making this awesome training code in pytorch. I was trying to reproduce some results which can be reproduced using tensorflow, but I'm having hard time when I use pytorch.

I noticed that this repository tried to minimize the difference between pytorch and tensorflow. There are custom RMSPropTF, and even the data transformation looks similar to official mnasnet tensorflow code etc.. But I also noticed that you probably used a little different configuration to make your results. For example, you used bilinear interpolation instead when you trained spnas_100.

I have two questions..!

  1. You tried to minimize the difference between pytorch and tensorflow, am I right..?
  2. Why different configuration..? Can you let us know the configuration you used to make the results..?
rwightman commented 4 years ago

@TPFRL

  1. I tried to minimize the difference between PyTorch and Tensorflow when having difficulty reproducing a given level of performance from the paper
  2. spnasnet was trained relatively early, before I managed to match results on mnasnet, mobilnet-v3, etc ... no particular reason why bilinear was used there, perhaps I forgot to add the flag to change it, now I default to randomly selecting between bilinear/bicubic as an augmentation during trainig to reduce the output model's dep on the scaling method used. That's also 'different'. I haven't revisted spnasnet since.