search

facebookresearch / mae

PyTorch implementation of MAE https//arxiv.org/abs/2111.06377
Other
7.23k stars 1.21k forks source link

Do you mind sharing your system environment used to reproduce the results in this repo? #91

Open yxchng opened 2 years ago

yxchng commented 2 years ago

Do you mind sharing your system environment used to reproduce the results in this repo? 

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

My 5 complete runs (from pretrain to finetune, following instructions in PRETRAIN.md and FINETUNE.md) of base, large and huge gives the following results:

base: 83.4193058512719, 83.5672585024562, 83.4053103286184, 83.4432981575596, 83.4373000903688
large: 85.6325975925741, 85.5486244568059, 85.7085732893538, 85.6585892671511, 85.65459056306328
huge: 86.9345429086929, 86.90457167009563, 86.9025735882542, 86.8086637487192

average-base: 83.454 average-large: 85.637 average-huge: 86.882

This is quite a bit far from base: 83.6 | large: 85.9 | huge: 86.9, especially base and large.

Do you have any idea what might have caused the differences?

Based on this thread https://github.com/facebookresearch/mae/issues/30, it seems like this is a widespread problem. Maybe we need some specific versions of certain libraries or frameworks to reproduce the result.

The instruction in FINETUNE.md are different from the hyperparameters used in the paper though

Screenshot 2022-06-08 at 11 40 16 PM

which should i use? the results above are reproduced following exactly instruction in the PRETRAIN.md and FINETUNE.md

Wallace-222 commented 2 years ago

I have the same request. Thanks.

CA4GitHub commented 1 year ago

@yxchng did you get an answer? Do you have a requirements.txt or something similar to share?

Welding-Robot-Association commented 1 year ago

---- Replied Message ---- | From | @.> | | Date | 03/01/2023 14:51 | | To | @.> | | Cc | @.***> | | Subject | Re: [facebookresearch/mae] Do you mind sharing your system environment used to reproduce the results in this repo? (Issue #91) |

@yxchng did you get an answer? Do you have a requirements.txt or something similar to share?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>