Get 400 error when submitting jsonl to firebase, but successfully submit Slack notification

C-1995-O1-HALE-POPP commented 1 year ago
(datacomp114514) cometp@LAPTOP-COMETP:~/datacomp-main$ python evaluate.py --track=filtering  --train_output_dir="0_clip_mask/" --samples="0_clip_masked.npy" --submit --method_name="text masked CLIP 30%" --author="NUM" --email="zihan1.22@intl.zju.edu.cn" --hf_username="Cometp" --hf_repo_name="datacomp_0.3_clip_mask"
Found 40 eval result(s) in 0_clip_mask/eval_results.jsonl.
Warning, did not find or could not read checkpoint at datacomp/test1/0_clip_mask/checkpoints/epoch_latest.pt
Defaulting to 0_clip_mask/checkpoints/epoch_latest.pt
Evaluating
Skipping Caltech-101 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.2320
Skipping CIFAR-10 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.4359
Skipping CIFAR-100 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1781
Skipping CLEVR Counts since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1359
Skipping CLEVR Distance since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.2010
Skipping Country211 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0146
Skipping Describable Textures since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0787
Skipping EuroSAT since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1917
Skipping FGVC Aircraft since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0106
Skipping Food-101 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0804
Skipping GTSRB since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0496
Skipping ImageNet 1k since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0483
Skipping ImageNet Sketch since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0194
Skipping ImageNet v2 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0422
Skipping ImageNet-A since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0161
Skipping ImageNet-O since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1430
Skipping ImageNet-R since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0657
Skipping KITTI Vehicle Distance since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.3924
Skipping MNIST since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1322
Skipping ObjectNet since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0534
Skipping Oxford Flowers-102 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0447
Skipping Oxford-IIIT Pet since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0711
Skipping Pascal VOC 2007 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.3037
Skipping PatchCamelyon since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.5121
Skipping Rendered SST2 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.5008
Skipping RESISC45 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0803
Skipping Stanford Cars since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0295
Skipping STL-10 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.5185
Skipping SUN397 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1045
Skipping SVHN since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.1426
Skipping Flickr since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0323
Skipping MSCOCO since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0199
Skipping WinoGAViL since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.2827
Skipping iWildCam since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0116
Skipping Camelyon17 since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.5009
Skipping FMoW since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.0000
Skipping Dollar Street since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.2757
Skipping GeoDE since results are already in 0_clip_mask/eval_results.jsonl
Score: 0.2714
Skipping FairFace since results are already in 0_clip_mask/eval_results.jsonl
Score: No summary metric
Skipping UTKFace since results are already in 0_clip_mask/eval_results.jsonl
Score: No summary metric
Evaluation time: 0 hour(s) 0 minute(s) 0 second(s)

=== Final results ===
ImageNet 1k: 0.0483
Caltech-101: 0.2319868753931739
CIFAR-10: 0.4359
CIFAR-100: 0.1781
CLEVR Counts: 0.13593333333333332
CLEVR Distance: 0.201
Country211: 0.014644549763033175
Describable Textures: 0.07872340425531915
EuroSAT: 0.19166666666666668
FGVC Aircraft: 0.010561497326203208
Food-101: 0.08043564356435644
GTSRB: 0.0496437054631829
ImageNet Sketch: 0.019434455383285188
ImageNet v2: 0.0422
ImageNet-A: 0.016133333333333333
ImageNet-O: 0.143
ImageNet-R: 0.06573333333333334
KITTI Vehicle Distance: 0.3924050632911392
MNIST: 0.1322
ObjectNet: 0.053407989662969745
Oxford Flowers-102: 0.04470542875600805
Oxford-IIIT Pet: 0.07114696955308561
Pascal VOC 2007: 0.3036858974358974
PatchCamelyon: 0.51214599609375
Rendered SST2: 0.500823723228995
RESISC45: 0.08031746031746032
Stanford Cars: 0.029473946026613605
STL-10: 0.5185
SUN397: 0.10447431818599776
SVHN: 0.14255531653349723
Flickr: 0.032300000078976154
MSCOCO: 0.01987708918750286
WinoGAViL: 0.2826642132262741
iWildCam: 0.011556535188310048
Camelyon17: 0.5008582782702754
FMoW: 0.0
Dollar Street: 0.2757009267807007
GeoDE: 0.27139875292778015
FairFace: None
UTKFace: None
=====================
Average: 0.16377880796211722
Done with evaluations. Preparing your submission...
Pushing files to HF Hub (Cometp/datacomp_0.3_clip_mask). This may take a while.
Done uploading files to HF Hub.
====================================================================================================

            Error: something went wrong when submitting your results.
            Please check if your HF credentials are correct, and contact the team if errors persist.

==================================================================================================== 400
Sucessfully submitted your results. Thanks for participating, and good luck!
gabrielilharco commented 1 year ago
Hi @C-1995-O1-HALE-POPP, I'm not able to reproduce this. Are you using the latest version of this codebase? Maybe it's being blocked by a firewall?
gabrielilharco commented 1 year ago
Ah, I think I found the issue. It seems the problem is because of the . character in your hf_repo_name. I just pushed a fix, can you try again?
mlfoundations / datacomp

Get 400 error when submitting jsonl to firebase, but successfully submit Slack notification #51
