search

AvivNavon / deep-align

Official implementation of Equivariant Deep Weight Space Alignment [ICML 2024]
MIT License
4 stars 1 forks source link

Data, code and notebook #4

Open yonatansverdlov opened 2 months ago

yonatansverdlov commented 2 months ago

Hi!

Could you please add the notebook that replicates the results from your paper? Additionally, I need the CIFAR-trained models, similar to what you uploaded for MNIST.

Specifically, I'm looking for the results from Table 1, Table 2, and Table 3. If time is limited, Table 3 is the most important.

Thanks!

AvivNavon commented 2 months ago

WDYM by Table 3? Table 3 in the current version of the paper lists the CNN arch. we used.

AvivNavon commented 2 months ago

We released more datasets, see here: https://github.com/AvivNavon/deep-align?tab=readme-ov-file#datasets

yonatansverdlov commented 2 months ago

I saw it, thanks! Could you please add code to download also CIFAR using wget? In addition, is the generate splits file adopted to CIFAR? Is the same hyperparams were used for training CIFAR?

AvivNavon commented 2 months ago

To download just use e.g. wget "https://www.dropbox.com/s/lxy11dgi8epn731/cifar10_mlps.zip". The splits file should work for different datasets, just adopt the arguments. Regarding HPs for CIFAR10 (MLPs) we used batch size of 8 and image bs of 32

yonatansverdlov commented 2 months ago

Ok great. I see the models are much heavier, do you have an estimate of the time it took to train relatively to mnist? Thx!

‫בתאריך שבת, 24 באוג׳ 2024 ב-0:17 מאת ‪Aviv Navon‬‏ <‪ @.***‬‏>:‬

To download just use e.g. wget " https://www.dropbox.com/s/lxy11dgi8epn731/cifar10_mlps.zip". The splits file should work for different datasets, just adopt the arguments. Regarding HPs for CIFAR10 (MLPs) we used batch size of 8 and image bs of 32

— Reply to this email directly, view it on GitHub https://github.com/AvivNavon/deep-align/issues/4#issuecomment-2307822155, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVTIJUDQOOZTDASIJPD4FIDZS6RFDAVCNFSM6AAAAABMJ27R5OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBXHAZDEMJVGU . You are receiving this because you authored the thread.Message ID: @.***>

AvivNavon commented 2 months ago

We trained for 25K steps, and it took around 14 hours on a single A100

yonatansverdlov commented 2 months ago

Around 32 epochs?

‫בתאריך שבת, 24 באוג׳ 2024 ב-10:15 מאת ‪Aviv Navon‬‏ <‪ @.***‬‏>:‬

We trained for 25K steps, and it took around 14 hours on a single A100

— Reply to this email directly, view it on GitHub https://github.com/AvivNavon/deep-align/issues/4#issuecomment-2308161847, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVTIJUHO4QQYDMSP3P4JGVTZTAXHRAVCNFSM6AAAAABMJ27R5OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBYGE3DCOBUG4 . You are receiving this because you authored the thread.Message ID: @.***>

AvivNavon commented 2 months ago

Depends on your batch-size. We used 25 epochs