search

loeweX / Greedy_InfoMax

Code for the paper: Putting An End to End-to-End: Gradient-Isolated Learning of Representations
https://arxiv.org/abs/1905.11786
MIT License
283 stars 36 forks source link

Training time and memory usage #20

Closed freor closed 2 years ago

freor commented 2 years ago

Hi, GIM looks super cool and thanks for the code! For vision and audio task, I would like to ask how does it take for the network to converge and how much big memory should i need to train! (I only have one RTX 3080) It would be appreciate to respond to this! :)

loeweX commented 2 years ago

Hey,

Thanks for your interest in GIM!

I'm afraid GIM (and contrastive approaches in general) take quite some time to train. I cannot give you the exact timing, but I would say the vision model trained across 8 GeForce 1080 Ti took <2 days to train, the audio model trained across 4 GPUs took substantially longer until it was fully converged.

I hope this helps! Best, Sindy