search

biomap-research / scFoundation

Apache License 2.0
252 stars 37 forks source link

Training is slow for Norman perturbation prediction benchmark #47

Closed const-ae closed 2 months ago

const-ae commented 2 months ago

Hi,

I am currently trying to reproduce the results from the perturbation prediction benchmark for the Norman data. I am able to run the run_singlecell_maeautobin-0.1B-res0-norman.sh script on our NVIDIA L40S graphics card. However, the training takes quite long: I was able to complete one epoch in ~8h. If I run GEARS (without scfoundation) on the same data, I can complete 20 epochs in half the time. Is this expected? How long did the training for the Norman benchmark take for you?

Best, Constantin

WhirlFirst commented 2 months ago

It depends on the card you used, in our case, we used the A100, and it took hours to finish one epoch.

const-ae commented 2 months ago

Thanks, that is useful. I was worried that I was doing something wrong, but I guess that my experience is not unusual then. Just as another data point, I now managed to complete 5 epochs in 72 hours on a L40S.