juglab / n2v

This is the implementation of Noise2Void training.
Other
385 stars 107 forks source link

How to employ multiple GPUs during training #148

Open DennisDannecker opened 7 months ago

DennisDannecker commented 7 months ago

Hello,

I am working my self through the different exemplary notebooks and try to learn/understand/employ n2v to different use cases. For the installation I very much sticked the documentation provided in the README (thank you very much for this).

When I am running the training, only one of my 2 or 3 (depending on which station I am working on) of my GPUs are made use of. Since the the training takes quite some time, I would like to employ all available resources.

What should I add in the scripts or setup to tell tensorflow to use all GPUs available. The config file for training does not allow to specify the number of GPUs, right?

Thank you very much for your contribution.

Best regards Dennis

jdeschamps commented 6 months ago

Hi!

Sorry for the delay. To the best of my knowledge, there is currently no way to train multiple GPU. N2V is based on CSBDeep, you could have a look at the library to see if the library allows it.

We are currently working on a new library that should allow multi GPU training, hopefully to be released in the next two months (for N2V support).