search

nengo / nengo-dl

Deep learning integration for Nengo
https://www.nengo.ai/nengo-dl
Other
88 stars 22 forks source link

Multiple GPU #127

Open Jbwasse2 opened 4 years ago

Jbwasse2 commented 4 years ago

Can the use of multiple GPU be enabled? If so how?

drasmuss commented 4 years ago

You can switch between different GPUs by using the device parameter (e.g. nengo_dl.Simulator(net, device="/gpu:3")), but that's just running the model on one GPU at a time (although you could run separate model instances on separate GPUs in parallel). Runnning on multiple GPUs simultaneously (using e.g. tf.distribute.MirroredStrategy()) doesn't work at the moment, due to some incompatibilities between the Keras distribution strategy logic and some of the more unconventional Keras features used in Nengo DL. That might require changes upstream in TensorFlow/Keras. We're working on it though!