search

astooke / Synkhronos

Extension to Theano for multi-GPU data parallelism
MIT License
20 stars 5 forks source link

Reduce/gather on function results? #4

Closed mharradon closed 7 years ago

mharradon commented 7 years ago

If I have a function returning a result I want, can I gather the results from all of my gpus?

astooke commented 7 years ago

Yes you can!

Say you want to output variable x, which should be average, and y which should be gathered. Give as argument to synk.function: outputs=[(x, 'avg'), (y, 'gather')] (average is the default if you don't write anything).

Also, I've recently noticed that gather through pygpu and nccl may not be well-behaved if the gpuarray in each worker is not exactly the same shape. If you have different shapes, use 'cpu_gather'.

mharradon commented 7 years ago

Great, thanks!