Initial support for CuDNN R2 (tested only on OSX) - Githubissues

pluskid / Mocha.jl

Deep Learning framework for Julia

Other

1.29k stars 254 forks source link

Initial support for CuDNN R2 (tested only on OSX) #18

Closed JobJob closed 9 years ago

JobJob commented 9 years ago

There was a new CuDNN release (R2 RC1) with OSX support. These are the changes I made to get the GPU backend working on OSX, let me know what you think and you if want me to submit a PR against dev etc...

N.b. This code requires CuDNN R2, and is incompatible with R1.

P.S. the new CuDNN release supports padded pooling (but I haven't made modifications to support it as yet). P.P.S. Fantastic work on this lib : )

pluskid commented 9 years ago

@JobJob Thanks! This is great! I will have a look and test it locally today.

Concerning dev or master, I think master is OK now, even though this is breaking changes and cuDNN is still in RC.
For blob.to_cpu, it is OK to leave it there. I will try to look into what is julia's convention function to convert some object to an array, and I might rename it. It will make our current unit-test code much shorter.
Padded pooling, please go ahead and add that if you feel like. :)

Again, thanks for this great job!

pluskid commented 9 years ago

There is some issue with the workspace size, causing CUDA memory mapping error when I test on Linux. I fixed this by adding API wrapper to call cudnn to get proper workspace size.

pluskid commented 9 years ago

I'm merging this. Thanks again! It's quite exciting to learn that CuDNN is moving from 4D tensors to ND tensors.

pluskid commented 9 years ago

BTW: I just modified pooling to use cuDNN padding. :)

JobJob commented 9 years ago

Awesome, great stuff! Yes, good form re the workspace size, I just kludged that together quickly to get it initially working. Do you have plans for using the Nd tensors?

Once again, tremendous work on this library, so exciting to have a fast, GPU powered, well designed (and documented) NN library for Julia!

pluskid commented 9 years ago

@JobJob Yes, we have plan to move to ND tensor in the future. But that is not trivial. Not only involve a lot of code modification (fortunately we have relatively complete unit testing), but also need to think about how to make some algorithms work efficiently with ND tensor (e.g. CPU convolution & pooling).