Closed cwhy closed 9 years ago
I am confused: I found no word blocksparse in cost_layers.py. Can you please refer to a specific line?
blocksparse is in Theano and currently don't have a cpu implementation. I take note that it would be good. I don't know where it is used in groundhog.
On Tue, Sep 30, 2014 at 6:22 AM, Dmitry Bogdanov notifications@github.com wrote:
I am confused: I found no word blocksparse in cost_layers.py. Can you please refer to a specific line?
— Reply to this email directly or view it on GitHub https://github.com/lisa-groundhog/GroundHog/issues/13#issuecomment-57293766 .
Oh, I see: it has appeared recently when Kyung-Hyun added Hierarchical Softmax.
Yeah, we should fix that and work on computers with no cuda installed.
Until Theano implements the cpu version of blocksparse op's, I think we should simply constrain the use of HierarchicalSoftmaxLayer to the GPU-only environment. I'm thinking of adding the following lines to the 'init' function of HierarchicalSoftmaxLayer. What do you (@rizar, @cwhy) think?
assert theano.config.device[:3] == 'gpu', 'Hierarchical softmax is not supported without GPU'
from theano.sandbox.cuda.blocksparse import sparse_block_dot_SS
self.sparse_block_dot_SS = sparse_block_dot_SS
@nouiz, is there a plan to implement the CPU implementation of blocksparse in Theano anytime soon?
We will do it, but now the prio for Arnaud is to make work the multi-gpu. So this will wait.
If you want to do it, tell me. It can be done relatively shortly if we make a slow version that just use scans.
On Fri, Oct 3, 2014 at 12:41 PM, Kyunghyun Cho notifications@github.com wrote:
Until Theano implements the cpu version of blocksparse op's, I think we should simply constrain the use of HierarchicalSoftmaxLayer to the GPU-only environment. I'm thinking of adding the following lines to the 'init' function of HierarchicalSoftmaxLayer. What do you (@rizar https://github.com/rizar, @cwhy https://github.com/cwhy) think?
assert theano.config.device[:3] == 'gpu', 'Hierarchical softmax is not supported without GPU' from theano.sandbox.cuda.blocksparse import sparse_block_dot_SS self.sparse_block_dot_SS = sparse_block_dot_SS
@nouiz https://github.com/nouiz, is there a plan to implement the CPU implementation of blocksparse in Theano anytime soon?
— Reply to this email directly or view it on GitHub https://github.com/lisa-groundhog/GroundHog/issues/13#issuecomment-57821291 .
@kyunghyuncho, the temporary solution you propose is good. Can please you commit it?
In cost_layers.py, the dependency blocksparse in not default installed in computers without NVIDIA GPU.
Is CUDA a minimum system requirement for future versions of lisa-groundHog?