Simple linear nn

[cbasavaraj]

Hi, GPU newbie here. I am able to execute jcjohnson's torch-rnn fine on my GPU:

Using Apple , OpenCL platform: Apple Using OpenCL device: ATI Radeon HD 6770M ...

But cannot replicate this when I try to port some of my own nets from CPU to OpenCL. I may be missing something simple. Can you please help?

For starters, there are just two layers

model = nn.Sequential():type(dtype) if opt.model == 'linear' then model:add(nn.Reshape(3 * 224 * 224)) model:add(nn.Linear(3 * 224 * 224, #classes))

I have converted the dataset type as well trainSet.data = trainSet.data:type(dtype)

and I use optim for SGD:

optim.sgd(feval, parameters, hyper)

Stack trace below:

In 1 module of nn.Sequential: ...Chandrachud/torch/install/share/lua/5.1/torch/Tensor.lua:458: bad argument #1 to 'set' (expecting number or torch.DoubleTensor or torch.DoubleStorage at /tmp/luarocks_torch-scm-1-4207/torch7/generic/Tensor.c:1125) stack traceback: [C]: in function 'set' ...Chandrachud/torch/install/share/lua/5.1/torch/Tensor.lua:458: in function 'view' ...s/Chandrachud/torch/install/share/lua/5.1/nn/Reshape.lua:43: in function <...s/Chandrachud/torch/install/share/lua/5.1/nn/Reshape.lua:31> [C]: in function 'xpcall' ...Chandrachud/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors' ...handrachud/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward' 5.magique.lua:312: in function 'opfunc' ...rs/Chandrachud/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd' 5.magique.lua:344: in function 'train' 5.magique.lua:430: in main chunk [C]: in function 'dofile' ...chud/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x0108cc8bc0

[hughperkins]

You'd probably best provide a complete script, so I can just copy and paste. But anyway, seems your Sequential needs running :cl() against it, to convert it to cl form?

[cbasavaraj]

Thanks Hugh. In fact, I figured out what was wrong. I had this line dtype = torch.Tensor():cl():type() to get the cl() datatype, and then was executing model = nn.Sequential():type(dtype) but before defining the rest of the model, and not after!

[cbasavaraj]

I have another question though, can you please comment?

From everything I can see (Soumith's benchmarks, your discussion with PavelMiha in another issue, and my own experiments today), training on clnn seems to be no better than just doing it on nn. For example, a forward + backward pass in PavelMiha's network is 3x slower on clnn, and now I'm trying Szagoruyko's vgg+bn, which seems again ~2x slower.

I'm on Mac OS X with OpenCL device: ATI Radeon HD 6770M. I have 16 GB RAM and a 2.4Ghz Intel Core i7 CPU. Does my CPU have much better specs than my GPU, or is there really no benefit from running on clnn? I have tried different batch sizes (from 2 to 128 ... 512), but no parallelisation seems to happen.

I would really appreciate your thoughts. Thanks.

[hughperkins]

Well... an i7 is a good cpu. But yeah, on your hardware, it looks like you might as well train on cpu, rather than using gpu.

On my own laptop, gpu is quite a lot faster than my cpu, but generally speaking, I wouldnt actually train on my laptop as such: I'd just code on my laptop, and then transfer to some faster machine for the actual training task.

[umerkhalid]

Hi, I am just a newbie to LUA/Torch and was trying to execute : recurrent-visual-attention.lua. However, I am getting the same above error. can you help me if I find any fix ?

[umerkhalid]

about above comment: its resolved - the script is updated :)

[hughperkins]

Awesome!