search

IDSIA / brainstorm

Fast, flexible and fun neural networks.
Other
1.3k stars 152 forks source link

MergeLayer #50

Closed Qwlouse closed 8 years ago

Qwlouse commented 8 years ago

We need a layer that has multiple inputs and just concatenates them along the last feature dimension. For CPU that one can be omitted, because the NumpyHandler supports slicing the features, but for usage with the PyCudaHandler this is the only way of merging the outputs of two layers.

flukeskywalker commented 8 years ago

@untom Can this work on the GPU at all or should it be removed?

untom commented 8 years ago

it can work... I'll see if I can hack one up today.

flukeskywalker commented 8 years ago

Great! Is this essentially the same issue that holds back the optimized LSTM implementation in LSTMOpt from working with PyCudaHandler?

Qwlouse commented 8 years ago

By using some extra memory this could be accomplished with only a slice-enabled copy_to (no need for add_tt then. So if that is easier we could do that. Actually I haven't even checked if copy_to works on slices...

untom commented 8 years ago

Implemented via f36b57d6a87d567c535018c9ee2b46a4fb49d473 and f86bb996513b3c8fa395f3189b8f24f43ebc0065 (There might be faster ways to implement this on the GPU, so this might be worth revisiting if a profiling run shows that this is really a bottleneck)