search

IDSIA / brainstorm

Fast, flexible and fun neural networks.
Other
1.3k stars 152 forks source link

All buffers have to be of the same dtype #30

Open untom opened 9 years ago

untom commented 9 years ago

Currently, all buffers (parameters, internals, gradients, ...) are assumed to have the dtype (typically either float or double). This is a bit restrictive: For example, in a max-pooling operation, one would like to store which cell in the current window has the maximum value (as discussed in #29). Something similar would happen in a Maxout-Layer, or when implementing a Top-K Autoencoder. I can work around this for the max-pooling OP, but in general it would be nice to be able to specify an optional dtype for each ShapeTemplate.

flukeskywalker commented 9 years ago

Indeed, this is essentially since the layer has a handler and each handler currently only supports one dtype. This is more for simplicity than anything else -- we don't need separate kernels for inputs being int or floats, for example. We can transition to a default_dtype for each handler along with other supported dtypes in the future.

flukeskywalker commented 9 years ago

Let's set this aside as on the first TODO item after 0.5? This will require some changes in buffer allocation, and probably several additions to PyCudaHandler, but no major reorg.

Qwlouse commented 8 years ago

Yes! Let's definitely not tackle that before the release.