rakeshvar / rnn_ctc

Recurrent Neural Network and Long Short Term Memory (LSTM) with Connectionist Temporal Classification implemented in Theano. Includes a Toy training example.
Apache License 2.0
220 stars 80 forks source link

float32 vs float64 on GPU #3

Closed rakeshvar closed 9 years ago

rakeshvar commented 9 years ago
Using gpu device 0: GeForce GTX TITAN Black

Config: 0
   Midlayer: <class 'reccurent.RecurrentLayer'> {'nunits': 9}
Input Dim: 8
Num Classes: 10
Num Samples: 1000
FloatX: float32
Using log space: True

Preparing the Data
Building the Network
Traceback (most recent call last):
  File "train.py", line 115, in <module>
    ntwk = NeuralNet(nDims, nClasses, midlayer, midlayerargs, log_space)
  File "/home/rakesha/rnn_ctc/neuralnet.py", line 16, in __init__
    layer3 = CTCLayer(layer2.output, labels, n_classes, logspace)
  File "/home/rakesha/rnn_ctc/ctc.py", line 68, in __init__
    self.log_ctc()
  File "/home/rakesha/rnn_ctc/ctc.py", line 117, in log_ctc
    outputs_info=[safe_log(_1000)]
  File "/home/rakesha/.virtualenvs/python3.4/lib/python3.4/site-packages/theano/scan_module/scan.py", line 1017, in scan
    scan_outs = local_op(*scan_inputs)
  File "/home/rakesha/.virtualenvs/python3.4/lib/python3.4/site-packages/theano/gof/op.py", line 481, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/home/rakesha/.virtualenvs/python3.4/lib/python3.4/site-packages/theano/scan_module/scan_op.py", line 339, in make_node
    inner_sitsot_out.type.dtype))
ValueError: When compiling the inner function of scan the following error has been encountered: The initial state (`outputs_info` in scan nomenclature) of variable IncSubtensor{Set;:int64:}.0 (argument number 1) has dtype float32, while the result of the inner function (`fn`) has dtype float64. This can happen if the inner function of scan results in an upcast or downcast.
mfouadthabet commented 9 years ago

I think this can be solved in ctc.py by: -return tt.log(tt.maximum(x, eps)) +return tt.log(tt.maximum(x, eps).astype(theano.config.floatX))

-return tt.exp(tt.minimum(x, epsinv)) +return tt.exp(tt.minimum(x, epsinv).astype(theano.config.floatX))

Thanks

rakeshvar commented 9 years ago

Thank you @mfouadthabet . Your solution worked.