GCNN

[ErnstTmp]

Hi Ke, thank you very much for writing and publishing your code! I am loving this!

I am trying to run the GCNN, but was not successful. I converted it to Keras 1.2.0 first, by changing the add_weights commands in gcnn.py, see at the bottom.

But still, I got the message:

AssertionError: Can't store in size_t for the bytes requested 18446744073709551615 * 4

The network looks like this:

____________________________________________________________________________________________________
embedding_1 (Embedding)          (None, 400, 100)      0           word_input[0][0]
____________________________________________________________________________________________________
gcnn10 (GCNN)                    (None, 400, 30)       54090       embedding_1[0][0]
____________________________________________________________________________________________________
queue_input (InputLayer)         (None, 31)            0
____________________________________________________________________________________________________
flatten_1 (Flatten)              (None, 12000)         0           gcnn10[0][0]
____________________________________________________________________________________________________
dense_1 (Dense)                  (None, 512)           16384       queue_input[0][0]
____________________________________________________________________________________________________

Could you be so kind as to look into that?

Thanks again and kind regards Ernst

Fixes for Keras 1.2.0 in gcnn.py

        self.W_z = self.init(self.W_shape, name='{}_W_z'.format(self.name))
        self.W_f = self.init(self.W_shape, name='{}_W_f'.format(self.name))
        self.W_o = self.init(self.W_shape, name='{}_W_o'.format(self.name))
        self.trainable_weights = [self.W_z, self.W_f, self.W_o]
        self.W = K.concatenate([self.W_z, self.W_f, self.W_o], 
        if self.bias:
            #self.b = self.add_weight((self.output_dim * 2,),
            #                         initializer='zero',
            #                         name='{}_b'.format(self.name),
            #                         regularizer=self.b_regularizer,
            #                         constraint=self.b_constraint)
            self.b_z = K.zeros((self.output_dim,), name='{}_b_z'.format(self.name))
            self.b_f = K.zeros((self.output_dim,), name='{}_b_f'.format(self.name))
            self.b_o = K.zeros((self.output_dim,), name='{}_b_o'.format(self.name))
            self.trainable_weights += [self.b_z, self.b_f, self.b_o]
            self.b = K.concatenate([self.b_z, self.b_f, self.b_o])

[ErnstTmp]

I get the same error when I run your imdb_lm_gcnn.py:

WARNING: probably bad CudaNdarray_set_dim arguments: self->ndim=4, idx=3 stride=-1
Traceback (most recent call last):
  File "imdb_lm_gcnn.py", line 69, in <module>
    run_demo()
  File "imdb_lm_gcnn.py", line 65, in run_demo
    train_model()
  File "imdb_lm_gcnn.py", line 61, in train_model
    nb_epoch=nb_epoch, verbose=1)
  File "/home/ernst/anaconda2/envs/tensorflow_27/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/engine/training.py", line 1509, in fit_generator
    class_weight=class_weight)
  File "/home/ernst/anaconda2/envs/tensorflow_27/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/engine/training.py", line 1268, in train_on_batch
    outputs = self.train_function(ins)
  File "/home/ernst/anaconda2/envs/tensorflow_27/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/backend/theano_backend.py", line 957, in __call__
    return self.function(*inputs)
  File "/home/ernst/anaconda2/envs/tensorflow_27/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/compile/function_module.py", line 875, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/ernst/anaconda2/envs/tensorflow_27/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/ernst/anaconda2/envs/tensorflow_27/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/compile/function_module.py", line 862, in __call__
    self.fn() if output_subset is None else\
AssertionError: Can't store in size_t for the bytes requested 18446744073709551615 * 4
Apply node that caused the error: GpuAllocEmpty(Shape_i{0}.0, Elemwise{Composite{Switch(i0, (i1 + i2 + i3), i2)}}[(0, 1)].0, Elemwise{Composite{((i0 - (((i1 - i2) * i3) + i2)) + i2)}}[(0, 0)].0, Elemwise{Composite{((i0 - (((i1 - i2) * i0) + i2)) + i2)}}[(0, 1)].0)
Toposort index: 289
Inputs types: [TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar)]
Inputs shapes: [(), (), (), ()]
Inputs strides: [(), (), (), ()]
Inputs values: [array(32), array(60), array(30), array(-1)]
Outputs clients: [[GpuDnnConv{algo='time_once', inplace=True}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty.0, GpuDnnConvDesc{border_mode='valid', subsample=(1, 1), conv_mode='conv', precision='float32'}.0, Constant{1.0}, Constant{0.0})]]

Thanks Ernst

[DingKe]

I can reproduce the error for theano backend on CPU. The code is tested using tensorflow. As I don't know what's wrong with theano, so I will add a note for this issue.

[ErnstTmp]

Hi Ke, thank you very much for your help. I made the modifications above for running on Keras and TensorFlow 0.12, and I get the message

   raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 3 from 1 for 'Conv2D' (op: 'Conv2D') with input shapes: [32,?,1,20], [3,3,20,60].

when I run your original imdb_lm_gcnn.py. Did I do something wrong when porting to Keras 1.2? It looks like the dimension ordering is not working as it should. Thanks, Ernst

[DingKe]

Sorry for not mentioning it anywhere. Should using 'th' image_dim_ordering. see the docs for how to do it.

[ErnstTmp]

Thanks!

That is strange - my keras.json looks like this:

{ "image_dim_ordering": "th", "floatx": "float32", "epsilon": 1e-07, "backend": "tensorflow"

}

and I still get:

File "/home/ernst/anaconda2/envs/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py", line 670, in _call_cpp_shape_fn_impl status) File "/home/ernst/anaconda2/envs/anaconda3/lib/python3.5/contextlib.py", line 66, in exit next(self.gen) File "/home/ernst/anaconda2/envs/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 469, in raise_exception_on_not_ok_status pywrap_tensorflow.TF_GetCode(status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Negative dimension size caused by subtracting 3 from 1 for 'Conv2D' (op: 'Conv2D') with input shapes: [?,402,1,100], [3,3,100,60].

Can this be a problem of TF 0.12or Keras 1.2

Sorry,

Ernst

On 01/13/2017 03:07 AM, Ke Ding wrote:

Sorry for not mentioning it anywhere. Should using 'th' image_dim_ordering. see the docs https://keras.io/backend/ for how to do it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DingKe/nn_playground/issues/1#issuecomment-272342031, or mute the thread https://github.com/notifications/unsubscribe-auth/AQuv23a9NONJFnbuhFQzillTzIgS-IdVks5rRtx3gaJpZM4LgVfN.