Could you tell me the version of your tensorflow and keras?

[CuriousCat-7]

I am pretty sure that your tensorflow version is lower than 1.4 Although I have tried my best to fix it support 1.4+ version. But some unexpected errors alway trouble me.

Thank you Neo

[elhanan7]

Can you give more info about the errors you encountered?

[CuriousCat-7]

@elhanan7 Sorry for late, truly Yeah, there are lots of problems if we use tensorflow1.6+ I have forked and build a new project in
https://github.com/CuriousCat-7/simnets-tf.git

The first is in keras.py, in:

from keras.backend.tensorflow_backend import _initialize_variables

Actually tensorflow keras does not have keras.backend.tensorflow_backend._initialize_variables anymore.

And in gmm.py the return of _gmm has been changed since tensorflow ver 1.4.

And there are other error, you can find them in my commit of my projects.

I am not good at tensorflow, so some errors are hard to be fixed by myself. And the more I dug into, the more problem I got QAQ.

I assume your tensorflow version should be under 1.2. And the CUDA version should be 7.5. Am I right?

[CuriousCat-7]

WARNING:tensorflow:From /data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/simnets/keras.py:171: calling reduce_logsumexp (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 1, 28, 28)         0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 32, 28, 28)        288       
_________________________________________________________________
similarity_1 (Similarity)    (None, 32, 16, 16)        8192      
_________________________________________________________________
mex_1 (Mex)                  (None, 64, 16, 16)        8192      
_________________________________________________________________
average_pooling2d_1 (Average (None, 64, 8, 8)          0         
_________________________________________________________________
lambda_1 (Lambda)            (None, 64, 8, 8)          0         
_________________________________________________________________
mex_2 (Mex)                  (None, 128, 8, 8)         32768     
_________________________________________________________________
average_pooling2d_2 (Average (None, 128, 4, 4)         0         
_________________________________________________________________
lambda_2 (Lambda)            (None, 128, 4, 4)         0         
_________________________________________________________________
mex_3 (Mex)                  (None, 256, 4, 4)         131072    
_________________________________________________________________
average_pooling2d_3 (Average (None, 256, 2, 2)         0         
_________________________________________________________________
lambda_3 (Lambda)            (None, 256, 2, 2)         0         
_________________________________________________________________
mex_4 (Mex)                  (None, 512, 2, 2)         524288    
_________________________________________________________________
average_pooling2d_4 (Average (None, 512, 1, 1)         0         
_________________________________________________________________
lambda_4 (Lambda)            (None, 512, 1, 1)         0         
_________________________________________________________________
mex_5 (Mex)                  (None, 10, 1, 1)          5120      
_________________________________________________________________
flatten_1 (Flatten)          (None, 10)                0         
=================================================================
Total params: 709,920
Trainable params: 709,920
Non-trainable params: 0
_________________________________________________________________
None
running unsupervised initialization for layer conv2d_1:
Traceback (most recent call last):
  File "keras-example.py", line 91, in <module>
    sk.perform_unsupervised_init(model, 'kmeans', layers=None, data=x_train, batch_size=100)
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/simnets/keras.py", line 243, in perform_unsupervised_init
    u_update, u_assign = _pca_unsupervised_init(conv_op, layer.weights[0])
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/simnets/unsupervised/pca.py", line 88, in pca_unsupervised_init
    pca_for_filters = tf.reshape(pca, filters_var.get_shape().as_list())
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 6113, in reshape
    "Reshape", tensor=tensor, shape=shape, name=name)
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1734, in __init__
    control_input_ops)
  File "/data/limingyao/anaconda2/envs/tf/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1570, in _create_c_op
    raise ValueError(str(e))
ValueError: Cannot reshape a tensor with 81 elements to shape [3,3,1,32] (288 elements) for 'conv2d_1/convolution_pca_init/conv2d_1/convolution_pca_init/Reshape_1' (op: 'Reshape') with input shapes: [9,9], [4] and with input tensors computed as partial shapes: input[1] = [3,3,1,32].

And, pca init problem if I add Keras.layers.Conv2D layer

[CuriousCat-7]

solved by myself. The out_channls of conv2d should be equal or lower than kernel.size(0)kernel.size(1)kernel.size(2), because the conv2d there uses PCA to initialize. My fault for not fully understanding the paper.