What is the bottleneck of the computation?

[ghost]

I am noticing that even with all parameters that control for size, it still takes 20 seconds to complete an epoch. Any idea why that is?

So the summary looks like this:

_________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            (None, 28, 28, 1)    0
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, 20, 20, 1)    82          input_1[0][0]
__________________________________________________________________________________________________
primarycap_conv2d (Conv2D)      (None, 6, 6, 1)      82          conv1[0][0]
__________________________________________________________________________________________________
primarycap_reshape (Reshape)    (None, 36, 1)        0           primarycap_conv2d[0][0]
__________________________________________________________________________________________________
primarycap_squash (Lambda)      (None, 36, 1)        0           primarycap_reshape[0][0]
__________________________________________________________________________________________________
digitcaps (CapsuleLayer)        (None, 10, 1)        360         primarycap_squash[0][0]
__________________________________________________________________________________________________
input_2 (InputLayer)            (None, 10)           0
__________________________________________________________________________________________________
mask_1 (Mask)                   (None, 10)           0           digitcaps[0][0]
                                                                 input_2[0][0]
__________________________________________________________________________________________________
capsnet (Length)                (None, 10)           0           digitcaps[0][0]
__________________________________________________________________________________________________
decoder (Sequential)            (None, 28, 28, 1)    2367        mask_1[0][0]
==================================================================================================
Total params: 2,891
Trainable params: 2,891
Non-trainable params: 0

[eliphatfs]

What's your computing device? I guess that if it is a GPU, the communication between CPU and GPU, or the IO of the hard disk may be the bottleneck. And if it is a CPU, it is very fast indeed to finish processing an epoch of MNIST in 20 seconds.

[ghost]

I'm using a CPU, but with fully connected layers only I finish an epoch within a second. Even if I up the number of layers in the network shown above it's still 20 seconds. It seems that it's not the computation is causing how slow it is but that it's something else. It's probably not communication because I'm running it in a CPU.

[ghost]

@XifengGuo could you please suggest me the value of lam_recon to use for the RGB images of 100100 pixel resolution? I am not sure whether 10010030.0005 = 15 is the right value to use.

[XifengGuo]

@raaju-shiv I think 0.1～1.0 should be a better value.