Orthogonal initializer is being called on a matrix with more elements: Slowness may result.

[Raza25]

tensorflowjs version: 0.11.7 Keras version: 2.0.4

I am trying to run Keras converted model on browser. For conversion I have used tensorflowjs converter, the conversion went fine. However, at load time, Orthogonal initializer is being called on a matrix with more than 2000 (1000000) elements: Slowness may result. this message pops up and the memory blowup happens and lastly the browser stops working. Following is my keras model architecture:

Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 25)                0         
_________________________________________________________________
OneHot (Embedding)           (None, 25, 128)           186368    
_________________________________________________________________
bidirectional_1 (Bidirection (None, 2000)              9032000   
_________________________________________________________________
repeat_vector_1 (RepeatVecto (None, 25, 2000)          0         
_________________________________________________________________
bidirectional_2 (Bidirection (None, 25, 2000)          24008000  
_________________________________________________________________
dropout_1 (Dropout)          (None, 25, 2000)          0         
_________________________________________________________________
AttentionDecoder (AttentionD (None, 25, 1456)          7125040   
=================================================================
Total params: 40,351,408
Trainable params: 40,351,408
Non-trainable params: 0

Is there a way to speed up the Orthogonal initialization process ? so the model gets loaded in less time and with less effort. Any help will be appreciated.

[zaidalyafeai]

It seems that the number of parameters "40 mil" is huge to run in the browser. Try reducing the params to at most "20 mil". The bidirectional layer with 24 mil params is huge.

[Raza25]

@zaidalyafeai any # of elements in the matrix > 2000 results in popping up this message. Also elements < 2000 can't suffice to a deep learning seq to seq model. Do you have any other solution for the cause?

[zaidalyafeai]

You could divide the model into two models. Load the first, save the activations then load the second.

[zaidalyafeai]

This will take some time. The other alternative, is to look for smaller models with comparable accuracy.

[Raza25]

I have reduced the parameters to 7,927,065. Still the issue prevails

Following warning comes up.

.

[zaidalyafeai]

Send me the code. I will try to debug it.

[Raza25]

Changing the tensorflowjs version from 0.12.0 to 0.13.0 solved the problem. However any such limitation with old versions is not listed on the site. So, there must be a way around this.