constant output when using truncated version of wavenet

[ebadawy]

The original implementation of wavenet using 24 number of layers is working perfectly when using the provided trained model.

However, when I try to use a smaller version of wavenet (e.g. hparams.layers = 4 ) and load the corresponding weights for it from the pre-trained model, I get a constant output ( always 1 ). any idea why that might be happing and if there is a fix for it?

[edit]

By looking at the last conv layer out, I got the following for a different number of layers last_conv_layer.pdf

seems that using a number of layers less than the original one that the pre-trained model used, I get output with a different range, could that be the problem?

[r9y9]

Are you trying to change the number of layers of a pre-trained model? That doesn't seem right since the WaveNet was trained to minimize negative log lilkelihood which is defined using the output of the last layer. The output of each layer is supposed to have a different range.

[ebadawy]

I was trying to change the number of conv layers in the conv_layers ModuleList. so the first and last conv layers are the same. However, it seems that it is not working. What I am trying to do is to measure the wavenet performance using a different number of conv layers/ number of filters. The best way to do it is to re-train the model using these new hyperparameters. However, I am trying to see if I can make use of the pre-trained model for this task? Do you think changing the number of filters for each conv layer instead will work? (e.g. setting last 32 filters' weights to zero)

[r9y9]

That's expected. The only way to do that is to re-train the model with the new hyperparameters.