Gradients do not exist for variables ['kernel', 'bias'] when minimizing the loss

[ohilo12]

Hello,

I am not sure if it is a bug or if I am doing something wrong. If I execute the code at the buttom I get the following warning: Gradients do not exist for variables ['kernel', 'bias'] when minimizing the loss. If using model.compile(), did you forget to provide a loss argument? Does this error occurs in your environment, too?

from keras.layers import GlobalMaxPooling1D, Dense, Input
from keras.models import Sequential 
import keras
from tcn import TCN
import numpy as np
import warnings
import sys

warnings.filterwarnings("error") # Convert warnings to exceptions
try:
    # Your code here
    Modell = Sequential()
    # TCN
    Modell.add(Input(shape=(None, 4)))
    Modell.add( TCN(nb_filters=2, kernel_size=91, nb_stacks=1, dilations=[1], padding='causal', 
                    use_skip_connections=True, dropout_rate=0.0, return_sequences=True, activation='relu') )
    Modell.add(GlobalMaxPooling1D())   
    # DENSE
    Modell.add(Dense(4, activation='relu'))
    Modell.add(Dense(1, activation='relu'))
    Modell.add(Dense(4, activation='softmax'))
    Modell.compile(optimizer=keras.optimizers.Adam(learning_rate=0.006712461356073618), loss='SparseCategoricalCrossentropy', metrics=['sparse_categorical_accuracy'])
    batch = np.array([[-1.1,  0.5 , 0.9 , 0.02],
            [-1.0024, -0.18,  0.9, 0.02],
            [-1.0027, -0.52,  0.9, 0.02]])
    labels_batch = 1
    history = Modell.fit(np.array([batch]), np.array([labels_batch]), epochs=1, verbose=0)
except Warning as e:
    print(f"Warning encountered: {e}")
    sys.exit(1)  # Exit the program with a status code

Versions: tf: 2.16.1 keras: 3.4.1 keras-tcn: 3.5.0

[philipperemy]

Instead of using objects in Model.compile, can you try to give strings like adam, etc.

https://keras.io/api/losses/

[ohilo12]

Hey, thank you for your answer. I have tried:

Modell.compile(optimizer="adam", loss='SparseCategoricalCrossentropy', metrics=['sparse_categorical_accuracy'])

but same warning.

[philipperemy]

If it's just a warning it might be fine. Maybe upgrade or downgrade your libs (TF+Keras)

[ohilo12]

Do you get the same warning when you run my code? I'm not sure if this warning should be ignored. If I understand it correctly, it means that some of the gradients cannot be calculated, which means that some connections in the network are not connected correctly.

[ohilo12]

I've played a bit with the tf/keras versions, but the warning keeps coming up. However, I have received a more detailed warning:

"WARNING:tensorflow:Gradients do not exist for variables ['tcn_9/residual_block_0/matching_conv1D/kernel:0', 'tcn_9/residual_block_0/matching_conv1D/bias:0'] when minimizing the loss. If you're using model.compile(), did you forget to provide a loss argument?"

[ohilo12]

Another observation: If I add 2 dilation (so 2 convolution layers) the warnings disappear: ` # Modell.add( TCN(nb_filters=2, kernel_size=91, nb_stacks=1, dilations=[1], padding='causal',

use_skip_connections=True, dropout_rate=0.0, return_sequences=True, activation='relu') )`

Modell.add( TCN(nb_filters=2, kernel_size=91, nb_stacks=1, dilations=[1,2], padding='causal', use_skip_connections=True, dropout_rate=0.0, return_sequences=True, activation='relu') )

[philipperemy]

@ohilo12 that's interesting. I've never encountered this error but I haven't used the lib recently. It's probably better to add more than 1 dilation layer. The purpose of a TCN is to go do deep in this dimension. You might have encountered an edge case. But even though, I would not see why this warning happens.

[ohilo12]

What would be interesting: Does it only happens on my PC/environment? Can you try to run it, too?