The documentation for Conv1DTranspose does not state that the CPU does not support dilation rates larger than 1

[PhyllisJi]

Issue type

Documentation Bug

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

tf 2.14.0

Custom code

Yes

OS platform and distribution

Ubuntu 20.04

Mobile device

No response

Python version

No response

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

Throw Error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Exception encountered when calling layer 'conv4_mutated' (type Conv1DTranspose).

{{function_node __wrapped__Conv2DBackpropInput_device_/job:localhost/replica:0/task:0/device:CPU:0}} Current CPU implementations do not yet support dilation rates larger than 1. [Op:Conv2DBackpropInput] name: 

Call arguments received by layer 'conv4_mutated' (type Conv1DTranspose):
  • inputs=tf.Tensor(shape=(2, 2048, 64), dtype=float32)

Also, I'm not quite sure why the error message relates to Conv2DBackpropInput

Standalone code to reproduce the issue

import tensorflow as tf
import numpy as np
import os

os.environ['CUDA_VISIBLE_DEVICES'] = ''

def Model_HqTi1yjRFc7Dz1RLSJOvA9XX16R5Wp01(x):
    x = tf.keras.Input(shape=x)
    _x = x
    _zeropadding_x = tf.keras.layers.ZeroPadding1D(padding=(0, 0))(x)
    x = tf.keras.layers.Conv1D(filters=64, kernel_size=1, strides=1, padding="valid", data_format="channels_last", dilation_rate=1, groups=1, use_bias=True, name="conv1")(_zeropadding_x)
    x = tf.keras.layers.BatchNormalization(axis=-1, epsilon=1e-05, momentum=0.9, center=True, scale=True, name="bn1")(x)
    x = tf.nn.relu(x)
    _zeropadding_x = tf.keras.layers.ZeroPadding1D(padding=(0, 0))(x)
    x = tf.keras.layers.Conv1D(filters=64, kernel_size=1, strides=1, padding="valid", data_format="channels_last", dilation_rate=1, groups=1, use_bias=True, name="conv2")(_zeropadding_x)
    x = tf.keras.layers.BatchNormalization(axis=-1, epsilon=1e-05, momentum=0.9, center=True, scale=True, name="bn2")(x)
    x = tf.nn.relu(x)
    _zeropadding_x = tf.keras.layers.ZeroPadding1D(padding=(0, 0))(x)
    x = tf.keras.layers.Conv1D(filters=64, kernel_size=1, strides=1, padding="valid", data_format="channels_last", dilation_rate=1, groups=1, use_bias=True, name="conv3")(_zeropadding_x)
    x = tf.keras.layers.BatchNormalization(axis=-1, epsilon=1e-05, momentum=0.9, center=True, scale=True, name="bn3")(x)
    x = tf.keras.activations.relu(x)
    _zeropadding_x = tf.keras.layers.ZeroPadding1D(padding=(0, 0))(x)
    x = tf.keras.layers.Conv1DTranspose(filters=128, kernel_size=1, strides=1, padding="valid", output_padding=0, data_format="channels_last", dilation_rate=8, use_bias=True, name="conv4_mutated")(x)

    x = x
    model = tf.keras.models.Model(inputs=_x, outputs=x)
    return model

def go():
    with tf.device('/CPU:0'):
        shape = [2, 3, 2048]
        _numpy = np.random.random(shape).astype(np.float32)
        tf_input = tf.convert_to_tensor(_numpy.transpose(0, 2, 1), dtype=tf.float32)
        tf_model = Model_HqTi1yjRFc7Dz1RLSJOvA9XX16R5Wp01(tf_input.shape[1:])
        tf_output = tf_model(tf_input)

go()

[mehtamansi29]

Hi @PhyllisJi-

Thanks for reporting the issue. Here in your code there is multiple errors:

1. tf.nn.relu(x)- need to use tf.keras.activations.relu(x)' as latest tensorflow2.17(keras3.0.6) as kerasTensor output which is not compatible withtf.nn.relu(x`
2. There is no output_padding argument in tf.keras.layers.Conv1DTranspose. Here you can find more detail about Conv1DTranspose API.
3. And last error is causing due to dilation_rate=8 in this line x = tf.keras.layers.Conv1DTranspose(filters=128, kernel_size=1, strides=1, padding="valid", output_padding=0, data_format="channels_last", dilation_rate=8, use_bias=True, name="conv4_mutated")(x). So changing dilation_rate=1 will resolve the error.

Attached gist for the reference as well.

[PhyllisJi]

Hi @PhyllisJi-

Thanks for reporting the issue. Here in your code there is multiple errors:

1. tf.nn.relu(x)- need to use tf.keras.activations.relu(x)' as latest tensorflow2.17(keras3.0.6) as kerasTensor output which is not compatible withtf.nn.relu(x`
2. There is no output_padding argument in tf.keras.layers.Conv1DTranspose. Here you can find more detail about Conv1DTranspose API.
3. And last error is causing due to dilation_rate=8 in this line x = tf.keras.layers.Conv1DTranspose(filters=128, kernel_size=1, strides=1, padding="valid", output_padding=0, data_format="channels_last", dilation_rate=8, use_bias=True, name="conv4_mutated")(x). So changing dilation_rate=1 will resolve the error.

Attached gist for the reference as well.

We use the version is 2.14.0. Why tf.nn.relu is not compatible with tf.keras.activations.relu(x)?

[PhyllisJi]

Hi @PhyllisJi-

Thanks for reporting the issue. Here in your code there is multiple errors:

1. tf.nn.relu(x)- need to use tf.keras.activations.relu(x)' as latest tensorflow2.17(keras3.0.6) as kerasTensor output which is not compatible withtf.nn.relu(x`
2. There is no output_padding argument in tf.keras.layers.Conv1DTranspose. Here you can find more detail about Conv1DTranspose API.
3. And last error is causing due to dilation_rate=8 in this line x = tf.keras.layers.Conv1DTranspose(filters=128, kernel_size=1, strides=1, padding="valid", output_padding=0, data_format="channels_last", dilation_rate=8, use_bias=True, name="conv4_mutated")(x). So changing dilation_rate=1 will resolve the error.

Attached gist for the reference as well. Our code is automatically generated, so it may not look very polished—apologies for that. However, based on your previous response, we identified the following concerns:

1. Compatibility Issues Documentation: If there are any known compatibility issues, the official documentation should contain relevant information to notify users.
2. Unsupported Parameters Handling: If certain parameters are not supported, Keras should raise an exception and provide clear feedback to users, instead of failing silently.
3. Inconsistent Behavior on Different GPUs: We re-ran the same code on a 4090 GPU/CPU, and it produced the expected tf_output without issues. However, when running the same code on a 3090 GPU/CPU, we encountered the issue. This inconsistent behavior across GPU/CPU models raises concerns, and we are now confused about what might be causing this discrepancy.