tf.Keras Performance Degredation over TF versions

[ristllin]

Hi,

I am running a BI.LSTM model using Keras over CPU (attached the model configuration). After bumping up versions I noticed a significant performance degredation. When running a profiler I noticed the change originates in a deep TF function. I isolated the upgrade to TensorFlow. Upgrading TF from version 2.7.0 to 2.15.0 (minor adjustments required in code and in packages) and was surprised to see a degradation of around 70% in inference performance.

I immediately assumed it is some configuration default that changed over the versions or some new configuration that was added which I probably misconfigured. So, in an attempt to find in which version the degredation happened I ran a statistical test on all versions from 2.7.0 to 2.15.0 and saw the degradation is quite gradual (attached the performance graph). Indicating it's most likely a more systematic problem.

A 70% performance degradation in inference is very significant, though I couldn't find any reports or articles about this topic. Am I missing some configuration\ something-else or is this a known fact?

I would appreciate your help in unraveling this. If you require additional information please let me know.

Model Configuration (inference on CPU): `

max_len = 50  # max len of each sentance
trunc_type = "post"
padding_type = "post"
oov_tok = "<OOV>"  # replace words not in vocab with this token
vocab_size = 500000  # max vocab size
embeding_dim = 34
n_lstm = 12
drop_lstm = 0.45

vectorizer = Tokenizer(num_words=BiLSTMModel.vocab_size, char_level=False, oov_token=BiLSTMModel.oov_tok)
#Model Architecture
self.model = Sequential()
self.model.add(Embedding(BiLSTMModel.vocab_size, BiLSTMModel.embeding_dim, input_length=BiLSTMModel.max_len))
self.model.add(Bidirectional(LSTM(BiLSTMModel.n_lstm, dropout=BiLSTMModel.drop_lstm, return_sequences=False)))
self.model.add(Dense(1, activation='sigmoid'))
self.model.compile(loss='binary_crossentropy',
                   optimizer='adam',
                   metrics=['accuracy'])
`
![image](https://github.com/keras-team/keras/assets/32160010/3357cd68-3f8a-43f0-a4bf-fbf8990577f4)

[sachinprasadhs]

Hi, Keras is updated to Keras 3 to support multi backend frameworks, could you please install Keras 3 and let us know your observations.

For migrating your code from Keras 2 to Keras 3, please refer the guide here https://keras.io/guides/migrating_to_keras_3/

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No