NVIDIA-Merlin / models

Merlin Models is a collection of deep learning recommender system model reference implementations
https://nvidia-merlin.github.io/models/main/index.html
Apache License 2.0
262 stars 50 forks source link

Disable keras optimizer module change in SOK with `use_legacy_optimizer` #1064

Closed oliverholworthy closed 1 year ago

oliverholworthy commented 1 year ago

Fixes errors we were seeing from accesing tf.keras.optimizers.legacy due to the redefinition of the optimizers as a side-effect of the import of sparse_operation_kit.experiment

AttributeError: module 'keras.api._v2.keras.optimizers.legacy' has no attribute 'legacy'

Implementation Details

A change to redefine the optimizers module in the spare_operation_kit was introduced in this change in HugeCTR last week https://github.com/NVIDIA-Merlin/HugeCTR/commit/2df20eb10c54f2f1c140371d4f3a792515bc5fa2

A follow-up change has been made in HugeCTR (release 23.04) which adds the optional parameter use_legacy_optimizer to the init function and moves the optimizers module mutation from global scope to within the body of this function: https://github.com/NVIDIA-Merlin/HugeCTR/commit/8ec7fa135bbb8074d6b9b84de64c2a8a45c9762c

We need to pass this parameter as False because the default is True which will activate the redefinition of the tf.keras.optimizers module. This optimizers module is required to be able to refer to optimizers as string values with a lookup by name with the get function.

Before

import tensorflow as tf
​
tf.keras.optimizers
# => <module 'keras.api._v2.keras.optimizers' from '/usr/local/lib/python3.8/dist-packages/keras/api/_v2/keras/optimizers/__init__.py'>
​
import sparse_operation_kit.experiment
​
tf.keras.optimizers
# => <module 'keras.api._v2.keras.optimizers.legacy' from '/usr/local/lib/python3.8/dist-packages/keras/api/_v2/keras/optimizers/legacy/__init__.py'>

tf.keras.optimizers.legacy
# => Raises AttributeError: module 'keras.api._v2.keras.optimizers.legacy' has no attribute 'legacy'

After

import tensorflow as tf
​
tf.keras.optimizers
# => <module 'keras.api._v2.keras.optimizers' from '/usr/local/lib/python3.8/dist-packages/keras/api/_v2/keras/optimizers/__init__.py'>
​
import sparse_operation_kit.experiment
​
tf.keras.optimizers
# => <module 'keras.api._v2.keras.optimizers' from '/usr/local/lib/python3.8/dist-packages/keras/api/_v2/keras/optimizers/__init__.py'>

tf.keras.optimizers.legacy
# => <module 'keras.api._v2.keras.optimizers.legacy' from '/usr/local/lib/python3.8/dist-packages/keras/api/_v2/keras/optimizers/legacy/__init__.py'>
github-actions[bot] commented 1 year ago

Documentation preview

https://nvidia-merlin.github.io/models/review/pr-1064