NOTE: An initial version of this text was originally displayed in the PsiZ documentation. It is being added here so that it can be permanently referenced.
Motivation
Unfortunately, many embeddings are susceptible to discovering local optima during inference. While the usual tricks help, such as stochastic gradient decent, one often requires multiple restarts with different initializations to be confident in the solution.
Current Solution
In an effort to shield users from the burden of writing restart logic, PsiZ includes a psiz.keras.Restarter object that implements a fit method similar to tf.keras.Model. The state of most TensorFlow objects can be reset using a serialization/deserialization strategy. However, tf.keras.callbacks.Callback does not implement this serialization strategy.
To patch this problem, PsiZ implements a subclass of tf.keras.callbacks.Callback, which adds a :py:meth:reset method.
Remaining Issues
Subclassing tf.keras.callbacks.Callback is considered a burdensome solution because it forces users to create subclasses for all callbacks they want to use with Restarter. This strategy will likely change when a more elegant solution is found.
NOTE: An initial version of this text was originally displayed in the PsiZ documentation. It is being added here so that it can be permanently referenced.
Motivation
Unfortunately, many embeddings are susceptible to discovering local optima during inference. While the usual tricks help, such as stochastic gradient decent, one often requires multiple restarts with different initializations to be confident in the solution.
Current Solution
In an effort to shield users from the burden of writing restart logic, PsiZ includes a
psiz.keras.Restarterobject that implements afitmethod similar totf.keras.Model. The state of most TensorFlow objects can be reset using a serialization/deserialization strategy. However,tf.keras.callbacks.Callbackdoes not implement this serialization strategy.To patch this problem, PsiZ implements a subclass of
tf.keras.callbacks.Callback, which adds a :py:meth:resetmethod.Remaining Issues
tf.keras.callbacks.Callbackis considered a burdensome solution because it forces users to create subclasses for all callbacks they want to use withRestarter. This strategy will likely change when a more elegant solution is found.