Closed dusk666 closed 3 years ago
python version:3.7.6 tensoflow version:tensorflow-gpu 2.2.0 adabelief-tf :0.1.0 model using tf.keras.api
Hi, does the adabelief-tf support tensorflow 1, for instance, tensorflow-gpu 1.14? I found the adabelief-tf uninstalled my tensorflow 1 and installed tensorflow 2. Thanks in advance.
Hi, does the adabelief-tf support tensorflow 1, for instance, tensorflow-gpu 1.14? I found the adabelief-tf uninstalled my tensorflow 1 and installed tensorflow 2. Thanks in advance.
I think u need tensorflow version>=2.1.0 , because neither adabelief-tf 0.0.1 nor 0.1.0 work in my tensorflow 2.0.0 env. BUT we can run it in tensorflow 2.2 or 2.3.
@dusk666 The message is to guide you to set the default arguments for AdaBelief, popping up once you create AdaBeliefOptimizer.
For version, the newest pypi_package cache file of adabelief_tf0.1.0 is not update yet. Currently, please use the following commands to install:
git clone https://github.com/juntang-zhuang/Adabelief-Optimizer.git
pip install ./Adabelief-Optimizer/pypi_packages/adabelief_tf0.1.0
@louislbc Sorry, the package currently only supports tensorflow 2.
@louislbc @dusk666 We have released version 0.2.0, please try the new version by pip install adabelief-tf==0.2.0
, this version should be slightly faster than 0.1.0.
@louislbc Please see https://github.com/juntang-zhuang/Adabelief-Optimizer/tree/update_0.3.0/pypi_packages/adabelief_tf0.3.0, there are two subfolders, one for tf1 the other for tf2. It's not pushed to pip yet, so please download the source and try. Thanks for the implementation by @cryu854.
HI! I had some trouble using Adambelief in a simple lstm training. What could be the reason for this? CODE: from adabelief_tf import AdaBeliefOptimizer tf.keras.backend.clear_session() multivariate_lstmA = tf.keras.models.Sequential([ LSTM(100, input_shape=input_shape, return_sequences=True), Flatten(), Dense(200, activation='relu'), Dropout(0.1), Dense(1) ]) model_checkpoint = tf.keras.callbacks.ModelCheckpoint( 'multivariate_lstmA.h5', monitor=('val_loss'), save_best_only=True) optimizer = AdaBeliefOptimizer(learning_rate=1e-3, epsilon=1e-14, rectify=False) multivariate_lstmA.compile(loss=loss, optimizer=optimizer, metrics=metric)
RESULT: Please check your arguments if you have upgraded adabelief-tf from version 0.0.1. Modifications to default arguments: eps weight_decouple rectify
adabelief-tf=0.0.1 1e-08 Not supported Not supported Current version (0.1.0) 1e-14 supported default: True For a complete table of recommended hyperparameters, see https://github.com/juntang-zhuang/Adabelief-Optimizer