search

faustomorales / vit-keras

Keras implementation of ViT (Vision Transformer)
Apache License 2.0
329 stars 78 forks source link

module 'tensorflow.keras.initializers' has no attribute 'random_normal' #2

Closed anindabitm closed 3 years ago

anindabitm commented 3 years ago

I am trying to use vit-keras in Kaggle and when using it with TPU, the below error is getting generated.

The TF version is Tensorflow version 2.2.0

`--------------------------------------------------------------------------- AttributeError Traceback (most recent call last)

in 1 with tpu_strategy.scope(): # creating the model in the TPUStrategy scope means we will train the model on the TPU ----> 2 model = create_model() 3 model.summary() in create_model() 7 include_top=True, 8 pretrained_top=False, ----> 9 classes=5 10 ) 11 /opt/conda/lib/python3.7/site-packages/vit_keras/vit.py in vit_l32(image_size, classes, activation, include_top, pretrained, pretrained_top) 228 activation=activation, 229 include_top=include_top, --> 230 representation_size=1024 if pretrained and not pretrained_top else None, 231 ) 232 if pretrained: /opt/conda/lib/python3.7/site-packages/vit_keras/vit.py in build_model(image_size, patch_size, num_layers, hidden_size, num_heads, mlp_dim, classes, dropout, activation, include_top, representation_size) 99 y = tf.keras.layers.Reshape((-1, hidden_size))(y) 100 y = layers.ClassToken(name="class_token")(y) --> 101 y = layers.AddPositionEmbs(name="Transformer/posembed_input")(y) 102 for n in range(num_layers): 103 y, _ = layers.TransformerBlock( /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs) 895 # Build layer if applicable (if the `build` method has been 896 # overridden). --> 897 self._maybe_build(inputs) 898 cast_inputs = self._maybe_cast_inputs(inputs) 899 /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in _maybe_build(self, inputs) 2414 # operations. 2415 with tf_utils.maybe_init_scope(self): -> 2416 self.build(input_shapes) # pylint:disable=not-callable 2417 # We must set also ensure that the layer is marked as built, and the build 2418 # shape is stored since user defined build functions may not be calling /opt/conda/lib/python3.7/site-packages/vit_keras/layers.py in build(self, input_shape) 32 self.pe = tf.Variable( 33 name="pos_embedding", ---> 34 initial_value=tf.initializers.random_normal(stddev=0.06)( 35 shape=(1, input_shape[1], input_shape[2]) 36 ), AttributeError: module 'tensorflow.keras.initializers' has no attribute 'random_normal'`
faustomorales commented 3 years ago

Thanks for opening an issue! The traceback points to an attribute error on tf.initializers.random_normal. This works in TensorFlow 2.4.0. Please try upgrading and see if that corrects the issue.

anindabitm commented 3 years ago

Thanks. Works in TF>=2.3.0. Unfortunately for TPU, Kaggle comes with TF 2.2.0. Upgrading doesn't seem to work. Any workaround?