Fail to wrap albert output layer using tfhub.module

[acmilannesta]

I used albert_base_v2 and tfhub on python3.6 and tensorflow 1.15.0.

I tried to wrap the albert output layer with a pooling layer. And it gives the following erros:

---------------------------------------------------------------------------
FailedPreconditionError                   Traceback (most recent call last)
<ipython-input-6-d725e62752e5> in <module>()
----> 1 dense = keras.layers.Lambda(lambda x: x[:, 0])(albert_outputs)

7 frames
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py in __call__(self, *args, **kwargs)
   1470         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1471                                                self._handle, args,
-> 1472                                                run_metadata_ptr)
   1473         if run_metadata:
   1474           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

FailedPreconditionError: 2 root error(s) found.
  (0) Failed precondition: Error while reading resource variable module/bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/output/dense/bias from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/module/bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/output/dense/bias)
     [[{{node module_apply_tokens/bert/encoder/transformer/group_0_11/layer_11/inner_group_0/ffn_1/intermediate/output/dense/add/ReadVariableOp}}]]
  (1) Failed precondition: Error while reading resource variable module/bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/output/dense/bias from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/module/bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/output/dense/bias)
     [[{{node module_apply_tokens/bert/encoder/transformer/group_0_11/layer_11/inner_group_0/ffn_1/intermediate/output/dense/add/ReadVariableOp}}]]
     [[module_apply_tokens/bert/encoder/transformer/group_0_11/layer_11/inner_group_0/ffn_1/intermediate/output/dense/add/ReadVariableOp/_1]]
0 successful operations.
0 derived errors ignored.

Here is my code:

    from tensorflow import keras
    import tensorflow_hub as hub
    q_in = keras.layers.Input(shape=(None,), dtype=tf.int32, name="q_input_word_ids")
    q2_in = keras.layers.Input(shape=(None,), dtype=tf.int32, name="q_input_masks")
    q3_in = keras.layers.Input(shape=(None,), dtype=tf.int32, name="q_segment_ids")

    albert_module = hub.Module('https://tfhub.dev/google/albert_base/2', trainable=True)
    albert_inputs = dict(input_ids=q_in, input_mask=q2_in, segment_ids=q3_in)
    albert_outputs = albert_module(albert_inputs, signature="tokens", as_dict=True)["sequence_output"]

Up till now everything runs fine. But when I run the following, it gives the error above.

    pooled= keras.layers.GlobalAveragePooling1D()(albert_outputs)

Also I tried wrap it in tf.Session, but it didn't work:

with tf.Session() as session:
    keras.backend.set_session(session)
    session.run(tf.global_variables_initializer())  
    session.run(tf.tables_initializer())
    dense =  keras.layers.GlobalAveragePooling1D()(albert_outputs)

Any helps are appreciated!

[rmothukuru]

@acmilannesta, Can you please let us know the TF Hub Version you are using. Thanks!

[acmilannesta]

I'm using tfhub 0.7.0.

rmothukuru notifications@github.com 于2019年12月16日周一 上午12:37写道：

@acmilannesta https://github.com/acmilannesta, Can you please let us know the TF Hub Version you are using. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/hub/issues/447?email_source=notifications&email_token=ALL6NUUNKT5JU226PBSFHILQY4HZRA5CNFSM4J256DYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG5SLNI#issuecomment-565913013, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALL6NUQHNOJH35SYEGXRIJ3QY4HZRANCNFSM4J256DYA .

[arnoegw]

Hi @acmilannesta, you'r using the old hub.Module class as if if were a Keras Layer type. That's not supported; please review tensorflow.org/hub/migration_tf2 and its links to the new and old APIs.

https://tfhub.dev/google/albert_base/2 comes in the old hub.Module format. With tensorflow-hub 0.7.0, this can also be used in the new hub.KerasLayer API, but not with trainable=True.

[arnoegw]

I haven't heard back, so I suppose this has been solved.