How to get the outputs of albert model

[iamweiweishi]

Hi, I am tring to get the outputs of the albert model given an input,

` def load_pretrained_albert():

albert_dir = "albert_base"
model_params = bert.params_from_pretrained_ckpt(albert_dir)
l_bert = bert.BertModelLayer.from_params(model_params, name="albert")

# use in Keras Model here, and call model.build()
max_seq_len = 128

l_input_ids = Input(shape=(max_seq_len,), dtype='float32', name="l_input_ids")

output = l_bert(l_input_ids)
pooled_output = AveragePooling1D(pool_size=max_seq_len, data_format="channels_last")(output)
pooled_output = Flatten()(pooled_output)   # poooled_output: [batch_size, embedding_dimension=768]

model = Model(inputs=[l_input_ids], outputs=[pooled_output])
model.build(input_shape=(None, max_seq_len))

l_bert.embeddings_layer.trainable = False

bert_ckpt_file = os.path.join(albert_dir, "model.ckpt-best")
bert.load_albert_weights(l_bert, bert_ckpt_file)

vocab_file = os.path.join(albert_dir, "vocab_chinese.txt")
tokenizer = bert.albert_tokenization.FullTokenizer(vocab_file=vocab_file)

return model, tokenizer

albert_model, tokenizer = load_pretrained_albert()

tokens = tokenizer.tokenize(u"你好世界") token_ids = tokenizer.convert_tokens_to_ids(tokens) print(token_ids)

print(albert_model(inputs=np.array([token_ids]))) `

But, the final output is 'tf.Tensor([], shape=(768, 0), dtype=float32)' How to fetch the output correctly? Another question is, could I get the K, V values of the last layer? Thank you.