Use hub.text_embedding_column together with tf.keras.layers.DenseFeatures

tensorflow / hub

A library for transfer learning by reusing parts of TensorFlow models.

https://tensorflow.org/hub

Apache License 2.0

3.48k stars 1.66k forks source link

Use hub.text_embedding_column together with tf.keras.layers.DenseFeatures #333

Closed dfalbel closed 3 years ago

dfalbel commented 5 years ago

Here's a small reproducible code with TF 2.0:

import tensorflow as tf
import tensorflow_hub as hub
feature_columns = [hub.text_embedding_column(key = "text", module_spec = "https://tfhub.dev/google/elmo/2")]
embedding = tf.keras.layers.DenseFeatures(feature_columns = feature_columns)

Errors with:

ValueError: Items of feature_columns must be a FeatureColumn. Given (type <class 'tensorflow_hub.feature_column._TextEmbeddingColumn'>): _TextEmbeddingColumn(key='text', module_spec=<tensorflow_hub.native_module._ModuleSpec object at 0x7f8efcd58f60>, trainable=False).

Here's the same code in colab:

https://colab.research.google.com/drive/1MDFr9fhUYO3QSDayVrBjJBehMzZH8Otq

vbardiovskyg commented 5 years ago

Hi Daniel,

making hub feature_column compatible with the FeatureColumnV2 (FeatureColumn as opposed to _FeatureColumn) is a relatively new feature, a crucial part is implemented here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/feature_column/feature_column_v2.py#L312.

The reason why it fails for you is simply that the feature didn't get into TF2.0 beta release. Trying with !pip install tf-nightly-2.0-preview should work.

dfalbel commented 5 years ago

Thanks very much @vbardiovskyg ! I wonder if it's just the daily buiuld that is not working, but I get a different error when I try to use the created layer:

RuntimeError: variable_scope dense_features_9/text_hub_module_embedding/module/ was unused but the corresponding name_scope was already taken.

Again, this is the complete code:

import tensorflow as tf
import tensorflow_hub as hub
feature_columns = [hub.text_embedding_column(key = "text", module_spec = "https://tfhub.dev/google/nnlm-en-dim128/1")]
embedding = tf.keras.layers.DenseFeatures(feature_columns = feature_columns)
input = {'text': tf.keras.layers.Input(shape = [1,], dtype = 'string')}
output = embedding(input)

And the colab link: https://colab.research.google.com/drive/1MDFr9fhUYO3QSDayVrBjJBehMzZH8Otq I tried with other models in thub.dev too and got the same error.