Features module

[jimfleming]

TensorFlow briefly experimented with tf.feature_column as part of estimators. While tf.feature_column is not supported by Eager we've experimented with a custom implementation for a few client projects and it worked very well.

Pseudo-code:

class MyModel(tf.keras.Model):
    def __init__(self):
        self.feature_layers = pynr.layers.Featurize({
            'my_scalar': pynr.layers.StandardizeLayer(loc=<loc>, scale=<scale>),
            'my_categorical': pynr.layers.OneHotLayer(depth=<depth>),
            'my_other_categorical': keras.Embedding(bins=<bins>),
        })
    def call(features):
        return self.feature_layers(features)

some_features = {
    'my_scalar': [3.14],
    'my_categorical': [7],
    'my_other_categorical': [13],
}
my_model = MyModel()
my_model(some_features)

Features:

• Semantic: Interpreting dictionaries of features is more intuitive than opaque vectors.
• Flexible: easily remove features from a feature vector without refactoring the whole data pipeline.

Example Handlers:

• One-hot encoding categorical variables.
• Embedding categorical variables (tf.keras.layers.embeddings.Embedding).
• Standardizing scalar variables while persisting the moments with the checkpoint.

FAQ:

• How are handlers different from layers? They are implemented as layers that typically accept a single feature.

[wenkesj]

I like this. Would it be worth extending featurize as a layer? As in:

class MyModel(tf.keras.Model):
    def __init__(self):
        self.feature_layers = pynr.layers.Featurize({
            'my_scalar': pynr.layers.StandardizeLayer(loc=<loc>, scale=<scale>),
            'my_categorical': pynr.layers.OneHotLayer(depth=<depth>),
            'my_other_categorical': keras.Embedding(bins=<bins>),
        })
    def call(features):
        return self.feature_layers(features)

This removes the need for wrapping

[jimfleming]

Oh, I think I like that. I'll try it out in my implementation.

EDIT: Yep, that works very well.

[wenkesj]

Closing in favor of #6