tensorlab / tensorfx

TensorFlow framework for training and serving machine learning models
Apache License 2.0
196 stars 41 forks source link

Implement a schema field of type tokens #18

Open nikhilk opened 7 years ago

nikhilk commented 7 years ago

This is for representing tokenized set of values (eg. comma separated strings or numbers). The ideas is as part of data preparation, one can convert text into a set of tokens (after doing word breaking, stop-word removal, stemming etc.)