zeahmed
commented
6 years ago Most of the text models in TensorFlow (and in any other DNN platform in general) uses an embedding layer to handle text. This is against the bag-of-word model approach where a vector is formed for the words/characters in the text. The indices of the vector refer to the words/characters and the values represent the TF/TF-IDF or any other scores computed for words/characters.
The bag-of-word model requires vectors to be represented in sparse format because number of words/characters appearing in the text is very large. However, when using models with embedding layers, sparse format is not needed because input to embedding layers is not typically that large. So, we are fine with dense format.
However, when working with text models, I found out following issues.
- String as input/output is not supported at all in TensorFlowTransform. TensorFlowSharp also has limited functionality in this regard.
- The modes that are not based in string inputs are composed of two set of resources.
- Model file
- Text resources such as dictionary to convert text items (words, characters) into vector of integers.
- The conversion is a pre-processing step so we need to find out a way to convert text items into vector of integers. I tried using
TermLookupTransformandTermTransformboth did not work. - For the models that accept fixed length text input, we need to find out a way to trim and pad vectors so that appropriate sized vector can passed to TensorFlow. Variable sized inputs should not have an issue.
I currently don't see any issue with retrieving outputs from TensorFlow. I will write more if I encounter other issues.
asthana86
Ivanidzo4ka
We should know how TF handles text inputs, and whether it supports sparse inputs.