Performance with CPU TF & zero vector output

[daoudChami]

Hello,

For those of you who had an issue with the mdl.transform returning a zero vector, here how I managed to solve it : First, it seems that the input text must be at least 64 characters long (see nsteps variable in encoder.py). Also, the html.unescape function returns bytearray like data. If you replace it with HTMLParser().unescape, you will manipulate strings and this will probably cause an exception in the batch_pad function.

And when I finnaly managed to figure this out, I realized that the transformation was extreamly slow on tiny inputs (5 minutes on 10 imdb review).

So, my question is : Is it caused by my fix or is it caused by the cpu implementation ?

Thank you

[Newmu]

The model is pretty compute intensive. Larger batches should help speed things up per example to avoid wasting computation on padding and underutilization so it is not totally un-believable that it would take 5 minutes to process 1000 character long imdb reviews on CPU. For reference it takes about 20 minutes on a high-end GPU to process 25,000 imdb reveiws and this is using efficient batching.