search

bryanlimy / tf2-transformer-chatbot

Transformer Chatbot in TensorFlow 2 with TPU support.
MIT License
130 stars 58 forks source link

Predictions performance #7

Closed arthurflor23 closed 5 years ago

arthurflor23 commented 5 years ago

Hi! Thanks for sharing this implementation and the medium / jupyter notebook material (really great!)

I was able to run the entire project here, but realized that the prediction implementation takes a long time to execute if MAX_LENGTH is large and has several batches to run. (really long time)

My question is if you have any suggestions to optimize, or if anyone has already done so. I thought about doing batch predictions instead of single sentences at a time, but I'm not sure if it will work.

Best

bryanlimy commented 5 years ago

Hi, yes we specifically limited the sentence with MAX_LENGTH to save time. Making batch prediction should work, feel free to make a PR :)