Closed pkmital closed 5 years ago
Maybe you can try tensorflow==1.13.0 as: https://medium.com/@NPCollapse/replicating-gpt2-1-5b-86454a7f26af hints:
Tensorflow (I was using version 1.13) is…not perfect.
This is a known bug. I haven't yet had the time to track down the exact cause. Three things you can try are setting the precision to float32, use a GPU instead of a CPU or change the "train_batch_size" and "predict_batch_size" parameters to 1. Some of these seem to fix it sometimes. I will fix this bug when I have the time to actually track down its source.
The bug also shouldn't happen if you predict with a single word.
I got the same error, here is the full output and traceback I got: https://hasteb.in/wilupika.py
Maybe it will be helpful :)
I encountered the same issue working on gpt-2-simple: https://github.com/minimaxir/gpt-2-simple/issues/38
The solution was to subtract the length of the prefix tokens from the maximum length to prevent OOB.
Thanks minimaxir! I've implemented that fix now and think everything should be working. If this problem crops up again for anyone, feel free to reopen this issue.
Hi, I was interested in testing your
PrettyBig
model. I've downloaded the model and edited the PrettyBig.json to point to the downloaded encoder and model paths. When running:python3 main.py --model PrettyBig.eval.json --predict_text "Hello there! My name is"
I get the following error:
Any ideas appreciated. Thanks!