Idiotically, the prediction loop appends the predictions (the values returned by predict_step in the model...these are logits) to a huge list in CPU memory here,so if you have a long-running prediction you will eventually run out of CPU memory.
A temporary solution is to set --return_predictions False (this actually doesn't break prediction, believe it or not, it just doesn't save them in the big list), but we should make it so this is automatically enabled when running in predict mode. This is arguably also related to #41.
Idiotically, the prediction loop appends the predictions (the values returned by
predict_stepin the model...these are logits) to a huge list in CPU memory here,so if you have a long-running prediction you will eventually run out of CPU memory.A temporary solution is to set
--return_predictions False(this actually doesn't break prediction, believe it or not, it just doesn't save them in the big list), but we should make it so this is automatically enabled when running inpredictmode. This is arguably also related to #41.