The batch sizes used by predict and extrapolate functions depend
heavily on the available hardware. I saw a 4x speedup by increasing
one batch size from 10k to 30k, but I would expect other machines and
datasets to experience different payoffs.
Ideally, the batch sizes should be set as env vars that can be passed in.
The batch sizes used by predict and extrapolate functions depend heavily on the available hardware. I saw a 4x speedup by increasing one batch size from 10k to 30k, but I would expect other machines and datasets to experience different payoffs.
Ideally, the batch sizes should be set as env vars that can be passed in.