batching requests to OpenAI when using multiple completions, so we won't hit the rate limit

[urialon]

The rate limits have changed, and we can no longer ask for 32 completions using the same prompt. Instead, let's sample 8 completions N times, such that 8*N >= 32.

[luyug]

lgtm, thanks!