Closed sanchit-gandhi closed 2 months ago
To annotate 10k hours of description data with Mistral Instruct 7B v0.2 and an 8x H100 80GB node:
=> a 2x difference, which primarily comes from the continuous batching implemented by TGI.
To annotate 10k hours of description data with Mistral Instruct 7B v0.2 and an 8x H100 80GB node:
=> a 2x difference, which primarily comes from the continuous batching implemented by TGI.