wr-403: OpenAI-performance-check

BinaryStudioAcademy / bsa-2023-writorium

5 stars 3 forks source link

Closed canterbery closed 1 year ago

canterbery commented 1 year ago

Highlights:

ChatGPT model 4 on average 2x slower than 3.5-turbo but provides more interesting suggestions
3.5 model spends approximately 35ms to generate one response token
Token generation is the most time-consuming part of a request