How long did it take you to do these evaluation?

gersteinlab / MedAgents

188 stars 21 forks source link

How long did it take you to do these evaluation? #4

Closed hanrthu closed 5 months ago

hanrthu commented 6 months ago

Hello, I am running MedAgents' code and find it slow to get one answer (One question takes about 2~3 minutes). So I wonder if there is any method to speed it up. And how do you evaluate your methods on those datasets? How much time did it take?

tangxiangru commented 5 months ago

If you use ChatGPT API calling it will be much slower. However, we are using Azure and we set up 300K TPM tokens per minute for gpt-35-turbo. It would be 30-50s per example.