search

ethz-spylab / satml-llm-ctf

Code used to run the platform for the LLM CTF colocated with SaTML 2024
https://ctf.spylab.ai
MIT License
25 stars 6 forks source link

Availability of API #52

Closed epistoteles closed 9 months ago

epistoteles commented 9 months ago

Is it just me or has the API become unresponsive to the point where even setting the timeout to several minutes doesn't help?

s-zanella commented 9 months ago

It's not just you.

s-zanella commented 9 months ago

I'm also experiencing a very high latency in some API endpoints at the moment. For instance, requests to /api/v1/chat/attack/create consistently take upwards of 7 minutes to complete compared to seconds days ago. Understandably, the service must be under heavy load at the moment, but come tomorrow this will be even worse.

dedeswim commented 9 months ago

Hi, thanks a lot for pointing this out. I will work on a fix for this tonight.

dedeswim commented 9 months ago

We doubled the capacity of the server that runs the CTF platform, I hope it's going to be enough for the load we expect tomorrow

adrgs commented 9 months ago

Could the capacity be increased again? The API is pretty unusable right now:

Failed to send new message: 500 Server Error: Internal Server Error for url: https://ctf.spylab.ai/api/v1/chat/attack/<id>/new_message

It's been like that for the past ~30 minutes.

dedeswim commented 9 months ago

Let's discuss LLM provider API-related issues on #59

dpaleka commented 9 months ago

We released an announcement. Please rereport any errors that are not covered by the announcement, thanks!

dedeswim commented 9 months ago

Going to close this issue as it looks like the set-up managed to survive what was likely the highest load we expected. Feel free to reopen if the issue comes up again!