search

ethz-spylab / satml-llm-ctf

Code used to run the platform for the LLM CTF colocated with SaTML 2024
https://ctf.spylab.ai
MIT License
25 stars 6 forks source link

Availability of API #52

Closed epistoteles closed 10 months ago

epistoteles commented 10 months ago

Is it just me or has the API become unresponsive to the point where even setting the timeout to several minutes doesn't help?

s-zanella commented 10 months ago

It's not just you.

s-zanella commented 10 months ago

I'm also experiencing a very high latency in some API endpoints at the moment. For instance, requests to /api/v1/chat/attack/create consistently take upwards of 7 minutes to complete compared to seconds days ago. Understandably, the service must be under heavy load at the moment, but come tomorrow this will be even worse.

dedeswim commented 10 months ago

Hi, thanks a lot for pointing this out. I will work on a fix for this tonight.

dedeswim commented 10 months ago

We doubled the capacity of the server that runs the CTF platform, I hope it's going to be enough for the load we expect tomorrow

adrgs commented 10 months ago

Could the capacity be increased again? The API is pretty unusable right now:

Failed to send new message: 500 Server Error: Internal Server Error for url: https://ctf.spylab.ai/api/v1/chat/attack/<id>/new_message

It's been like that for the past ~30 minutes.

dedeswim commented 10 months ago

Let's discuss LLM provider API-related issues on #59

dpaleka commented 10 months ago

We released an announcement. Please rereport any errors that are not covered by the announcement, thanks!

dedeswim commented 10 months ago

Going to close this issue as it looks like the set-up managed to survive what was likely the highest load we expected. Feel free to reopen if the issue comes up again!