search

ethz-spylab / satml-llm-ctf

Code used to run the platform for the LLM CTF colocated with SaTML 2024
https://ctf.spylab.ai
MIT License
23 stars 5 forks source link

API experiencing high latency at the moment #53

Closed s-zanella closed 6 months ago

s-zanella commented 7 months ago

I'm experiencing a very high latency in some API endpoints at the moment. For instance, requests to /api/v1/chat/attack/create consistently complete in over 3 minutes compared to seconds days ago. Understandably, the service must be under heavy load at the moment, but come tomorrow this will be even worse.

s-zanella commented 7 months ago

Duplicate of #52

s-zanella commented 6 months ago

Any API call is taking longer than 10 minutes at the moment.

dedeswim commented 6 months ago

Are these using which model?

s-zanella commented 6 months ago

Llama-2. But even API calls that don't trigger queries to Together AI are failing.

s-zanella commented 6 months ago

Nevermind, it's already past the deadline.

dedeswim commented 6 months ago

We apologise for the issues. We further increased the server capacity three days ago, hoping that it would have been enough to handle last minute submissions, but it clearly wasn't enough

s-zanella commented 6 months ago

No worries. I really appreciate all the effort you put into running the competition and that's not going to be tarnished by last minute issues. It was great fun! Thanks a lot!

dedeswim commented 6 months ago

Thank you for the participation, and congrats on the second place! We're happy you had fun!