search

steineggerlab / foldseek

Foldseek enables fast and sensitive comparisons of large structure sets.
https://foldseek.com
GNU General Public License v3.0
696 stars 92 forks source link

API access rate upper limit. #153

Open johnnytam100 opened 1 year ago

johnnytam100 commented 1 year ago

Hi foldseek. I would like to know the access rate upper limit of the foldseek server. At what frequency of request from an IP address will foldseek server send out warning message email for banning access from that IP address?

martin-steinegger commented 1 year ago

We currently do not have proper rate limits setup, which we will in the next days. If you cannot access the API right now then you might be blocked. We did block one IP yesterday that flooded our API. Once we setup proper rate limits we will unblock it.

johnnytam100 commented 1 year ago

Hi Martin, thanks for the quick response. Would appreciate if foldseek can state the upper limit clealy or directly block for submission > the limit instead of sending an email at the institutional level. Relatedly, please investigate the difference between standalone and web server https://github.com/steineggerlab/foldseek/issues/146#issuecomment-1608603100 or it will be hard to use the standalone version.

martin-steinegger commented 1 year ago

Yes, as mentioned above we are working on proper rate limits. We do not send emails to institutions about the usage. However, I did contact one collaborator at Uni of T to ask if they might cause the high traffic. I am sorry if this caused any inconvenience. Please consider asking the providers of academic webservers beforehand about large scale usage. Most academic group do not have a lot of resources at hand.

milot-mirdita commented 11 months ago

I think we have set rate limits. You can submit up to ~90 search jobs per hour to the API. I think we can further raise this. How much would you need?

This will just result in the HTTP error code429, no other warning is sent out. You can repeat the requests (with a short wait in-between) until it returns a HTTP code of 200.

johnnytam100 commented 10 months ago

Hi @milot-mirdita , I don't need the API if this issue -> https://github.com/steineggerlab/foldseek/issues/146#issuecomment-1647659850 can be resolved.

milot-mirdita commented 10 months ago

--cluster-search is still a bit buggy. We will release a new version of both foldseek and the prebuilt PDB where this will work correctly. For now you can just drop the parameter and either just search against the pdb for the cluster representatives or pdb_seq to search against all.

milot-mirdita commented 10 months ago

The PDB and AFDB that you can download with the latest Foldseek version don't have any of the mentioned issues anymore. You can use the --cluster-search 1 with it.