web-arena-x / webarena

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
https://webarena.dev
Apache License 2.0
633 stars 90 forks source link

Postmill rate limit. #120

Closed afourney closed 2 months ago

afourney commented 3 months ago

We're hitting a rate limit when testing the Postmill/reddit questions. Is there a way to disable this?

image

shuyanzhou commented 3 months ago

@robert1003 can you explain how to reset the rate limit for trust users?

cc. @frankxu2004

frankxu2004 commented 3 months ago

Hi @afourney thank you for your interest. We are trying to update a new version of the postmill image by fixing some issues here, but to temporarily fix it:

Go inside the postmill docker, and make 'MarvelsGrantMan136' user a trusted user: UPDATE users SET trusted = true WHERE username = 'MarvelsGrantMan136';. Non-trusted user can only make 3 posts per hour while trusted user can make 15 posts per five minutes.

Hope it helps! If you want to try out the new version (not yet finalized but has several fixes inside already), you could download the tar image here: https://github.com/web-arena-x/webarena/issues/82#issuecomment-1873150016

frankxu2004 commented 3 months ago

As for resetting the rate limit, I think the simplest way is to reset the whole docker container with the provided image, and apply the above fix.

frankxu2004 commented 3 months ago

Actually to kill the rate limit completely, there's a hack. We left the website mostly as-is for it to mimic real websites as much as possible (e.g. real websites usually have anti-spam or anti-bot mechanism).

Nonetheless, if you think this is too much of a hassle, then go inside the docker for postmill, find source code for the website:

src/DataObject/CommentData.php and src/DataObject/SubmissionData.php and possibly src/DataObject/UserData.php

Delete all those decorators starting with @RateLimit(period=.....

Restart the docker container to reload the php code.

afourney commented 3 months ago

Fantastic. Thanks!