SPIKE: Blocking Bots - Githubissues

sshetenhelm commented 5 months ago

Story As per Mike on 2024.06.17 - "DCS seems very slow at the moment. We might think about a more targeted effort to block bots, we added rate limiting in Quicksearch and saw a decrease in harvesting traffic."

How are we blocking bots currently? How can we further block bots? Would blocks affect the actual user experience?

This ticket can be used either (a) as the "block bots" ticket or (b) as a spawn for other tickets, depending on what we would like to do.

Acceptance

[ ] Create new AWS WAF ruleset for DCS
[ ] Copy over TIFF rate limiting rule (I think this may be broken now)
[ ] Turn WAF Bot Control in monitoring mode so that we can develop a baseline for bot traffic (types, amount, etc.)

sshetenhelm commented 5 months ago

@mikeapp made this based off your comments in Teams last week. Do you have any strategies in mind? Happy to keep this as a spike, but also happy to hear any suggestions if you have them.

mikeapp commented 5 months ago

@sshetenhelm Suggested some acceptance criteria above. I suggest we monitor first, then decide what to block.

yalelibrary / YUL-DC

SPIKE: Blocking Bots #2862