wellcomecollection / platform-infrastructure

:building_construction: Infrastructure for the Wellcome Digital Platform
MIT License
24 stars 8 forks source link

Protect IIIF APIs from bot traffic #419

Closed kenoir closed 6 months ago

kenoir commented 7 months ago

It looks like the IIIF API Cloudfront distribution traffic contributes significantly to AWS costs in the platform account. The pattern of activity is largely inline with bot activity seen on the catalogue API where we added AWS WAF protections to mitigate. See: https://github.com/wellcomecollection/wellcomecollection.org/pull/10533

We should extend these protections to IIIF APIs, ensuring we gauge impact and being careful not to block normal user patterns of behaviour. We should deploy these changes in "count" mode initially to understand impact properly before deploying.

https://github.com/wellcomecollection/platform-infrastructure/pull/418