guardian / grid

The Guardian’s image management system
https://www.theguardian.com/info/developer-blog/2015/aug/12/open-sourcing-grid-image-service
Apache License 2.0
1.43k stars 119 forks source link

handle non-ASCII chars in usage CSV email attachments #4294

Closed twrichards closed 3 days ago

twrichards commented 3 days ago

What does this change?

Recent usage emails contained a rare non-breaking space character which broke quotas. We tackle this by dropping non-ASCII characters. Added a test containing CSV which broke things. Also corrected/improved logging of such errors.

## How should a reviewer test this change? The added unit test covers the problematic scenario pretty closely (given the csv content was taken from the email which started breaking stuff). ## How can success be measured? We can ingest usage emails more robustly/flexibly. ## Who should look at this?

Tested? Documented?

github-actions[bot] commented 3 days ago

Deploy build 12585 to TEST

All deployment options - [Deploy build 12585 to TEST](https://riffraff.gutools.co.uk/deployment/deployAgain?project=media-service%3A%3Agrid%3A%3Aall&build=12585&stage=TEST&updateStrategy=MostlyHarmless&action=deploy) - [Deploy parts of build 12585 to TEST by previewing it first](https://riffraff.gutools.co.uk/preview/yaml?project=media-service%3A%3Agrid%3A%3Aall&build=12585&stage=TEST&updateStrategy=MostlyHarmless)

From guardian/actions-riff-raff.

prout-bot commented 3 days ago

Seen on auth, usage, image-loader, metadata-editor, thrall, leases, cropper, media-api, kahuna (merged by @twrichards 8 minutes and 35 seconds ago) Please check your changes!

prout-bot commented 3 days ago

Seen on collections (merged by @twrichards 8 minutes and 49 seconds ago) Please check your changes!