Switch to Photon for thumbnail generation

sarayourfriend commented 1 year ago

Fixes

Fixes #979 by @AetherUnbound

Description

Now that the WordPress.com Photon instance supports an authentication header to allow us to forward query params, we're able to use it for thumbnail generation.

Note: This PR is still missing:

Add/move thumbnail unit tests to match new location catalog.api.utils.photon module

Testing Instructions

Checkout the branch and just recreate. Test thumbnails and confirm they work.

If you have access to the Openverse secrets, you can add the PHOTON_AUTH_KEY variable to your api/.env file to enable query forwarding.

I am using the following additional sample data to be able to test the query forwarding:

diff --git a/sample_data/sample_images.csv b/sample_data/sample_images.csv
index 79a691a9..ecbb61c6 100644
--- a/sample_data/sample_images.csv
+++ b/sample_data/sample_images.csv
@@ -4999,3 +4999,4 @@ d4307b61-bd39-44bb-a3cd-aa760029d051,2021-12-15 20:48:57.914727+00,2021-12-15 20
 954a4dce-12b3-4323-86f3-8af7177635d6,2021-12-15 20:48:57.914727+00,2021-12-15 20:48:57.914727+00,provider_api,stocksnap,stocksnap,UCWLBKL1OL,https://stocksnap.io/photo/UCWLBKL1OL,https://cdn.stocksnap.io/img-thumbs/960w/UCWLBKL1OL.jpg,https://cdn.stocksnap.io/img-thumbs/280h/UCWLBKL1OL.jpg,2574,3218,1088013,cc0,1.0,Matt Bango,https://mattbango.photo,Music Guitar Photo,"{""license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/"", ""downloads_raw"": ""5"", ""favorites_raw"": ""0"", ""page_views_raw"": ""225"", ""raw_license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/""}","[{""name"": ""music"", ""provider"": ""stocksnap""}, {""name"": ""guitar"", ""provider"": ""stocksnap""}, {""name"": ""play"", ""provider"": ""stocksnap""}, {""name"": ""string"", ""provider"": ""stocksnap""}, {""name"": ""man"", ""provider"": ""stocksnap""}, {""name"": ""hand"", ""provider"": ""stocksnap""}, {""name"": ""musician"", ""provider"": ""stocksnap""}, {""name"": ""event"", ""provider"": ""stocksnap""}, {""name"": ""playing"", ""provider"": ""stocksnap""}, {""name"": ""instrument"", ""provider"": ""stocksnap""}, {""name"": ""band"", ""provider"": ""stocksnap""}, {""name"": ""electric"", ""provider"": ""stocksnap""}, {""name"": ""live"", ""provider"": ""stocksnap""}, {""name"": ""stage"", ""provider"": ""stocksnap""}, {""name"": ""sitting"", ""provider"": ""stocksnap""}, {""name"": ""person"", ""provider"": ""stocksnap""}, {""name"": ""player"", ""provider"": ""stocksnap""}, {""name"": ""group"", ""provider"": ""stocksnap""}, {""name"": ""performance"", ""provider"": ""stocksnap""}]",f,2021-12-15 20:48:57.914727+00,f,jpg,photograph,0.19767441860465113
 20c931c6-c0d6-4395-a2ee-506ad6d9dd69,2021-12-15 20:48:57.914727+00,2021-12-15 20:48:57.914727+00,provider_api,stocksnap,stocksnap,ZFJEKSUY76,https://stocksnap.io/photo/ZFJEKSUY76,https://cdn.stocksnap.io/img-thumbs/960w/ZFJEKSUY76.jpg,https://cdn.stocksnap.io/img-thumbs/280h/ZFJEKSUY76.jpg,3888,2592,472754,cc0,1.0,Foodie Girl,https://stocksnap.io/author/121423,Fresh Garlic Photo,"{""license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/"", ""downloads_raw"": ""5"", ""favorites_raw"": ""0"", ""page_views_raw"": ""160"", ""raw_license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/""}","[{""name"": ""fresh"", ""provider"": ""stocksnap""}, {""name"": ""garlic"", ""provider"": ""stocksnap""}, {""name"": ""ingredient"", ""provider"": ""stocksnap""}, {""name"": ""vegetable"", ""provider"": ""stocksnap""}, {""name"": ""raw"", ""provider"": ""stocksnap""}, {""name"": ""food"", ""provider"": ""stocksnap""}, {""name"": ""bulb"", ""provider"": ""stocksnap""}, {""name"": ""white"", ""provider"": ""stocksnap""}, {""name"": ""healthy"", ""provider"": ""stocksnap""}, {""name"": ""organic"", ""provider"": ""stocksnap""}, {""name"": ""plant"", ""provider"": ""stocksnap""}, {""name"": ""vegetarian"", ""provider"": ""stocksnap""}, {""name"": ""harvest"", ""provider"": ""stocksnap""}, {""name"": ""head"", ""provider"": ""stocksnap""}, {""name"": ""bunch"", ""provider"": ""stocksnap""}, {""name"": ""clove"", ""provider"": ""stocksnap""}, {""name"": ""green"", ""provider"": ""stocksnap""}, {""name"": ""nature"", ""provider"": ""stocksnap""}, {""name"": ""closeup"", ""provider"": ""stocksnap""}, {""name"": ""market"", ""provider"": ""stocksnap""}]",f,2021-12-15 20:48:57.914727+00,f,jpg,photograph,0.19767441860465113
 230c6951-7f83-4263-a0da-a151f9f67b24,2021-12-15 20:48:57.914727+00,2021-12-15 20:48:57.914727+00,provider_api,stocksnap,stocksnap,NLJESZLWIU,https://stocksnap.io/photo/NLJESZLWIU,https://cdn.stocksnap.io/img-thumbs/960w/NLJESZLWIU.jpg,https://cdn.stocksnap.io/img-thumbs/280h/NLJESZLWIU.jpg,4437,3328,950814,cc0,1.0,The Building Envelope,https://stocksnap.io/author/129953,Concrete Wall Photo,"{""license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/"", ""downloads_raw"": ""5"", ""favorites_raw"": ""0"", ""page_views_raw"": ""210"", ""raw_license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/""}","[{""name"": ""concrete"", ""provider"": ""stocksnap""}, {""name"": ""wall"", ""provider"": ""stocksnap""}, {""name"": ""building"", ""provider"": ""stocksnap""}, {""name"": ""architecture"", ""provider"": ""stocksnap""}, {""name"": ""abstract"", ""provider"": ""stocksnap""}, {""name"": ""background"", ""provider"": ""stocksnap""}, {""name"": ""city"", ""provider"": ""stocksnap""}, {""name"": ""simplicity"", ""provider"": ""stocksnap""}, {""name"": ""dirty"", ""provider"": ""stocksnap""}, {""name"": ""material"", ""provider"": ""stocksnap""}, {""name"": ""monochrome"", ""provider"": ""stocksnap""}, {""name"": ""pillars"", ""provider"": ""stocksnap""}, {""name"": ""pattern"", ""provider"": ""stocksnap""}, {""name"": ""texture"", ""provider"": ""stocksnap""}, {""name"": ""exterior"", ""provider"": ""stocksnap""}, {""name"": ""weathered"", ""provider"": ""stocksnap""}, {""name"": ""rough"", ""provider"": ""stocksnap""}, {""name"": ""surface"", ""provider"": ""stocksnap""}, {""name"": ""weathered"", ""provider"": ""stocksnap""}]",f,2021-12-15 20:48:57.914727+00,f,jpg,photograph,0.19767441860465113
+230c6951-7f83-4263-a0da-a151f9f67b21,2021-12-15 20:48:57.914727+00,2021-12-15 20:48:57.914727+00,provider_api,stocksnap,stocksnap,NLJESZLWIZ,https://stocksnap.io/photo/NLJESZLWIZ,https://images.nypl.org/index.php?id=489681&t=w&suffix=510d47dc-3a80-a3d9-e040-e00a18064a99.001,https://images.nypl.org/index.php?id=489681&t=w&suffix=510d47dc-3a80-a3d9-e040-e00a18064a99.001,4437,3328,950814,cc0,1.0,The Building Envelope,https://stocksnap.io/author/129953,Concrete Wall Photo,"{""license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/"", ""downloads_raw"": ""5"", ""favorites_raw"": ""0"", ""page_views_raw"": ""210"", ""raw_license_url"": ""https://creativecommons.org/publicdomain/zero/1.0/""}","[{""name"": ""concrete"", ""provider"": ""stocksnap""}, {""name"": ""wall"", ""provider"": ""stocksnap""}, {""name"": ""building"", ""provider"": ""stocksnap""}, {""name"": ""architecture"", ""provider"": ""stocksnap""}, {""name"": ""abstract"", ""provider"": ""stocksnap""}, {""name"": ""background"", ""provider"": ""stocksnap""}, {""name"": ""city"", ""provider"": ""stocksnap""}, {""name"": ""simplicity"", ""provider"": ""stocksnap""}, {""name"": ""dirty"", ""provider"": ""stocksnap""}, {""name"": ""material"", ""provider"": ""stocksnap""}, {""name"": ""monochrome"", ""provider"": ""stocksnap""}, {""name"": ""pillars"", ""provider"": ""stocksnap""}, {""name"": ""pattern"", ""provider"": ""stocksnap""}, {""name"": ""texture"", ""provider"": ""stocksnap""}, {""name"": ""exterior"", ""provider"": ""stocksnap""}, {""name"": ""weathered"", ""provider"": ""stocksnap""}, {""name"": ""rough"", ""provider"": ""stocksnap""}, {""name"": ""surface"", ""provider"": ""stocksnap""}, {""name"": ""weathered"", ""provider"": ""stocksnap""}]",f,2021-12-15 20:48:57.914727+00,f,jpg,photograph,0.19767441860465113

This is a quick and dirty way to get a URL with a query parameter into the sample data. Go directly to the image URL using the identifier: http://localhost:50280/v1/images/230c6951-7f83-4263-a0da-a151f9f67b21/

It should work fine. Now click the "thumbnail" link and see that the thumbnail is as expected (compressed, resized version of the upstream image).

Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow [best practices][best_practices].
[x] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.

[best_practices]: https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project#_commit_guidelines

Developer Certificate of Origin

``` Developer Certificate of Origin Version 1.1 Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 1 Letterman Drive Suite D4700 San Francisco, CA, 94129 Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ```

github-actions[bot] commented 1 year ago

API Developer Docs Preview: Ready

https://wordpress.github.io/openverse-api/_preview/1056

Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again.

You can check the GitHub pages deployment action list to see the current status of the deployments.

sarayourfriend commented 1 year ago

@obulat @dhruvkb Please review this PR when you have a chance. It will block further progress on the API ECS deployment work at a certain point.

AetherUnbound commented 1 year ago

@sarayourfriend I rebased this PR so it could include the fix from #1059 and build properly. It looks like the new .env file in the docker-compose is causing some issue in CI as well so I tried to add a fix for that, hope that's alright!

sarayourfriend commented 1 year ago

Thanks, Madison!

sarayourfriend commented 1 year ago

@obulat We only logged thumbnail requests as debug logs, as far as I could tell. I'm not entirely sure what the usefulness of them is unless we have a specific issue we're trying to diagnose. I merged the PR as is because it'd been open for quite some time, but we can iterate in a separate PR to add additional logging if desired.

WordPress / openverse-api