search

isomerpages / isomercms-backend

A static website builder and host for the Singapore Government
5 stars 1 forks source link

Feat/modify private image retrieval #740

Closed alexanderleegs closed 1 year ago

alexanderleegs commented 1 year ago

This PR modifies our image retrieval method for private repos.

Context

The image retrieval operation on private repos is very expensive currently, as we have to retrieve each image via a github call separately - this uses up a lot of our token capacity and is a major blocker for repo privatisation. This is because the image view in the main album overview needs to show a preview of all our images - on public repos, we can retrieve the image to be used as preview by querying raw.githubusercontent.com/, but this same link doesn't work for private repos (it returns a 404 instead as read access to the repo is restricted). Previously, we were making a query via the github API to retrieve the image preview instead, which is very expensive, especially for sites which have a large number of images.

Solution

The solution to this requires us to retrieve image information with relatively low token cost as far as possible. It turns out that images on private repos can still be retrieved from <token>@raw.githubusercontent.com/ - this has been verified to not use up our token capacity as well. However, this url cannot be linked directly on our frontend, as this would expose our access tokens - instead, we pass a dataUrl to obfuscate the image origin source.

Tests are updated in #741

kishore03109 commented 1 year ago

Hey I am confused about some the aspects of this pr. Correct me if I am wrong, but does this PR need to have a corresponding FE PR to display the data URI if exists, else display the URL? I understand that there is a corresponding PR for updating our unit tests, but are we confident that this wouldnt break pre-existing private repos?