Migrate dataset images from CNGB deployment to AWS deployment

rija commented 1 year ago

User story

As a website user I want to see the dataset images So that the dataset page looks more engaging

Acceptance criteria

Given a dataset has an image associated on the CNGB deployment
When I navigate to the corresponding dataset pages on the AWS deployment
Then I can see the dataset images

Additional Info

Loop through all datasets:
   If there is no custom image on the AWS deployment, try scrape the custom image thumbnail from `old.gigadb.org`
   if there is a custom image found from `old.gigadb.org`, save it in the cloud and use it in the AWS deployment
   if there no custom image found from `old.gigadb.org`, use generic image
end loop
report list of dataset that use generic images

Curators will manually update ~100 of the most viewed datasets as soon as possible after the new server is deployed live. Curators will manually add dataset thumbnail images to new datasets as they are generated.

Product Backlog Item Ready Checklist

[ ] Business value is clearly articulated
[ ] Item is understood enough by the IT team so it can make an informed decision as to whether it can complete this item
[ ] Dependencies are identified and no external dependencies would block this item from being completed
[ ] At the time of the scheduled sprint, the IT team has the appropriate composition to complete this item
[ ] This item is estimated and small enough to comfortably be completed in one sprint
[ ] Acceptance criteria are clear and testable
[ ] Performance criteria, if any, are defined and testable
[ ] The Scrum team understands how to demonstrate this item at the sprint review

Product Backlog Item Done Checklist

[ ] Item(s) in increment pass all Acceptance Criteria
[ ] Code is refactored to best practices and coding standards
[ ] Documentation is updated as needed
[ ] Data security has not been compromised (with particular reference to the personal information we hold in GigaDB)
[ ] No deviation from the team technology stack and software architecture has been introduced
[ ] The product is in a releasable state (i.e. the increment has not broken anything)

kencho51 commented 11 months ago

Test image flow for public dataset in the following cases:

[x] delete any existing image, will show generic image on public dataset page
[x] upload new image, save, image url path becomes `https://assets.gigadb-cdn.net/$env/images/datasets/3$datasetUuid/new.jpg
[x] upload new image, save, upload test image, save, image url path becomes `https://assets.gigadb-cdn.net/$env/images/datasets/3$datasetUuid/test.jpg
[x] delete new image, will show generic image on public dataset page
[x] upload new image, then remove image record, show generic image

Test image flow for private dataset, given PR #1578 has been merged

[x] delete any existing image, save, will show generic image
[x] upload new image, save, image url path becomes `https://assets.gigadb-cdn.net/$env/images/datasets/3$datasetUuid/new.jpg
[x] upload new image, save, upload test image, save, image url path becomes `https://assets.gigadb-cdn.net/$env/images/datasets/3$datasetUuid/test.jpg
[x] delete new image, will show generic image
[x] upload new image, then remove image record, show generic image

The above cases have been tested on my dev and staging gigadb website, the case are all passing.

kencho51 commented 10 months ago

Test image flow for public dataset On upstream staging, eg: 100295

[x] delete any existing image, will show generic image on public dataset page
[x] upload test image, save, will redirect to public dataset page with test image
[x] then upload dataset image to replace test image, save, will redirect to public dataset page with dataset image
[x] remove image record and wait for a few seconds, will show generic image on the admin page, and public dataset page with generic image
[x] upload dataset image, save, will redirect to public dataset page with dataset image

pli888 commented 10 months ago

I tested dataset image upload and deletion functionality on https://beta.gigadb.org/dataset/100295. This dataset initially had the generic image on beta site.

[x] After uploading test image on dataset 100295 admin page, and clicking save button, the browser was redirected to public dataset page with test image. N.B. this required ~10 second wait for public dataset page to be displayed.
[x] Deleted test image on dataset 100295 admin page led to generic image being displayed on dataset admin page. N.B. need to wait ~5 seconds for generic image to show on dataset admin page. This triggered a Tideways email alert:

[x] Uploaded actual dataset image to replace test image on dataset 100295 admin page and clicked saved button. Was redirected to public dataset page with uploaded actual dataset image. N.B. need to wait ~15 seconds before dataset page is displayed.

Notes:

Curators should ensure image files are small - less than 100 KB.
Curators should standardise image file names.

rija commented 10 months ago

@pli888 has confirmed it works on beta, so closing this ticket

gigascience / gigadb-website