The granules in the PSScene3Band collection contain thumbnail files with a _thumb suffix, but Aaron wants us to duplicate the thumbnail files as "browse" files, like we have with our Maxar granules, where the thumbnail file goes to the public bucket and the "browse" file goes to the protected bucket (so the "browse" file is in the list of downloadable files for a granule).
One possible option for meeting the following acceptance criteria is to use AWS Batch with a Lambda function. This would also require generating an S3 inventory for the objects in s3://csdap-cumulus-prod-protected/planet/PSScene3Band/, and filtering for *_thumb files (filtering might be possible as part of the batch job config, but if not, the logic can be added to the Lambda function).
UPDATE (Sprint 4, March 2024):
Source Bucket is the 5982 protected bucket
The Source files for the thumbnails look kind of like this 20150601_121238_090c_thumb Note that there is no extension on this file name however, these files are png type. So when making the inventory, we need to select it by *_thumb*
Destination 1 is public bucket of 5982 - with filename ending like this: *-thumb.png
Destination 2 is protected bucket of 5982 - with filename ending like this: *-thumb.png
Destination 3 is protected bucket of 5982 - with filename ending like this: *-BROWSE.png
Note: This is all prep for being able to do the migration ingest which will use cumulus to move files from 5982 to 5047
SubTasks
[ ] Make an inventory of just the thumbnail files located in protected bucket from OLD PROD (AWS -5982).
[ ] Create test environment for copying s3 objects from one bucket to two destinations with different names.
[ ] Creating a Lambda to do the copying from an inventory file's single row of data over to the 3 destinations, including any renaming that is described above
[ ] Creating a batch job to process the entire file (each row in the inventory file calls the lambda)
[ ] Test the two above items with a few objects to ensure it will work within the UAH cloud
[ ] Take the above test environment and execute it against the operational environment
Acceptance criteria
[ ] Copies of the *_thumb files in the csdap-cumulus-prod-protected bucket (account csdap-cumulus-prod-5982) with key prefix planet/PSScene3Band/ exist next to the *_thumb files and named *-BROWSE.png
The granules in the PSScene3Band collection contain thumbnail files with a
_thumb
suffix, but Aaron wants us to duplicate the thumbnail files as "browse" files, like we have with our Maxar granules, where the thumbnail file goes to the public bucket and the "browse" file goes to the protected bucket (so the "browse" file is in the list of downloadable files for a granule).One possible option for meeting the following acceptance criteria is to use AWS Batch with a Lambda function. This would also require generating an S3 inventory for the objects in
s3://csdap-cumulus-prod-protected/planet/PSScene3Band/
, and filtering for*_thumb
files (filtering might be possible as part of the batch job config, but if not, the logic can be added to the Lambda function).UPDATE (Sprint 4, March 2024): Source Bucket is the 5982 protected bucket
20150601_121238_090c_thumb
Note that there is no extension on this file name however, these files arepng
type. So when making the inventory, we need to select it by*_thumb*
*-thumb.png
*-thumb.png
*-BROWSE.png
Note: This is all prep for being able to do the migration ingest which will use cumulus to move files from 5982 to 5047
SubTasks
protected
bucket from OLD PROD (AWS -5982).Acceptance criteria
*_thumb
files in thecsdap-cumulus-prod-protected
bucket (accountcsdap-cumulus-prod-5982
) with key prefixplanet/PSScene3Band/
exist next to the*_thumb
files and named*-BROWSE.png
*_thumb
files are renamed to*-thumb.png