NASA-IMPACT / csdap-cumulus

SmallSat Cumulus Deployment
Other
1 stars 0 forks source link

Make copies of PSScene3Band thumbnails #305

Open chuckwondo opened 7 months ago

chuckwondo commented 7 months ago

The granules in the PSScene3Band collection contain thumbnail files with a _thumb suffix, but Aaron wants us to duplicate the thumbnail files as "browse" files, like we have with our Maxar granules, where the thumbnail file goes to the public bucket and the "browse" file goes to the protected bucket (so the "browse" file is in the list of downloadable files for a granule).

One possible option for meeting the following acceptance criteria is to use AWS Batch with a Lambda function. This would also require generating an S3 inventory for the objects in s3://csdap-cumulus-prod-protected/planet/PSScene3Band/, and filtering for *_thumb files (filtering might be possible as part of the batch job config, but if not, the logic can be added to the Lambda function).

UPDATE (Sprint 4, March 2024): Source Bucket is the 5982 protected bucket

Note: This is all prep for being able to do the migration ingest which will use cumulus to move files from 5982 to 5047

SubTasks

Acceptance criteria

chuckwondo commented 4 months ago

Here's an example Lambda function that works with AWS Batch: https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-ops-invoke-lambda.html