search

usap-dc-dev / usap-dc-website

repository for usap-dc website. Includes javascript client side app and python/flask server side.
2 stars 0 forks source link

aws archiving from prod not working #57

Open fnitsche2001 opened 6 months ago

fnitsche2001 commented 6 months ago

the script "archiveUSAPDC_batch.py" runs using a cron job once a day and will check which datasets have been set to "ready to be archived" and then will run the archiceUSAPDC.py script to archive each dataset. The command is from /web/ht/usr/bin/python3 archiveUSAPDC_readyDatasets.py The archiving consists of building bag file and package it into a tar.gz file and then copy it over to AWS and also to seafloor-ph. The bag.tar.gz files get created, but it seems both connections are not working after the we had the server failure and Eric rebuild prod early February.

The error is ERROR: unable to upload file /web/usap-dc/htdocs/archive/601771_bag.tar.gz to AWS S3 Unable to locate credentials

The credentials are stored in the /aws directory in my home directory ,but I am not sure if they are the correct ones. I copied them over from Seafloor-ph where they seem to work.

astrong-ldeo commented 5 months ago

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html

The credentials file is located at ~/.aws/credentials on Linux or macOS

Note the dot at the beginning of the directory name. That could be the issue.

astrong-ldeo commented 1 month ago

Is there any use of sudo in the command?