Standup Cumulus development deployment in CBA sandbox account for Kris

chuckwondo commented 1 year ago

Standup Cumulus 13.4.0 deployment to csda-app-sbx-7894 CBA AWS account for Kris to use for development, by following the instructions in this repo's README.md file.

Acceptance criteria:

[x] Successful Terraform deployment per the README.md instructions, with the exception of running a successful E2E test (i.e., all resources managed by Terraform are successfully created)
[x] Stretch: successfully run sample E2E discovery/ingestion test, per the last part of the instructions in the README.md file (if this cannot be completed in the same sprint as the item above, this issue should still be closed, and new issues for any problems blocking this criterion should be created and added to the subsequent sprint)

krisstanton commented 1 year ago

I was able to do a successful sandbox deploy, but not yet able to run the smoke test.

Making a note here about this deployment in case it helps anyone as a future reference.

It was a little bit of a bumpy road but here is what I experienced. Commands used 'make pre-deploy-setup' // 2 times 'make all-up-yes' // multiple times 'make up-cumulus' // To enter EarthData Credentials 'make up-cumulus-yes' // multiple times to solve timing based errors

Here is a kind of chronology of what the deployment.

(1) make pre-deploy-setup
    Result
        #181 terraspace init cumulus]: ERROR: Aws::Errors::MissingCredentialsError: unable to sign request without credentials set
        #181 terraspace init cumulus]: It doesnt look like AWS credentials and access has been setup.
        Error running: terraspace init cumulus. Fix the error above or check logs for the error.
    Resolution
        I double checked .env files and credentials - everything looked good so I ran it again.

(2) make pre-deploy-setup
    Result
        Successful but still had these 'errors' listed
        terraspace init rds-cluster:  ERROR: Aws::Errors::MissingCredentialsError: unable to sign request without credentials set
        terraspace init rds-cluster:  ERROR: Aws::Errors::MissingCredentialsError: unable to sign request without credentials set
        terraspace init rds-cluster:  Terraform has been successfully initialized!
    Resolution
        I went ahead to the next step of running 'make all-up-yes'

(3) make all-up-yes
    Result
        Error: error creating Lambda Function (1): InvalidParameterValueException: The provided execution role does not have permissions to call CreateNetworkInterface on EC2
    Resolution
        Ran 'make all-up-yes' again to solve the deployment racing conditions.

(4) make all-up-yes
    Result
        EarthData Login Error
        On this run, there was never an obvious pause or prompt to enter the credentials.
        I think the 'yes' part of the command causes that.
    Resolution
        Ran 'make up-cumulus' to

(5) make up-cumulus
    Result
        After the Earthdata login, the deploy timed out waiting for a 'yes' 
    Resolution
        Ran 'make up-cumulus-yes'  // Now that we are past the Earth Data login, this should work fine.

(6) make up-cumulus-yes
    Result
        This part of the deploy took a long time.
        For the most part, it ran smoothly and created mosst of the services.

        At one point I did have to manually remove a lock with these commands
            make bash
            terraspace force_unlock cumulus <LOCK_GUID>  // The deployment script gives you this command

        At another point I got a resource already exists error and had to remove it with this command.
            make terraform-doctor-cumulus

The final result of the deploy were these lines
    archive_api_redirect_uri = https://t9maohtgsi.execute-api.us-west-2.amazonaws.com:8000/dev/token
    archive_api_uri = https://t9maohtgsi.execute-api.us-west-2.amazonaws.com:8000/dev/
    cumulus_distribution_api_redirect_uri = login
    cumulus_distribution_api_uri = 
    report_executions_sns_topic_arn = arn:aws:sns:us-west-2:772632177894:cumulus-kris-sbx7894-report-executions-topic
    report_granules_sns_topic_arn = arn:aws:sns:us-west-2:772632177894:cumulus-kris-sbx7894-report-granules-topic
    report_pdrs_sns_topic_arn = arn:aws:sns:us-west-2:772632177894:cumulus-kris-sbx7894-report-pdrs-topic
    stepfunction_event_reporter_queue_url = https://sqs.us-west-2.amazonaws.com/772632177894/cumulus-kris-sbx7894-sfEventSqsToDbRecordsInputQueue

Note: The reason the line above cumulus_distribution_api_uri = was blank is because I did not comment out the line TF_VAR_cumulus_distribution_url= in the .env file for this fresh deployment. When setting up the .env file for a FIRST deploy to any given environment, the line TF_VAR_cumulus_distribution_url= Should be commented out like this #TF_VAR_cumulus_distribution_url=

krisstanton commented 1 year ago

Re: The Smoke Test - Stretch goal

Here is a link to the description of why the smoke test was not able to be completed. https://github.com/NASA-IMPACT/csdap-cumulus/pull/115#issuecomment-1430046463

krisstanton commented 1 year ago

Attempted to run the smoke test in a less automated way by running these two commands. make bash aws s3 cp --recursive "app/stacks/cumulus/resources/granules/PSScene3Band" "s3://csda-cumulus-kris-sbx7894-provider-7894/storage-ss-ingest-prod-ingesteddata-uswest2/planet/" >/dev/null The granules do make it up to the bucket that way, but when attempting to run the rules, another NoSuchBucket error pops up when trying to load the granules from the provider bucket.

krisstanton commented 1 year ago

Smoke Tests passed after incorporating the changes in this Pull Request: https://github.com/NASA-IMPACT/csdap-cumulus/pull/116

NASA-IMPACT / csdap-cumulus

Standup Cumulus development deployment in CBA sandbox account for Kris #92