Audit – Production deploy optimize - Githubissues

CDLUC3 / mrt-doc

Documentation and Information regarding the Merritt repository

8 stars 4 forks source link

Audit – Production deploy optimize #647

Closed elopatin-uc3 closed 3 years ago

elopatin-uc3 commented 3 years ago

Rollout of Audit optimization changes to prod. First deploy to Audit 04 at 9am 4/20.

Optimize

This code deploy to stage audit servers to resolve two issues:

expand the number of inv_audit entries that can be processed in a given amount of time without reducing IO burst credits
remove entire file downloads for generating fixity digest

Changes

Issues

Audit - optimize using segment io digests for fixity #642 Audit - the oldest unprocessed item should be between 60 - 90 days old #591

Jenkins

http://builds.cdlib.org/job/mrt-s3-pub/113/console http://builds.cdlib.org/job/mrt-build-audit/67/console

GitHub

https://github.com/CDLUC3/mrt-audit/commit/14725df6ac8925e2b949994fb8ab9d54d9ee2f71' https://github.com/CDLUC3/mrt-cloud/commit/master https://github.com/CDLUC3/mrt-cloud/commit/d2010fb8f4c0849b02886e1bd4fe98d4a8b18677 https://github.com/CDLUC3/mrt-audit/commit/f17be0c73e917a78b5a79ff37cfed51cf1dbbfa5

SSM

Production values needed:
aws ssm put-parameter

Deploy stage audit servers

Deploy to a single server 4/20 9A.M.

[x] uc3-mrtaudit04-prd.cdlib.org

Deploy remaining servers: TBD

dloy commented 3 years ago

SSM:

aws ssm put-parameter  --type String --overwrite --region us-west-2 --name /uc3/mrt/prd/audit/config/threadPool  --value "6"
aws ssm put-parameter  --type String --overwrite --region us-west-2 --name /uc3/mrt/prd/audit/config/queueSleepMs  --value "0"
aws ssm put-parameter  --type String --overwrite --region us-west-2 --name /uc3/mrt/prd/audit/config/queueCapacity  --value "248"
aws ssm put-parameter  --type String --overwrite --region us-west-2 --name /uc3/mrt/prd/audit/config/log/messageMaximumLevel  --value "1"
aws ssm put-parameter  --type String --overwrite --region us-west-2 --name /uc3/mrt/prd/audit/config/sizeChecksumBuffer  --value "16000000"