[Batch Server] Cron management

[mreyescdl]

This ticket will supersede the following ticket. This ticket will only address management of crons onto the now operational batch host.

• https://github.com/CDLUC3/mrt-doc/issues/687

Crons to include

• [x] Consistency reports. (move from https://github.com/CDLUC3/mrt-admin-lambda/tree/main/consistency-driver)
• [x] Billing cron
  • ${HOME}/bin/uc3-mysql.sh billing -- -e 'call update_billing_range();'
  • ${HOME}/bin/uc3-mysql.sh billing -- -e 'call update_object_size();'
• [x] Zookeeper daily reports

Zookeeper Queue cleanup

• [ ] Inventory and Access need a helper cron (Ingest cleans up its own queue)

Spinning off into another project: Nuxeo feeds - (selectively move from https://github.com/CDLUC3/mrt-dashboard/tree/main/lib)

• This could necessitate an instance type comparable to the UI boxes

[mreyescdl]

Puppet Controlled Crons

Host level

• uc3-mrtinv01x2-stg
  • /dpr2/bin/dbSyncCron.sh
• uc3-mrtui03x2-prd
  • Nuxeo prod - Needs UI env
• uc3-mrtreplic01x2-prd
  • Clean up temp files and dirs
• uc3-mrtreplic02x2-prd
  • Clean up temp files and dirs
• uc3-mrtui01x2-stg
  • Nuxeo stage - Needs UI env
• uc3-mrtinv01x2-prd
  • /dpr2/bin/dbtest/db-perf.pl
  • /dpr2/bin/merritt_billing/run_mrt_dbquery.bash
  • /dpr2/bin/dbSyncCron.sh
• uc3-ingest01x2-stg|uc3-ingest02x2-stg
  • Pause Ingest for EZID maint
  • Clean ZK INV queue (worker 1 only)
  • ZK Failure notifications (worker 1 only) [Still needed?? Admin report has this ability]
  • Update profiles
• uc3-ingest01x2-prd|uc3-ingest02x2-prd|uc3-ingest03x2-prd
  • Pause Ingest for EZID maint
  • Clean ZK INV queue (worker 1 only)
  • ZK Failure notifications (worker 1 only) [Still needed?? Admin report has this ability]
  • Update profiles
  • Clean EFS Disk (worker 1 only)

Sub-Service level

• uc3-mrt-store
  • cleanup_storage_containers
  • cleanup_storage_temp
  • cleanup_tomcat_logs
• uc3-mrt-access
  • cleanup_storage_containers
  • cleanup_storage_temp

Manual Crons

uc3-ldap-stg.cdlib.org (ELB)

• uc3-ldap01x2-stg
  • export_ldap
  • uc3ldap-rotate_and_compress_logs

uc3-ldap.cdlib.org (ELB)

• uc3-ldap01x2-prd
  • check_replication status
  • export_ldap
  • uc3ldap-rotate_and_compress_logs
• uc3-ldap02x2-prd
  • check_replication status
  • export_ldap
  • uc3ldap-rotate_and_compress_logs