ORACLE MIGRATION EPIC 2/13 - 5/5

[libdgg]

NOTE: see list of databases for both Oracle and MySQL

Oracle databases will stay on-premise. Ours our not part of the campus move of Oracle to AWS.

NOTE: Oracle migration is over spring break. The fedora application it impacts is the backend for approximately 3 of our legacy older applications and they are business critical. They are student facing.

NOTE: MySQL is after spring break (support will run out in August) - looking at May

JANUARY/FEBRUARY COMPLETE: Initiating: 2 weeks

• COMPLETE: Project one-pager
• COMPLETE: Socializing the project in LTDS
• COMPLETE (ONGOING): Socializing the effort to EUL stakeholders, ECDS, and OIT

FEB 13 - 24 IN PROGRESS: Planning: 2 Weeks

• COMPLETED: Technology Design for the Solution

• COMPLETED: TICKET 39: Staffing allocations

• COMPLETED: Input from subject experts in database migration efforts (OIT, LTDS, etc.)

• COMPLETED: Project Plan, Timeline and Scope

• COMPLETED: Confirm list of databases

• COMPLETED: Cost estimates (confirm no change for Oracle) - there will be a change - we are paying for 1 server and will continue - we now have a second server for test

• COMPLETED: Confirm ECDS has been informed and they have reported back approval on their end with no impact to them. DG emailed back to let them know that their lack of response indicated that ECDS does not recognize, use or depend on the databases. NOTE: ECDS never followed up in Feb 2022 with my request for them to review their app support transition documentation either.

FEB 27 - APR 21 IN PROGRESS: Executing and Monitoring: 8 weeks

• COMPLETED: TICKET 39: Provision of Staff Resources

• COMPLETED: Implementation of the technology solution

• COMPLETED: Project status and risk management

• FEB 27 - MAR 3: Migration Prep and Testing - DBs set to Read-Only -- COMPLETED: 2/21 Collin emailed stakeholders on status and read-only mode coming up -- COMPLETED: 3/2 Beth emailed stakeholders that read-only mode was underway -- COMPLETED: 3/2 Beth emailed stakeholders -- The database copy went well and we have switched our production fedora environment to using the new DB. So far all seems to be working without requiring a reindex. Please test out your usual workflows and let us know if you encounter any issues,

• MAR 6 -10 : Migration (for Oracle during spring break)

• COMPLETED PLAN/COMMUNICATE : QA and Acceptance - ETA March 6 - 10

• TBD: Determine planning/roadmapping etc. to prepare for this in another 3 years (2026)

APR 24 - MAY 5 Closing: 2 Weeks

• TBD: Project closeout
• TBD: Lessons learned
• Project Closeout: 1 week

[libdgg]

CB emailed stakeholders (and the OIT team) on 2023-02-21 with status update (parsed out here for timeline steps):

2/21: Zo and Beth have successfully reindexed the Fedora QA database after adding more data from production.

2/21: Email stakeholders to please check the testing version of your applications to ensure they are working properly and let us know when you have done so.

2/24: Date for any feedback from stakeholders. We would like to hear from you before Friday.

2/27 - 3/3: Putting the production Fedora into read-only mode next week to let OIT copy its data to a new, upgraded database. Read-only mode is necessary to prevent changes entering the database after the copy starts, so you don’t lose work in the migration.

3/6 - 3/10:

[libdgg]

2023-02-10: UPDATE

Now, In QA We attempted to reindex the Fedora QA database this week, but the process failed. The issue appears similar to one previously encountered in 2015. Zo is reaching out to someone in OIT who was involved back then for help. Once the reindexing process is working, we will rerun it with more data copied from Production to ensure all data types are covered. Despite the documentation, we found the reindexing process requires Fedora to be shut down. So, the affected applications will be missing critical functionality throughout the reindexing process. The applications themselves do not need to be shut down, but interactions with Fedora (uploads, downloads, etc.) will not work. Search and metadata retrieval should function normally. We still need to confirm that the old database works after the reindexing process completes. We plan to confirm this before we meet with the OIT database team next Friday, February 10th. As we complete more of this work, I will reach out to you to test the QA applications against the upgraded database.

Plan for production We will put Fedora into read-only mode for the week before Spring Break. This will allow the database team to copy the production database to the new Oracle version. If this copy goes smoothly, then no reindexing may be necessary. If the copy does not produce useable results, we will need to reindex the Fedora data into a new, empty database. During Spring Break, we will shut Fedora down and start this process. Once Fedora is running with the upgraded database, I will again reach out to complete testing for the production applications.

Backout plan Since all our plans involve moving data to a new database, our primary plan in case of issues is to revert to the old database. If this is not possible, we must request the latest backup of data available, and go through disaster recovery. As the current version of Oracle is reaching end-of-life, we will need to complete more testing and reattempt the upgrade at a later date.

[libdgg]

confirmed with Naiqi that the fedora qa servers are shut down. that closes the last ticket in this epic