Ongoing devops [SOLR UPGRADE MOSTLY] June 6 - June 19

IQSS / dataverse.harvard.edu

Custom code for dataverse.harvard.edu and an issue tracker for the IQSS Dataverse team's operational work, for better tracking on https://github.com/orgs/IQSS/projects/34

5 stars 1 forks source link

Ongoing devops [SOLR UPGRADE MOSTLY] June 6 - June 19 #278

Closed landreev closed 4 months ago

landreev commented 5 months ago

The single most important devops issue for this time period is the upcoming upgrade of the prod. Solr node, dvn-cloud-solr.lib.harvard.edu, scheduled for Friday, June 14. This effort will be traced here. Need to post a downtime announcement today, as well as make the final decision as to whether to perform a full reindex in the process.

landreev commented 5 months ago

Posted the announcement banner in prod., leaving room for further clarifying the scope of the effort and other details that are still tbd:

Harvard Library Technology Services (LTS) will be performing a scheduled upgrade of the server running our Solr (Search Engine) instance on Friday, June 14 (the exact time TBD). There will be a downtime window during which search functionality is going to be unavailable. We are also planning to use this downtime to apply some patches our development team has been working on in order to address the recent performance issues with searches and collection pages. Please stay tuned for more information.

landreev commented 5 months ago

I don't want to jinx it, but the solr server upgrade is looking like it has been a success, so far, with all 3 major goals achieved:

The server has been upgraded to RH8 by LTS;
Solr instance upgraded to 8.4.1 (mostly to address the security issue in 9.3.0);
The first optimization patch deployed in prod.

The holdings were fully reindexed in process, with the index largely pre-generated off-site thus minimizing exposing the users to empty search results.

landreev commented 4 months ago

Going to close this one. Will open a new ongoing issue.