Platform sites indexing issue

[StudioZut]

We're still having issue with site content (on flagship and other platform sites) not always appearing in site search or sometimes in resource lists.

Example (Brasil): This profile was created, but not appearing in site search or in the staff list: https://www.wribrasil.org.br/equipe/laura-rossler -- I re-index manually 10-16, which fixed it.

Example (Brasil): "we noticed that our events page is also not updating correctly. We posted this (https://www.wribrasil.org.br/eventos/2024/wri-brasil-na-cop-16-biodiversidade) and its not showing on the events page (https://www.wribrasil.org.br/eventos)." -- this also was fixed after re-indexing manually.

Example (Brasil): 2 posts weren't showing up in their library at https://www.wribrasil.org.br/biblioteca : https://www.wribrasil.org.br/noticias/creditos-de-biodiversidade-podem-impulsionar-conservacao-da-natureza and https://www.wribrasil.org.br/noticias/o-que-e-uma-estrategia-climatica-de-longo-prazo -- Maria fixed 10-15 "I ran the queries to disconnect and reconnect to solr and that seems to have done it"

Example (Brasil): they found 2 posts that weren't appearing in past events (also fixed with re-indexing)

Dev tasks

• [x] Install https://www.drupal.org/project/dblog_persistent and enable to permanently log any messages with the text "DOCS_AND_FREQS_AND_POSITIONS"
• [x] Add enabling this module and its settings as an install hook in wri_sites
• [ ] Upgrade Search Api Pantheon to the latest release 8.2.1
• [ ] Translate the steps in here:

  terminus drush your-site.your-env -- sapi-sc pantheon
  terminus drush your-site.your-env -- sapps pantheon /code/web/modules/contrib/search_api_solr/jump-start/solr8/config-set/
  terminus drush your-site.your-env -- en search_api_solr_admin
  terminus drush your-site.your-env -- solr-reload pantheon
  terminus drush your-site.your-env -- sapi-rt
  terminus drush your-site.your-env -- sapfc pantheon
  terminus drush your-site.your-env -- sapi-sc pantheon
  terminus drush your-site.your-env -- sapi-sd pantheon
  terminus drush your-site.your-env -- sapi-se pantheon
  terminus drush your-site.your-env -- sapi-ena
  terminus drush your-site.your-env -- sapi-i

  Into things that can be done in the admin area and document that somewhere.

WIP admin version of above:

1. On /admin/config/search/search-api/server/pantheon push the "Delete all indexed data on this server" button.
2. Enable the "Search API Pantheon Admin" module and the "search_api_solr_admin" module, if it isn't already, on /admin/modules
3. ON /admin/config/search/search-api/server/pantheon/pantheon-admin push the schema -- (this is broken, Open issue: https://www.drupal.org/project/search_api_pantheon/issues/3467907)
4. From /admin/config/search/search-api/server/pantheon click "Reload Core"
5. sapi-rt -- Run cron from /admin/config/system/cron
6. sapifc -- unsure how to do on the backend. The command documentation is "Force Solr server cleanup if hash has changed."
7. On /admin/config/search/search-api/server/pantheon push the "Delete all indexed data on this server" button. again
8. sapi-sd -- on /admin/config/search/search-api click the dropdown beside "Edit" next to the Pantheon server and click "Disable"
9. On /admin/config/search/search-api click the dropdown beside "Edit" next to the Pantheon server and click "Enable"
10. On /admin/config/search/search-api click the dropdown beside "Edit" next to the Sitewide index and click "Enable"
11. On /admin/config/search/search-api/index/sitewide click " Index now

[mariacha]

Pantheon caught some logs:

Uncaught PHP Exception Error: "Call to a member function getSearchId() on null" at /code/web/modules/contrib/search_api/search_api.module line 602[17-Oct-2024 04:28:23 UTC]

The search_api module doesn't have any issues open for this, and doesn't have a new release, so there must be something handing the search_api.module a call to getSearchId on a null item in an unexpected way.

Looking at the code, the error would only be thrown in the unlikely circumstance that the $view->getQuery() function didn't return anything.

So I don't think we'll get far down that path.

We had noticed the same issue a few days before, where the logs were returning the error we were seeing when the Solr database was on version 4.2, but it was on version 4.3.5 when I checked. Running through the commands to clear out and re-index eventually started working, but I wasn't convinced that these commands had fixed anything. It felt a lot more like the server was temporarily pointing to an old, bad schema version, and it just fixed itself with some cron run or something.

Perhaps at this point it's worth writing some kind of script to notify us if an error about "DOCS_AND_FREQS_AND_POSITIONS" happens on one of the sites, so we can get a sense of how often this comes up.

[mariacha]

Perhaps this module would help for sending out the notifications: https://www.drupal.org/project/logging_alerts

And/or this one for permanently logging any search notifications: https://www.drupal.org/project/dblog_persistent

[mariacha]

The action item out of here was to install a couple new modules, and that's in the description.

We'll be able to review any Search API errors at https://develop-wriflagship.pantheonsite.io/admin/reports/persistent-log

and if we want to set up error notifications, we can do that on the live site.

[mariacha]

Quick note about this one, Pantheon is recognizing their issues with Solr reverting in this incident: https://status.pantheon.io/incidents/rv8bw0v6rbjy

I'm taking this ticket back to keep working on the admin instructions.