Examine sometimes fails to register after restart

bjarnef commented 8 months ago

Which Umbraco version are you using? (Please write the exact version, example: 10.1.0)

12.3.7

Bug summary

Once in a while I have noticed Examine fails to register after a restart of application. For this specific project on Umbraco Cloud, but we have also seen it sometimes on Azure after swap on slots (with recommended configuration for Azure apps).

For this specific project on Umbraco Cloud and after this issue the other day, which was resolves delete the specific media. https://github.com/umbraco/Umbraco-CMS/issues/15815 However because the failed registration of Examine, it also mean to frontend search using the Examine searchers was broken.

I tried restarting the site from the portal and Kudo, but didn't seem to help.

I suspect the image with large dimensions in https://github.com/umbraco/Umbraco-CMS/issues/15815 made the media in External and Internal indexes corrupt and thus fails to initialize the indexers or searchers.

Specifics

No response

Steps to reproduce

May not be ease to reproduce, but sometimes happen after an application restart.

Expected result / actual result

No response

github-actions[bot] commented 8 months ago

Hi there @bjarnef!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
We'll replicate the issue to ensure that the problem is as described.
We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face:

bjarnef commented 8 months ago

I solved this by accessing Kudo and deleting each index folder under site/wwwroot/umbraco/Data/TEMP/ExamineIndexes/ and afterward restart site from the portal.

I don't see this would have affected UmbracoForms index though, but when the registration of indexes fails, it happens to all indexes. I think this should handle try handling any errors, so not all indexes fails, e.g. in this case use of Forms and Member indexes would could possible still work although the an media in this specific case made the External and Internal indexes corrupt.

bjarnef commented 8 months ago

Furthermore I am not sure why it need to register DeliveryApiContentIndex, when delivery API isn't enabled? :) @kjac @elit0451 can't the DeliveryApiContentIndex be registered only when the delivery API is enabled?

Regarding the corrupt index, it was more a side-effect or this index, broken External an/or Internal indexes. https://github.com/umbraco/Umbraco-CMS/issues/15815

CyberReiter commented 8 months ago

We are facing the same issue quite regularly in an Umbraco Cloud project running on 13.1.1

Migaroez commented 6 months ago

bjarnef commented 6 months ago

On got a similar issue on an Umbraco v8 project on Umbraco Cloud today, which we haven't worked on for a long time and a restart doesn't solve this.

bjarnef commented 6 months ago

Just noticed it again on an Umbraco Cloud project using Umbraco v12.3.9 If would be nice with some better error handling in case one index is corrupt and showed an error at the index. Perhaps it could be possible to rebuild it from dashboard then. Currently we have to go to Kudo it delete Examine indexes, restart environment and rebuild.

jrunestone commented 5 months ago

Just adding my 2c here but I've had issues with our indexes not being rebuilt automatically after a swap in Azure until our scheduled jobs kick in.

It was however related to our solution being load balanced and the indexes on the subscribers weren't rebuilt there.

I had to add a manual rebuild trigger at the UmbracoApplicationStartedNotification event.

umbraco / Umbraco-CMS