ExamineX in combination with custom index

[sandervandepas-iodigital]

Hi @Shazwazza

We are expirimenting with the free editions of ExamineX and Azure Cognitive Search for our Umbraco 10.6.1 project.

Steps we've taken so far:

1. Add Azure Cognitive Search (Free tier) to our TEST resource group in Azure Portal
2. Install ExamineX version 4.1.7 whithout a license
3. Added configuration settings (AzureSearchServiceName, AzureSearchServiceKey, DisabledIndexes, SiteFriendlyName) to appsettings.json
4. Added CustomIndex following exactly this documentation: https://docs.umbraco.com/umbraco-cms/v/10.latest-lts/reference/searching/examine/indexing

Issue is the following: We've added the MembersIndex and the UmbracoFormsRecordsIndex to the DisabledIndexes because we won't use it in Azure Search. So only the Internal and External indexes are in Azure Search right now. Because of the limitations of the free versions (200 items and max 3 indexes) we don't see the items of a specific documenttype in these index, that's the reason we want to add a custom index for these documenttype as 3rd and latest index for testing perposes before we buy a license and upgrade the tier.

After adding the CustomIndex to the DisabledIndexes as well, we see in Umbraco below the Examine Management section the CustomIndex and that is filled with items, so far so good.

If we remove the custom index from the DisabledIndexes list and want to add it to Azure Search it doesn't appear in the list in Umbraco anymore and also not in Azure Search as index.

It seems the PopulateIndexes method of our CustomIndex has no indexes at that moment.

After that we took a look into the source code of ExamineX and added an extra row to the builder Services like below: builder.Services.AddExamineXAzureSearchIndexFromUmbraco<UmbracoAzureSearchContentIndex>("CustomIndex");

At this point we see the CustomIndex again in Umbraco (and in Azure Search) and it refers to Azure Search. But after hitting the Rebuild Index button it keeps spinning and nothing happend.

Are we missing some extra settings to configure our custom index right?

Here is the link to our code: https://gist.github.com/sandervandepas-iodigital/cd26a6afdae2a33ab38aa42fe3c3b87a

Hopefully you can help us out with this problem.

Thanks in advance!

Grtz Sander

[AndrewBarta-SC]

Hey @sandervandepas-iodigital thanks for reaching out!

It sounds like the effort is to work around some of the limitations of the free version. Getting you a full trial license might be the route to go here. That could solve your testing purposes without needing to go down the rabbit hole of custom indexing for the time being.

Let me know if you'd like to go that route and we can get a trial license setup for you and the team!

Cheers, Andrew

[sandervandepas-iodigital]

Hi @AndrewBarta-SC,

That sounds like a plan!

Please provide me the full trial license so I can go on testing for now.

Thanks in advance!

Grtz Sander

[sandervandepas-iodigital]

Hi @AndrewBarta-SC or @Shazwazza

Could one of you guys give me a reaction on my latest post?

Thanks!

Grtz Sander

[AndrewBarta-SC]

Hey @sandervandepas-iodigital thanks for the multiple touchpoints! I followed up via email. Will get that trial spun up once we have the info needed.

Thanks! Andrew

[sandervandepas-iodigital]

Hi @AndrewBarta-SC @Shazwazza

Since we're using the paid license we still can't use custom indexes in Azure Cognitive search.

We created our own custom index, but after excluding all default Umbraco indexes (Internal, External, MembersIndex and UmbracoFormsRecordsIndex) and only our custom index will be used in Azure Search, we don't see that custom index in Azure Search.

After adding the line builder.Services.AddExamineXAzureSearchIndexFromUmbraco<UmbracoAzureSearchContentIndex>("CustomIndex"); we'll see the index in Azure Search but after a re-index nothing happens and we got the following errors:

System.NullReferenceException: Object reference not set to an instance of an object. at ExamineX.AzureSearch.Umbraco.UmbracoAzureSearchContentIndex.A(ValueSet ) at System.Linq.Lookup2.Create(IEnumerable1 source, Func2 keySelector, IEqualityComparer1 comparer) at System.Linq.GroupedEnumerable2.ToList() at ExamineX.AzureSearch.Umbraco.UmbracoAzureSearchContentIndex.PerformIndexItems(IEnumerable1 values, Action1 onComplete) at Examine.BaseIndexProvider.IndexItems(IEnumerable1 values) at ExamineX.AzureSearch.Umbraco.UmbracoContentAzureSearchIndex.Examine.IIndex.IndexItems(IEnumerable1 ) at Umbraco.Cms.Infrastructure.Examine.ContentIndexPopulator.IndexAllContent(Int32 contentParentId, Int32 pageIndex, Int32 pageSize, IReadOnlyList1 indexes) at Umbraco.Cms.Infrastructure.Examine.ContentIndexPopulator.PopulateIndexes(IReadOnlyList1 indexes) at Umbraco.Cms.Infrastructure.Examine.IndexPopulator.Populate(IIndex[] indexes) at Umbraco.Cms.Infrastructure.Examine.ExamineIndexRebuilder.RebuildIndexes(Boolean onlyEmptyIndexes, TimeSpan delay, CancellationToken cancellationToken)`

System.NullReferenceException: Object reference not set to an instance of an object. at ExamineX.AzureSearch.Umbraco.UmbracoAzureSearchContentIndex.A(ValueSet ) at System.Linq.Lookup2.Create(IEnumerable1 source, Func2 keySelector, IEqualityComparer1 comparer) at System.Linq.GroupedEnumerable2.ToList() at ExamineX.AzureSearch.Umbraco.UmbracoAzureSearchContentIndex.PerformIndexItems(IEnumerable1 values, Action1 onComplete) at Examine.BaseIndexProvider.IndexItems(IEnumerable1 values) at ExamineX.AzureSearch.Umbraco.UmbracoContentAzureSearchIndex.Examine.IIndex.IndexItems(IEnumerable1 ) at Umbraco.Cms.Infrastructure.Examine.ContentIndexPopulator.IndexAllContent(Int32 contentParentId, Int32 pageIndex, Int32 pageSize, IReadOnlyList1 indexes) at Umbraco.Cms.Infrastructure.Examine.ContentIndexPopulator.PopulateIndexes(IReadOnlyList1 indexes) at Umbraco.Cms.Infrastructure.Examine.IndexPopulator.Populate(IIndex[] indexes) at Umbraco.Cms.Infrastructure.Examine.ExamineIndexRebuilder.RebuildIndex(String indexName, TimeSpan delay, CancellationToken cancellationToken) at Umbraco.Cms.Infrastructure.Examine.ExamineIndexRebuilder.<>c__DisplayClass9_1.b1() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task.<>c.<.cctor>b272_0(Object obj) at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location --- at Umbraco.Cms.Infrastructure.HostedServices.QueuedHostedService.BackgroundProcessing(CancellationToken stoppingToken)`

Could you help us out how to fix this issue?

Do we miss some extra code changes to get a custom index working on Azure Search?

Hope to hear from you as quick as possible.

Thanks in advance.

Grtz Sander

[AndrewBarta-SC]

Hey @sandervandepas-iodigital

Shannon and I will need to gather some more info to investigate further. Could you provide all the code you've written for the custom index? You can share that over email if you prefer. We would need to see more than the line you provided in this post to better understand what could be causing the issue.

Could you also let us know the purpose of the custom index? Is it for Umbraco data or custom data?

Thanks!