SDKits / ExamineX

Issue tracker for ExamineX
https://examinex.online
5 stars 0 forks source link

Set filterable to 'false' on custom field definition of type 'Edm.String' #95

Closed chrden closed 5 months ago

chrden commented 8 months ago

We have a field on our index called 'searchableContent' which collates multiple Umbraco document properties together, which we then search against.

This is set on Startup with a FieldDefinition of AzureSearchFieldDefinitionTypes.FullText.

We also have documents external to the Umbraco database that we are pushing into the External index, however we are getting the following error when attempting to push data into the 'searchableContent` field.

Field 'searchableContent' contains a term that is too large to process. The max length for UTF-8 encoded terms is 32766 bytes. The most likely cause of this error is that filtering, sorting, and/or faceting are enabled on this field, which causes the entire field value to be indexed as a single term. Please avoid the use of these options for large fields.

We do not need 'filterable' to be set on this field as it will never need a $filter expression to be run against it for range queries, for example.

Is there a simple way of setting 'Filterable' to false on the index for a single field such as this through the ExamineX API? Or is the best way to handle this in the CreatingOrUpdatingIndex event?

Thanks

Shazwazza commented 8 months ago

Hi @chrden,

You can entirely control the field definitions via the CreatingOrUpdatingIndex event, see https://examinex.online/customization#events

That event gives you access to pretty much modify anything at a lower level directly on the azure search models.

That said, I think removing having filtering enabled for string based fields is probably best moving forward.

What version are you using again? 4 or 5?

chrden commented 8 months ago

@Shazwazza Using version 4 and version 5 on two different projects respectively but I discovered this error whilst working on the project that is using version 5.

Shazwazza commented 8 months ago

Ok thanks. For the time being, your best option will be to use the CreatingOrUpdatingIndex event. Alternatively, you could go so far as to create and register your own IAzureSearchFieldValueType for your field type.

I'll keep this open for now and might look to remove filtering for string type fields in the future. That would have to be a configurable option though since I don't want to break anyone's functionality that might be relying on filtering string based fields.

chrden commented 8 months ago

Thanks @Shazwazza, I went ahead and created a CreatingOrUpdatingIndex for now but will keep my eyes open for any updates around this 👀

Shazwazza commented 8 months ago

This change will be targeted to a v6 release which will target Umbraco 13 and have the ContentDeliveryApi available in ExamineX (a change was required in Umbraco 13 to be compatible).