Indexing fails for date conversion

[SATC-Ben]

Which Umbraco version are you using? (Please write the exact version, example: 10.1.0)

11.4.0

Specifics

Hi, We updated from 11.1.0 to 11.3.1 and started getting this issue when the index builds. I then saw #14096 and updated it to 11.4.0 but got the same results. For context, we use the default language (en-US) but are situated in Australia with dd/mm/yyyy formats.

[13:43:15 ERR] Error occurred executing workItem. System.InvalidOperationException: Cannot assign value "8/31/2022 12:00:00 AM" of type "System.String" to property "toDate" expecting type "System.DateTime". at Umbraco.Cms.Core.Models.Property.ThrowTypeException(Object value, Type expected, String alias) at Umbraco.Cms.Core.Models.Property.TryConvertAssignedValue(Object value, Boolean throwOnError, Object& converted) at Umbraco.Cms.Core.Models.Property.SetValue(Object value, String culture, String segment) at Umbraco.Cms.Core.PropertyEditors.NestedPropertyIndexValueFactoryBase2.GetNestedResults(String keyPrefix, String culture, String segment, Boolean published, IDictionary2 propertyTypeDictionary, TItem nestedContentRowValue, IEnumerable1 availableCultures)+MoveNext() at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at Umbraco.Cms.Core.PropertyEditors.NestedPropertyIndexValueFactoryBase2.Handle(TSerialized deserializedPropertyValue, IProperty property, String culture, String segment, Boolean published, IEnumerable1 availableCultures) at Umbraco.Cms.Core.PropertyEditors.JsonPropertyIndexValueFactoryBase1.GetIndexValues(IProperty property, String culture, String segment, Boolean published, IEnumerable1 availableCultures) at Umbraco.Cms.Core.PropertyEditors.NestedPropertyIndexValueFactoryBase2.GetNestedResults(String keyPrefix, String culture, String segment, Boolean published, IDictionary2 propertyTypeDictionary, TItem nestedContentRowValue, IEnumerable1 availableCultures)+MoveNext() at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection) at Umbraco.Cms.Core.PropertyEditors.NestedPropertyIndexValueFactoryBase2.Handle(TSerialized deserializedPropertyValue, IProperty property, String culture, String segment, Boolean published, IEnumerable1 availableCultures) at Umbraco.Cms.Core.PropertyEditors.JsonPropertyIndexValueFactoryBase1.GetIndexValues(IProperty property, String culture, String segment, Boolean published, IEnumerable1 availableCultures) at Umbraco.Cms.Infrastructure.Examine.BaseValueSetBuilder1.AddPropertyValue(IProperty property, String culture, String segment, IDictionary2 values, IEnumerable1 availableCultures) at Umbraco.Cms.Infrastructure.Examine.ContentValueSetBuilder.GetValueSetsEnumerable(IContent[] content, Dictionary2 creatorIds, Dictionary2 writerIds)+MoveNext() at System.Collections.Generic.LargeArrayBuilder1.AddRange(IEnumerable1 items) at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable1 source) at Umbraco.Cms.Infrastructure.Examine.ContentIndexPopulator.IndexPublishedContent(Int32 contentParentId, Int32 pageIndex, Int32 pageSize, IReadOnlyList1 indexes) at Umbraco.Cms.Infrastructure.Examine.ContentIndexPopulator.PopulateIndexes(IReadOnlyList1 indexes) at Umbraco.Cms.Infrastructure.Examine.IndexPopulator.Populate(IIndex[] indexes) at Umbraco.Cms.Infrastructure.Examine.ExamineIndexRebuilder.RebuildIndex(String indexName, TimeSpan delay, CancellationToken cancellationToken) at Umbraco.Cms.Infrastructure.Examine.ExamineIndexRebuilder.<>c__DisplayClass9_1.b__1() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location --- at Umbraco.Cms.Infrastructure.HostedServices.QueuedHostedService.BackgroundProcessing(CancellationToken stoppingToken)

Steps to reproduce

Settings > Examine Management > ExternalIndex > Rebuild index

Expected result / actual result

Expected: Index runs without issue. Actual: After a while .. The process is taking longer than expected, check the Umbraco log to see if there have been any errors during this operation also the error in the log mentioned above.

[github-actions[bot]]

Hi there @SATC-Ben!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

• We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
• If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
• We'll replicate the issue to ensure that the problem is as described.
• We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face:

[elit0451]

Hi @SATC-Ben 👋

Thanks for reaching out! To understand correctly your setup to be able to reproduce the issue, you have a Date Picker property with the alias"toDate" and can you tell me the "Date format" of the used Date Picker data type?

For example, this is the default format:

[SATC-Ben]

Hi @elit0451 ,

Thanks for the response. The date picker for the field/s that are called toDate are set to YYYY-MM-DD yes. I'm really unsure of how it's changing it to the American format along the way.

We have also just recently upgraded from V8 to V11.

The issue occurs on both Internal and External Index rebuilds.

[elit0451]

After trying to hardcode the value to "8/31/2022 12:00:00 AM", it seems like the error you get is due to having the Date Picker value saved in the wrong format to the database. Can you try to save and publish your document type again and try to see if you get the same error when rebuilding the index. Hopefully saving and publishing will fix the Date Picker value format in the database.

Ahh and it is important to note that the wrong value is within a BlockList for example with a Date Picker property editor.

Let me know how it goes 🙂

[SATC-Ben]

Right, thanks for that. I've done some digging and found the following:

• The issue we are seeing appears to be in NestedContentItems that use DatePickers
• During our upgrade from 8 to 11 we haven't changed the format of the dates that are having the issue. I confirmed this from looking in the database and checking the usync files from pre/post-upgrade.
  • 
  • DB [umbracoDataType] same for both V8 & V11.4.0 (Img attached)
  • 
  • Usync comparison for V8 & V11.4.0 (Img attached)
• Attempting to publish again made no change, unfortunately. The following was taken from the database:
  • Pre-Publish: \"toDate\": \"8/31/2022 12:00:00 AM\",\r\n
  • Post-Publish: \"toDate\": \"8/31/2022 12:00:00 AM\",\r\n

So there are a few things that would be helpful to know with this:

• What sets the date format of what is stored in a NestedContentItem? It doesn't appear to have ever been the DatePicker format. -- Is it the global language that influences this? Ours is set to En-US
• What has changed between version V11.1.0 to V11.3.1 that has caused the indexing system to not account for the date format that's used in NestedContentItems?

[SATC-Ben]

@elit0451 Unfortunately we have the updated version with the issue on our live server which has high traffic. We also can't roll back as an error occurs due to DB structure changes made from 11.1.0 to 11.4.0 . Is someone free to respond to the above? Or can I help in any way?

[elit0451]

Hi again @SATC-Ben

We are sorry to hear that you have this issue on your live server. We still need to figure out what is causing this issue, but for now, you can try to work around it by implementing the old behaviour:

https://github.com/umbraco/Umbraco-CMS/issues/14096#issuecomment-1511545767

After you've implemented it, try to rebuild the index 🙂

And regarding, what has changed between version V11.1.0 to V11.3.1 that has caused the indexing system to not account for the date format that’s used in NestedContentItems? - The nested types are indexed correctly in 11.3+

[SATC-Ben]

Hi @elit0451, Thanks so much for the workaround and answer! Good luck with the search.

[andr317c]

Hey!

I just took a look at this issue, and what I can see is that you are using the default Date Picker Format which is YYYY-MM-DD, for the Australian time. But the date you are inserting into the Date Picker is in the format of MM-DD-YYYY HH:mm:ss A. The error you get is because the formats are not the same.

To fix this, I will recommend creating another DatePicker DataType which has the date format: MM-DD-YYYY HH:mm:ss A.

It's very likely that this problem was hiding there all along, however, up until 10.3 the contents of the block list and block grid were not indexed properly, so that would be why it worked before, since the data wasn't processed in the indexer.

Since there is no issue with the Date Picker, I will go ahead and close this issue. Please feel free to comment if this fix does not work! Then we can have another look 😄