search

IHTSDO / snowstorm

Scalable SNOMED CT Terminology Server using Elasticsearch
Other
208 stars 83 forks source link

Elasticsearch 8 upgrade guide is missing reindex of branch-review #630

Open jonmdtn opened 3 weeks ago

jonmdtn commented 3 weeks ago

After following the elasticsearch upgrade guide we were successfully able to load our data from Snowstorm 8.3.0 to Snowstorm 10.4.2. We discovered quite quickly however that branch merging was not possible and failed with the error:

{ "error" : "INTERNAL_SERVER_ERROR", "message" : "[es/index] failed: [document_parsing_exception] [1:151] failed to parse field [lastUpdated] of type [date] in document with id 'c9ec258f-f6fb-49ff-a4e2-bc76df6910db'. Preview of field's value: '1728998846139'" }

From this we were able to identify that lastUpdated only existed on the branch-review index and we followed the steps to change the datatype from date to long after verifying this is the correct datatype created by Snowstorm in version 10.4.2.

This works as expected, and we are now able to merge branches.

Is there any reason this was excluded other than not being known? Just curious if we are doing something that might cause unexpected behavior in the future.

I will attach a pull request updating the docs as well :)

jonmdtn commented 3 weeks ago

PR 631 created which we used to resolve the issue.

kaicode commented 3 weeks ago

Hi @CoderMChu did you come across this issue? There must be thousands of branch review documents on the managed service platform, it looks like they don't get deleted.

CoderMChu commented 2 weeks ago

@jonmdtn @kaicode We didn't pick up this issue during ES8 upgrade because we didn't have any unmarged branches before data migration. This issue will only occur when you apply branch merge using review created before ES8 data migration. Code system version data is the only issue we encountered during migration because this serves as part of referenced data.