IHTSDO / snowstorm

Scalable SNOMED CT Terminology Server using Elasticsearch
Other
204 stars 80 forks source link

Error when using international characters on ECL #28

Closed alopezo closed 5 years ago

alopezo commented 5 years ago

Using this content on POST /{branch}/concepts/search search returns an error:

{ "activeFilter": true, "conceptIds": [], "eclFilter": "<<19923001 |catéter (objeto físico)|", "limit": 2, "offset": 0, "statedEclFilter": "", "termFilter": "" }

Returns this result:

{ "error": "INTERNAL_SERVER_ERROR", "message": "Failed to parse ECL '<<19923001 |catéter (objeto físico)|'" }

Removing accented characters from the ECL resolves the issue:

<<19923001 |catéter (objeto físico)| -> <<19923001 |cateter (objeto fisico)|

The term comes from the Spanish Edition. Tested the ECL on the APG parser site and parses OK with accents.

Thanks

danka74 commented 5 years ago

This has been fixed by a fix in this project: https://github.com/IHTSDO/snomed-ecl-parser Please use the latest ecl-parser and you will be fine sending Arabic, Chinese, Spanish and Swedish (and so on) to the parser. The develop branch uses this ECL parser.

danka74 commented 5 years ago

@alopezo , I tried a APG parser based on the grammar (https://github.com/IHTSDO/SNOMEDCT-Languages) and it failed with the same error. What was your successful setting? Mine was V8 v3.14 running on CentOS7.

alopezo commented 5 years ago

Sorry for the late reply, I tried here: https://www.ihtsdo.org/SCT-ecl-brief.html But I now realize it does not parse, I was expecting a warning but is just the "false" value in one of the properties.

danka74 commented 5 years ago

@alopezo, that page is using the javascript parser running internally in the browser (as you probably know, perhaps you wrote it?). There was work on this done in the above mentioned project by fixing the ANTLR code generated from ABNF. The problem seems to be that the ABNF code assumes a byte stream instead of a character stream for Unicode characters beyond ASCII. If you are familiar with ABNF, perhaps you can have a go at fixing the ABNF?

kaicode commented 5 years ago

Included in release 2.2.0.

alopezo commented 5 years ago

Thanks @kaicode