search

timsbiomed / owl-on-fhir-content

Static OWL content, pre and post converted to FHIR.
MIT License
0 stars 0 forks source link

Concept properties with dots prevent the loading of code systems #7

Closed ShahimEssaid closed 1 year ago

ShahimEssaid commented 1 year ago

I think having dots in the property names (i.e. the "code" value) is an issue for HAPI as of now. It leads to the following exception and the loading transaction is then aborted. Let's try to fix this for the rxnorm file as a test case to see if replacing dots with underscores will fix this.

Caused by: org.hibernate.search.util.common.SearchException: HSEARCH000598: Invalid type: field 'P:http://purl' is not composite.
    at org.hibernate.search.engine.search.common.spi.SearchIndexSchemaElementContextHelper.throwingToComposite(SearchIndexSchemaElementContextHelper.java:22)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexValueField.toComposite(AbstractIndexValueField.java:49)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexFieldTemplate.lambda$createNodeIfMatching$0(AbstractIndexFieldTemplate.java:54)
    at java.base/java.util.Optional.map(Optional.java:260)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexFieldTemplate.createNodeIfMatching(AbstractIndexFieldTemplate.java:51)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexModel.fieldOrNullIgnoringInclusion(AbstractIndexModel.java:120)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexModel.fieldOrNull(AbstractIndexModel.java:97)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexFieldTemplate.lambda$createNodeIfMatching$0(AbstractIndexFieldTemplate.java:53)
    at java.base/java.util.Optional.map(Optional.java:260)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexFieldTemplate.createNodeIfMatching(AbstractIndexFieldTemplate.java:51)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexModel.fieldOrNullIgnoringInclusion(AbstractIndexModel.java:120)
    at org.hibernate.search.engine.backend.document.model.spi.AbstractIndexModel.fieldOrNull(AbstractIndexModel.java:97)
    at org.hibernate.search.backend.lucene.document.impl.AbstractLuceneDocumentElementBuilder.addValue(AbstractLuceneDocumentElementBuilder.java:80)
    at org.hibernate.search.backend.lucene.document.impl.LuceneRootDocumentBuilder.addValue(LuceneRootDocumentBuilder.java:19)
    at ca.uhn.fhir.jpa.entity.TermConceptPropertyBinder$TermConceptPropertyBridge.write(TermConceptPropertyBinder.java:73)
    at ca.uhn.fhir.jpa.entity.TermConceptPropertyBinder$TermConceptPropertyBridge.write(TermConceptPropertyBinder.java:62)
    at org.hibernate.search.mapper.pojo.processing.impl.PojoIndexingProcessorPropertyBridgeNode.process(PojoIndexingProcessorPropertyBridgeNode.java:45)
    at org.hibernate.search.mapper.pojo.processing.impl.PojoIndexingProcessorPropertyNode.process(PojoIndexingProcessorPropertyNode.java:63)
    ... 85 common frames omitted
ShahimEssaid commented 1 year ago

This is related to Hibernate Search trying to create field names with the value of "code" but field names should not have dots in them. If we confirm this issue, the real fix probably belongs here:

https://github.com/hapifhir/hapi-fhir/blob/a9ecc18986cb5a1c66a6271daaffd197dc5eff53/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/TermConceptPropertyBinder.java#L74

and

https://github.com/hapifhir/hapi-fhir/blob/a9ecc18986cb5a1c66a6271daaffd197dc5eff53/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/TermConceptPropertyBinder.java#L76

ShahimEssaid commented 1 year ago

A follow up thought about the current fix, it is applying the . to _ to all URLs. This is not needed but we can revisit later. Only the concept.property.code path has to have values without ".". The other paths need to have the values as they really are intended to be as "code", "system", etc. so the value is the true identity of the concept being referenced. Let's discuss during a meeting.

The problem was (and this is a HAPI bug) that the HAPI indexing implementation was not sanitizing the value of "concept.property.code" before using it as an index field name, and the dots in our values caused the problem since they have a special meaning in index field names. I will create an issue in the HAPI repo to report our use case and the bug it created.

joeflack4 commented 1 year ago

Thanks for that explanation. I was thinking I'd have to not put URLs inside code field anyway.

Marking this closed as fixed in latest release. Reopen it and we can rename it if we want to talk about how to standardize this change. I know that the _ on URL thing is temporary. Personally, I was thinking we should use CURIEs, and I think I'll talk to Gaurav for the best standard practice on how to fully implement that.