marklogic / entity-services

Data modeling and code scaffolding for data integration in MarkLogic
https://docs.marklogic.com/guide/entity-services
Apache License 2.0
7 stars 10 forks source link

Additional Query won't grab json Entities #359

Closed paxtonhare closed 6 years ago

paxtonhare commented 6 years ago

The search-options generated by search-options-generate has es: namespaces in the path. That will not allow it to work with JSON entity instances.

Can we get this changed to support both?

current output for my particular entity:

<search:options xmlns:search="http://marklogic.com/appservices/search">
  <search:constraint name="entity-type">
    <search:value>
      <search:element ns="http://marklogic.com/entity-services" name="title"/>
    </search:value>
  </search:constraint>
  <search:constraint name="sku">
    <search:range type="xs:string" facet="true" collation="http://marklogic.com/collation/codepoint">
      <search:path-index xmlns:es="http://marklogic.com/entity-services">//es:instance/Product/sku</search:path-index>
    </search:range>
  </search:constraint>
  <search:constraint name="id">
    <search:value>
      <search:element ns="" name="id"/>
    </search:value>
  </search:constraint>
  <search:constraint name="price">
    <search:range type="xs:decimal" facet="true">
      <search:element ns="" name="price" xmlns:es="http://marklogic.com/entity-services"/>
    </search:range>
  </search:constraint>
  <search:values name="Product">
    <search:range type="xs:string" facet="true" collation="http://marklogic.com/collation/codepoint">
      <search:path-index xmlns:es="http://marklogic.com/entity-services">//es:instance/Product/sku</search:path-index>
    </search:range>
  </search:values>
  <search:values name="Order">
    <search:range type="xs:decimal" facet="true">
      <search:element ns="" name="price" xmlns:es="http://marklogic.com/entity-services"/>
    </search:range>
  </search:values>
  <!--Uncomment to return no results for a blank search, rather than the default of all results
 <search:term xmlns:search="http://marklogic.com/appservices/search">
  <search:empty apply="no-results"/>
</search:term> 
-->
  <search:values name="uris">
    <search:uri/>
  </search:values>
  <!--Change to 'filtered' to exclude false-positives in certain searches-->
  <search:search-option>unfiltered</search:search-option>
  <!--Modify document extraction to change results returned-->
  <search:extract-document-data selected="include">
    <search:extract-path xmlns:es="http://marklogic.com/entity-services">//es:instance/(Product|Order)</search:extract-path>
  </search:extract-document-data>
  <!--Change or remove this additional-query to broaden search beyond entity instance documents-->
  <search:additional-query>
    <cts:element-query xmlns:cts="http://marklogic.com/cts">
      <cts:element xmlns:es="http://marklogic.com/entity-services">es:instance</cts:element>
      <cts:true-query/>
    </cts:element-query>
  </search:additional-query>
  <!--To return facets, change this option to 'true' and edit constraints-->
  <search:return-facets>false</search:return-facets>
  <!--To return snippets, comment out or remove this option-->
  <search:transform-results apply="empty-snippet"/>
</search:options>
grechaw commented 6 years ago

Yes, this is a bug. I missed the change in 9.0-3 work. I also ran into it the issue recently, annoying. Of course the short term workaround is to edit the options. If you generate a TDE template, you'll see the approach I took to address namespaces -- do you think something here would be similarly appropriate?

grechaw commented 6 years ago

Since this was brought to my attention, ill get on having a fix in 9.05

grechaw commented 6 years ago

Looks like this will do the trick -- what do you think @paxtonhare ?

 <search:additional-query>
             <cts:or-query xmlns:cts="http://marklogic.com/cts">
                 <cts:json-property-scope-query>
                     <cts:property>instance</cts:property>
                     <cts:true-query/>
                 </cts:json-property-scope-query>
                 <cts:element-query>
                     <cts:element xmlns:es="http://marklogic.com/entity-services">es:instance</cts:element>
                     <cts:true-query/>
                 </cts:element-query>
             </cts:or-query>
</search:additional-query>
paxtonhare commented 6 years ago

Looks good to me.

On Tue, Feb 13, 2018 at 2:38 PM, Charles Greer notifications@github.com wrote:

Looks like this will do the trick -- what do you think @paxtonhare https://github.com/paxtonhare ?

1580 1581 1582 instance 1583 1584 1585 1586 es:instance 1587 1588 1589 1590 , — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or mute the thread .
grechaw commented 6 years ago

So this fix is in develop now, assigned to @bsrikan to ensure testing coverage (which in this case is not too critical i think...)

bsrikan commented 6 years ago

Tests added and running in regression. Shipping this.

epilith commented 4 years ago

This bugfix has caused another bug. Line 144 of this file causes problems with any namespace ending in es:. We have discovered this because we are using an entity with namespace pasdes: and it is replaced by pasd*: by this fix.

At the minimum this needs to be replaced by fn:replace($n, "^es:$", "*:") but ideally the namespace itself should be checked for the entity services namespace, rather than relying on the prefix to be set to a particular value.

ehennum commented 4 years ago

@epilith , this repository no longer reflects the current state of Entity Services, which was moved into the internal repository for MarkLogic server (because Entity Services is distributed with a server).

The best way to report an issue with Entity Services now is to work with MarkLogic support to file the issue.

I'll explore having this repository removed now so other users don't make the natural assumption that it reflects the current state of the code or that it is the right place to file issues. Apologies about the misdirect.