Index for new range field doesn’t show keys [BUG]

[amclark42]

Describe the bug I created a range index field for a collection, but Monex doesn’t show any keys for that index. However, a query using the range functions confirms that the index has been populated.

Expected behavior I expected to see the keys indexed for the given field.

To Reproduce

xquery version "3.1";

  module namespace t="http://example.org/ns/test";
(:  NAMESPACES  :)
  declare namespace range="http://exist-db.org/xquery/range";
  declare namespace test="http://exist-db.org/xquery/xqsuite";
  declare namespace xdb="http://exist-db.org/xquery/xmldb";

(:~
  Check to make sure a named field exists in the new range index.

  @author Ash Clark
  2022
 :)

(:  VARIABLES  :)
  declare variable $t:pseudojson :=
    <fn:map>
      <fn:string key="id">apple</fn:string>
      <fn:array key="related">
        <fn:string>orange</fn:string>
        <fn:string>pear</fn:string>
      </fn:array>
    </fn:map>;

  declare variable $t:xconf :=
    <collection xmlns="http://exist-db.org/collection-config/1.0">
      <index xmlns:fn="http://www.w3.org/2005/xpath-functions"
         xmlns:xs="http://www.w3.org/2001/XMLSchema">
        <range>
          <create qname="fn:string">
            <condition attribute="key" operator="eq" value="id"/>
            <field name="string-id" type="xs:string"/>
          </create>
          <create qname="@key" type="xs:string"/>
        </range>
      </index>
    </collection>;

(:  FUNCTIONS  :)

  declare %test:setUp function t:set-up() {
    let $dir := xdb:create-collection('/db', 'sudoJson')
    let $cDir := xdb:create-collection('/db/system/config/db', 'sudoJson')
    let $storeFiles := (
        xdb:store($cDir, 'collection.xconf', $t:xconf),
        xdb:store($dir, 'example.xml', $t:pseudojson)
      )
    return xdb:reindex('/db/sudoJson')
  };

  (:declare %test:tearDown function t:tear-down() {
    xdb:remove('/db/sudoJson'), xdb:remove('/db/system/config/db/sudoJson')
  };:)

  declare
    %test:assertEquals('apple')
  function t:id-string-field-indexed() {
    collection('/db/sudoJson')/range:index-keys-for-field('string-id', '', 
      function ($i, $d) { $i }, 10)
  };

  declare
    %test:assertExists
  function t:query-finds-apple-id() {
    collection('/db/sudoJson')//fn:string[@key eq 'id'][. eq 'apple']
  };

  declare
    %test:assertExists
  function t:range-finds-apple-id() {
    collection('/db/sudoJson')/range:field-eq('string-id', 'apple')
  };

  declare
    %test:assertEquals(2)
  function t:key-attributes-indexed() {
    let $instances :=
      util:index-keys(collection('/db/sudoJson')//@key, '', function ($i, $d) {
        $d[2]
      }, 10, 'range-index')
    return sum($instances)
  };

The test suite above should completely pass, and return something like:

<testsuites>
    <testsuite package="http://example.org/ns/test" timestamp="2022-03-25T10:06:51.3-04:00" tests="4" failures="0" errors="0" pending="0" time="PT0.017S">
        <testcase name="id-string-field-indexed" class="t:id-string-field-indexed"/>
        <testcase name="key-attributes-indexed" class="t:key-attributes-indexed"/>
        <testcase name="query-finds-apple-id" class="t:query-finds-apple-id"/>
        <testcase name="range-finds-apple-id" class="t:range-finds-apple-id"/>
    </testsuite>
</testsuites>

The tear-down function is commented out, because now you can check what Monex has:

• Visit the Monex index browsing page for /db/sudoJson.
• Click the link “Node” for the string-id indexed item.
• Check the number of keys returned.

There will be 0 results when there should be 1.

Context

• OS: macOS 11.6
• eXist-db Version: 6.0.1
• Java Version: openjdk 1.8.0_292
  • App Version: 3.0.4

[amclark42]

Putting my analysis here: I am able to get results in Monex if I change this module to use collection($indexes:collection)/range:index-keys-for-field instead of the function in the variable. I think this is because the function reference retrieved from function-lookup and stored in the variable $indexes:range-lookup (l.81) was not initialized with the explicit context of the collection.

I can definitely submit a pull request for this. However, I found that while my fix does produce a table of keys, the same key occurs in multiple rows with a frequency of 1, suggesting there’s a bigger underlying issue in eXist. eXist issue 4074 may be related.

[joewiz]

@amclark42 Thanks for sharing your findings. Your idea to submit a PR with your fix would be great.

I'd like to see us add some sample data to monex, along with a configuration file that demonstrates all of the indexes that monex is capable of displaying. This would help both people who are trying to debug their own datasets and people debugging/supporting monex and eXist itself. Your small test here would be a good starting point!

Other good candidates would be the test suites for the various types of indexes in eXist, e.g.:

• Lucene: https://github.com/eXist-db/exist/tree/develop/extensions/indexes/lucene/src/test
• NGram: https://github.com/eXist-db/exist/blob/develop/extensions/indexes/ngram/src/test
• Range: https://github.com/eXist-db/exist/tree/develop/extensions/indexes/range/src/test
• Sort: https://github.com/eXist-db/exist/blob/develop/extensions/indexes/sort/src/test
• Spatial: https://github.com/eXist-db/exist/tree/develop/extensions/indexes/spatial/src/test
• Combined: https://github.com/eXist-db/exist/blob/develop/extensions/indexes/indexes-integration-tests/src/test

I'd be happy to review the PR, help, etc. Thanks!