search

EUDAT-B2HANDLE / B2HANDLE-HRLS

Provides a Java servlet for local Handle System reverse-lookups and searching.
Apache License 2.0
0 stars 6 forks source link

retrieverecords=true does not adhere to prefix when it is supplied. #54

Closed cookie33 closed 2 years ago

cookie33 commented 4 years ago

The following works as advertised:

curl -u "21.T12996:<password>" "https://epic-pid.storage.surfsara.nl:8003/hrls/handles/21.T12996?URL=*&limit=2"
["21.T12996/000000003C000009999","21.T12996/0000022C-1B5C-11EA-A670-FA163EEF9493"]

When adding the retrieverecords=true parameter it dos not adhere to the prefix anymore. We would expect the same handles. But this is not happening:

curl -u "21.T12996:<password>" "https://epic-pid.storage.surfsara.nl:8003/hrls/handles/21.T12996?URL=*&limit=2&retrieverecords=true"
{"21.T12995//DOC/OBJECT/1":[{"type":"URL","value":"https://www.rijksmuseum.nl/collection/SK-C-5"},{"type":"HS_ADMIN","value":"\u0007ó\u0000\u0000\u0000\u001021.T12995/USER01\u0000\u0000\u00011"}],"21.T12995/00000BA2-6710-11E8-A12B-FA163E7B6737":[{"type":"URL","value":"irods://130.186.13.178:1247/sdcCineca/cloud/import01june_rabbithole_500000_0263/000540_ODV_8031121_V0.txt"},{"type":"EUDAT/CHECKSUM_TIMESTAMP","value":"2018-06-03T09:25:01Z"},{"type":"EUDAT/PROFILE_VERSION","value":"1"},{"type":"EUDAT/CHECKSUM","value":"sha2:3aOQDc452gDzC63JZXX2A4BxFgDlJSt7IZA91/WgY5U"},{"type":"EUDAT/FIXED_CONTENT","value":"True"},{"type":"HS_ADMIN","value":"\u0007ó\u0000\u0000\u0000\u000E0.NA/21.T12995\u0000\u0000\u0000È"}]}

It shows handles for prefix 21.T12995 and NOT for 21.T12996 as requested.

cookie33 commented 4 years ago

I think it fails here in HandleReverseLookupResource.java:

        private void makeSearchSubquery(String prefix, String key, List<String> list, StringBuffer sb,
                        List<String> stringParams, Integer limit, Integer page, boolean retrieveRecords) {
                if (retrieveRecords) {
                        sb.append(
                                        "select handle, type, data from handles as allvalues inner join (select handle as subhandle from handles where type=?");
                } else {
                        if (prefix != null) {
                                sb.append("select handle from handles where handle like '" + prefix + "%'");
                                sb.append(" and type=?");
                        } else {
                                sb.append("select handle from handles where type=?");
                        }
                }

If retrieverecords is set it does not take in to account the prefix it seems.

cookie33 commented 4 years ago

Fixed in #55

cookie33 commented 3 years ago

Fixed in #57

cookie33 commented 3 years ago

The code is fixed. And the pull request merged it OK in the devel branch. Build a new reverselookupservlet-1.0.6.war and tested it:

$ ./testHrlsCmd.py -test hrls
Test hrls Script
test_ping (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that ping works. ... ok
test_search_handle_by_existing_key_value_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/000001'] returns specific handle. ... ok
test_search_handle_by_existing_key_value_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/000001','HS_ADMIN=*'] returns specific handle. ... ok
test_search_handle_by_existing_key_value_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['HS_ADMIN=*','URL=http://www.test_hrls_check.com/000001'] returns specific handle. ... ok
test_search_handle_by_existing_key_value_limit_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*'] returns 1000 handles. ... ok
test_search_handle_by_existing_key_value_limit_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','limit=10000'] returns 10000 handles. ... ok
test_search_handle_by_existing_key_value_limit_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','limit=100000'] returns 100000 handles. ... ok
test_search_handle_by_existing_key_value_limit_4 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','limit=200000'] returns 100000 handles. ... ok
test_search_handle_by_existing_key_value_page_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','page=0'] returns first 1000 handles. ... ok
test_search_handle_by_existing_key_value_page_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','page=1'] returns second 1000 handles. ... ok
test_search_handle_by_existing_key_value_page_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','page=2'] returns third 1000 handles. ... ok
test_search_handle_by_existing_key_value_page_and_limit_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','page=0','limit=10'] returns first 10 handles. ... ok
test_search_handle_by_existing_key_value_page_and_limit_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','page=1','limit=10'] returns second 10 handles. ... ok
test_search_handle_by_existing_key_value_page_and_limit_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','limit=10','page=0'] returns first 10 handles. ... ok
test_search_handle_by_existing_key_value_page_and_limit_4 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','limit=10','page=1'] returns second 10 handles. ... ok
test_search_handle_by_existing_key_value_retrieverecords_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/000001','retrieverecords=true'] returns all records for that handle. ... ok
test_search_handle_by_existing_key_value_retrieverecords_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/00000*','retrieverecords=true','limit=9'] returns all records for those handles. ... ok
test_search_handle_by_existing_key_value_retrieverecords_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/*','retrieverecords=true','limit=100000'] returns all records for those handles. ... ok
test_search_handle_by_existing_key_value_retrieverecords_except_hs_seckey_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=http://www.test_hrls_check.com/000001','retrieverecords=true'] returns all records for that handle except HS_SECKEY. ... ok
test_search_handle_by_non_existing_key_value_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=my_unknown_handle_url'] returns no matching handle. ... ok
test_search_handle_by_non_existing_key_value_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=my_unknown_handle_url','HS_ADMIN=*'] returns no matching handle. ... ok
test_search_handle_by_non_existing_key_value_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['HS_ADMIN=*','URL=my_unknown_handle_url'] returns no matching handle. ... ok
test_search_handle_by_prefix_existing_key_value_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['prefix','URL=http://www.test_hrls_check.com/*'] returns 1000 handles. ... ok
test_search_handle_by_prefix_existing_key_value_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['prefix;xyz','URL=http://www.test_hrls_check.com/*'] returns 1000 handles. ... ok
test_search_handle_by_prefix_existing_key_value_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['prefixi','URL=http://www.test_hrls_check.com/*'] returns no handles. ... ok
test_search_handle_by_prefix_existing_key_value_4 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['prefix i','URL=http://www.test_hrls_check.com/*'] returns error. ... ok
test_search_handle_by_prefix_existing_key_value_5 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['prefix','URL=http://www.test_hrls_check.com/000001','retrieverecords=true'] returns all records for that handle. ... ok
test_search_handle_by_prefix_existing_key_value_6 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['prefixi','URL=http://www.test_hrls_check.com/*','retrieverecords=true'] returns no handles. ... ok
test_search_handle_by_prohibited_key_value_1 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['HS_SECKEY=*'] returns specific message. ... ok
test_search_handle_by_prohibited_key_value_2 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['HS_SECKEY=*','URL=*'] returns specific message. ... ok
test_search_handle_by_prohibited_key_value_3 (testHrlsCmd.hrlsintgtest.HrlsIntegrationTests)
Test that search by ['URL=*','HS_SECKEY=*'] returns specific message. ... ok

----------------------------------------------------------------------
Ran 31 tests in 11.416s

OK

Also did install on a handle server with test prefixes only. Tested OK:

curl -u "21.T12995:<passwd>" https://url.nl:8003/hrls/handles/841?URL=*\&retrieverecords=true\&limit=1
{}

curl -u "21.T12995:<passwd>" https://url.nl:8003/hrls/handles/21.T12995?URL=*\&retrieverecords=true\&limit=1
{"21.T12995//DOC/OBJECT/1":[{"type":"URL","value":"<value>"},{"type":"HS_ADMIN","value":"\<value>"}]}

curl -u "21.T12995:<passwd>" https://url.nl:8003/hrls/handles/21.T12996?URL=*\&retrieverecords=true\&limit=1
{"21.T12996/000000003C000009999":[{"type":"URL","value":"<value>"},{"type":"HS_ADMIN","value":"<value>"}]}

So fixed..

cookie33 commented 2 years ago

Fixed in #56 and #57. They have been merged. So close issue