basho / riak-nodejs-client

The Riak client for Node.js.
Apache License 2.0
72 stars 28 forks source link

Search does not return key in _yz_rk [JIRA: CLIENTS-908] #165

Closed bryanburgers closed 8 years ago

bryanburgers commented 8 years ago

When using Riak.Commands.YZ.Search.Builder, I get incorrect _yz_rk values. I expect the key of the item, but I get what seems like gibberish. _yz_id values are also not what I would expect.

For example, this is one of the returned docs from the node code:

{
  "score": 1,
  "_yz_rb": "ssh-sessions",
  "_yz_rt": "ssh-sessions",
  "_yz_rk": 400000,
  "_yz_id": 1,
  "hardwareId": 2,
  "state": "connected"
}

and the corresponding doc returned from a curl request: curl "http://localhost:8098/search/query/ssh-sessions?wt=json&q=hardwareId:\[0%20TO%20100\]"

{
  "state": "connected",
  "hardwareId": 2,
  "_yz_id": "1*ssh-sessions*ssh-sessions*4e05ed89-2a49-4f17-8885-ca79f0d292c0*41",
  "_yz_rk": "4e05ed89-2a49-4f17-8885-ca79f0d292c0",
  "_yz_rt": "ssh-sessions",
  "_yz_rb": "ssh-sessions"
}

This is in a test environment that has two nodes in a cluster.

I went as far as uninstalling riak from each of these servers, deleting all of /var/lib/riak, reinstalling, and then setting up just this search index with the four dummy values, and it continues to be a problem.

If there's any other information I can provide, let me know. I'm just trying to get spun up on riak development, so I'm not sure what to check or what to try next.

Additional information

Component Version
riak 2.1.4
Ubuntu 14.04.4
basho-riak-client 2.2.1
node 6.2.2
ssh-sessions.xml schema definition ``` xml _yz_id ```
Index set up commands ``` bash # Create the schema for the ssh-sessions index curl -X PUT http://localhost:8098/search/schema/ssh-sessions -H "Content-Type: application/xml" --data-binary @ssh-sessions.xml # Create the ssh-search index itself curl -X PUT http://localhost:8098/search/index/ssh-sessions -H "Content-Type: application/json" -d '{"schema":"ssh-sessions"}' # Create the bucket type sudo riak-admin bucket-type create ssh-sessions '{"props":{"search_index":"ssh-sessions","allow_mult":true}}' # Activate the bucket type sudo riak-admin bucket-type activate ssh-sessions ```
riak-search.js output (incorrect) ``` json { "numFound": 4, "maxScore": 1, "docs": [ { "score": 1, "_yz_rb": "ssh-sessions", "_yz_rt": "ssh-sessions", "_yz_rk": 9, "_yz_id": 1, "hardwareId": 1, "state": "request-connect" }, { "score": 1, "_yz_rb": "ssh-sessions", "_yz_rt": "ssh-sessions", "_yz_rk": 400000, "_yz_id": 1, "hardwareId": 2, "state": "connected" }, { "score": 1, "_yz_rb": "ssh-sessions", "_yz_rt": "ssh-sessions", "_yz_rk": 9490, "_yz_id": 1, "hardwareId": 3, "state": "request-disconnect" }, { "score": 1, "_yz_rb": "ssh-sessions", "_yz_rt": "ssh-sessions", "_yz_rk": 862, "_yz_id": 1, "hardwareId": 4, "state": "disconnected" } ] } ```
curl output (correct) ``` json { "responseHeader": { "status": 0, "QTime": 18, "params": { "shards": "192.168.1.135:8093/internal_solr/ssh-sessions", "q": "hardwareId:[0 TO 100]", "192.168.1.135:8093": "(_yz_pn:62 AND (_yz_fpn:62)) OR _yz_pn:61 OR (_yz_pn:58 AND (_yz_fpn:58)) OR _yz_pn:57 OR (_yz_pn:54 AND (_yz_fpn:54)) OR _yz_pn:53 OR (_yz_pn:50 AND (_yz_fpn:50)) OR _yz_pn:49 OR (_yz_pn:46 AND (_yz_fpn:46)) OR _yz_pn:45 OR (_yz_pn:42 AND (_yz_fpn:42)) OR _yz_pn:41 OR (_yz_pn:38 AND (_yz_fpn:38)) OR _yz_pn:37 OR (_yz_pn:34 AND (_yz_fpn:34)) OR _yz_pn:33 OR (_yz_pn:30 AND (_yz_fpn:30)) OR _yz_pn:29 OR (_yz_pn:26 AND (_yz_fpn:26)) OR _yz_pn:25 OR (_yz_pn:22 AND (_yz_fpn:22)) OR _yz_pn:21 OR (_yz_pn:18 AND (_yz_fpn:18)) OR _yz_pn:17 OR (_yz_pn:14 AND (_yz_fpn:14)) OR _yz_pn:13 OR (_yz_pn:10 AND (_yz_fpn:10)) OR _yz_pn:9 OR (_yz_pn:6 AND (_yz_fpn:6)) OR _yz_pn:5 OR (_yz_pn:2 AND (_yz_fpn:2)) OR _yz_pn:1", "wt": "json" } }, "response": { "numFound": 4, "start": 0, "maxScore": 1, "docs": [ { "state": "request-connect", "hardwareId": 1, "_yz_id": "1*ssh-sessions*ssh-sessions*9cdfe8d5-dd41-45f9-88bb-735c48ac3f15*41", "_yz_rk": "9cdfe8d5-dd41-45f9-88bb-735c48ac3f15", "_yz_rt": "ssh-sessions", "_yz_rb": "ssh-sessions" }, { "state": "connected", "hardwareId": 2, "_yz_id": "1*ssh-sessions*ssh-sessions*4e05ed89-2a49-4f17-8885-ca79f0d292c0*41", "_yz_rk": "4e05ed89-2a49-4f17-8885-ca79f0d292c0", "_yz_rt": "ssh-sessions", "_yz_rb": "ssh-sessions" }, { "state": "request-disconnect", "hardwareId": 3, "_yz_id": "1*ssh-sessions*ssh-sessions*09490ce5-6c0e-4004-ac27-1b7f361dde4e*17", "_yz_rk": "09490ce5-6c0e-4004-ac27-1b7f361dde4e", "_yz_rt": "ssh-sessions", "_yz_rb": "ssh-sessions" }, { "state": "disconnected", "hardwareId": 4, "_yz_id": "1*ssh-sessions*ssh-sessions*862b4a4f-f1f4-4f39-b3f1-cc8c66ee63db*1", "_yz_rk": "862b4a4f-f1f4-4f39-b3f1-cc8c66ee63db", "_yz_rt": "ssh-sessions", "_yz_rb": "ssh-sessions" } ] } } ```
riak-search.js – node.js code used to search ``` js const Riak = require('basho-riak-client'); const bucketType = 'ssh-sessions'; const bucket = 'ssh-sessions'; const client = new Riak.Client(['127.0.0.1:8087'], (err, c) => { if (err) { console.log(err); return; } const search = new Riak.Commands.YZ.Search.Builder() .withIndexName('ssh-sessions') .withQuery('hardwareId:[0 TO 100]') .withCallback((err, rslt) => { if (err) { console.log(err); return; } console.log(rslt); c.stop(() => {}); }) .build(); c.execute(search); }); ```
riak-populate.js – the code I used to populate the data ``` js const Riak = require('basho-riak-client'); const async = require('async'); const data = { '9cdfe8d5-dd41-45f9-88bb-735c48ac3f15': { hardwareId: 1, state: 'request-connect' }, '4e05ed89-2a49-4f17-8885-ca79f0d292c0': { hardwareId: 2, state: 'connected', host: 'host1', port: 22 }, '09490ce5-6c0e-4004-ac27-1b7f361dde4e': { hardwareId: 3, state: 'request-disconnect', host: 'host2', port: 22 }, '862b4a4f-f1f4-4f39-b3f1-cc8c66ee63db': { hardwareId: 4, state: 'disconnected' }, }; const bucketType = 'ssh-sessions'; const bucket = 'ssh-sessions'; new Riak.Client(['127.0.0.1:8087'], (err, c) => { if (err) { console.log(err); return; } function populateKey(key, value, callback) { c.fetchValue({ bucketType, bucket, key, convertToJs: true }, (err, rslt) => { if (rslt.isNotFound) { c.storeValue({ bucketType, bucket, key, value: data[key] }, (err, rslt) => { if (err) { return callback(err); } console.log(`${key} stored.`); return callback(null); }); } else { const riakObj = rslt.values[0]; const value = riakObj.value; value.date = new Date().toISOString(); riakObj.setValue(value); c.storeValue({ value: riakObj }, (err, rslt) => { if (err) { return callback(err); } console.log(`${key} updated.`); return callback(null); }); } }); } async.parallel( Object.keys(data).map(key => { const value = data[key]; return callback => populateKey(key, value, callback); }), (err, results) => { if (err) { console.log(err); c.stop(() => {}); return; } c.stop(() => {}); } ); }); ```
lukebakken commented 8 years ago

I'll look into this. Thanks for the comprehensive issue report - I wish they were all like this :smile:

lukebakken commented 8 years ago

It's probably in this code where it tries to be smart about decoding the information returned from search.

To be honest it probably makes more sense to just leave everything as a string and let the user decide. Thoughts about that?

lukebakken commented 8 years ago

One clue: 4e05ed89 ... 4e05 is (probably) parsed as an integer in exponential format. How helpful. Using your data in a unit test now.

bryanburgers commented 8 years ago

That seems consistent with what I'm seeing: "9cdfe8d5-dd41-45f9-88bb-735c48ac3f15" => 9, "4e05ed89-2a49-4f17-8885-ca79f0d292c0" => 4e05 => 400000.

Another option I'll throw out there: specifically handle the well-known _yz_* entries in the result, and don't try to change those?

Though I personally like your idea of leaving everything a s astring and letting the user decide: I would rather have unconverted values than incorrect values.

lukebakken commented 8 years ago

This is a case of parseFloat trying to be a bit too helpful.

What I'll do is add an option to the search command builder to not convert documents. This way the change will remain backwards-compatible if anyone depends on this (admittedly wonky) behavior.

lukebakken commented 8 years ago

Actually I'm going to add that option and implement your suggestion about the _yz_* entries since those really should only be interpreted as strings.

lukebakken commented 8 years ago

@bryanburgers 2.2.2 is released, thanks again for the excellent issue report:

https://github.com/basho/riak-nodejs-client/releases/tag/v2.2.2

bryanburgers commented 8 years ago

@lukebakken You're welcome. I wanted this figured out quickly, and a thorough issue was the best way I knew how. #everybodywins

lukebakken commented 8 years ago

@bryanburgers - feel free to file an issue if you have questions about or suggestions for the Node.js client. I also keep an eye on the riak-users mailing list if you have Riak issues. Thanks again.