basho / riak

Riak is a decentralized datastore from Basho Technologies.
http://docs.basho.com
Apache License 2.0
3.92k stars 534 forks source link

Riak Admin Bucket-type create command fails when parsing JSON argument #1050

Closed aramallo closed 3 years ago

aramallo commented 3 years ago

The command fails when parsing the second argument to the riak admin bucket-type create command.

The json string is received by the riak_kv_console: bucket_type_create/2 as N lists as opposed to one.

$ sudo riak admin bucket-type create index_data '{"props":{"n_val":3, "pw":"quorum", "pr":"quorum"}}'
RPC to 'riak@127.0.0.1' failed: {'EXIT',
                                 {function_clause,
                                  [{riak_kv_console,bucket_type_create,
                                    [["index_data","{\"props\":{\"n_val\":3,",
                                      "\"pw\":\"quorum\",",
                                      "\"pr\":\"quorum\"}}"]],
                                    [{file,
                                      "/root/riak/rel/pkg/out/riak-3.0.3-OTP20.3/_build/default/lib/riak_kv/src/riak_kv_console.erl"},
                                     {line,499}]},
                                   {rpc,'-handle_call_call/6-fun-0-',5,
                                    [{file,"rpc.erl"},{line,197}]}]}}

The following works (as a workaround):

sudo riak admin bucket-type create index_data '{"props":{"datatype":"map"}}'

An later on update via HTTP API, as in:

curl -v -XPUT http://127.0.0.1:8098/buckets/index_data/props \
    -H "Content-Type: application/json" \
    -d '{"props":{"n_val":3, "pw":"quorum", "pr":"quorum"}}'

Riak Version: 3.0.3 OS: Ubuntu 18.04 OTP Version: 20.3 Installed from binary package: https://files.tiot.jp/riak/kv/3.0/3.0.3/ubuntu/bionic64/riak_3.0.3-OTP20.3_amd64.deb

martinsumner commented 3 years ago

The problem is the whitespace (within the json):

sudo riak admin bucket-type create index_data '{"props":{"n_val":3,"pw":"quorum","pr":"quorum"}}'
index_data created

WARNING: After activating index_data, nodes in this cluster
can no longer be downgraded to a version of Riak prior to 2.0

This is though, a new issue in 3.0 - with 2.9 the whitespace was tolerated

aramallo commented 3 years ago

Thanks a lot @martinsumner !!

martinsumner commented 3 years ago

I've had a quick look, and I'm not sure how to fix it without breaking something else. All the command line stuff is now parsed differently following the move to use rebar3 relx rather than the basho node_package. I think it would be easy to possibly escape out the whitespace here - https://github.com/basho/riak/blob/riak-3.0.3/rel/files/riak#L51 - but not without breaking other things.

Is removing whitespace in json a tolerable workaround? I would prefer not to fix, and risk breaking anything else (we don't have effective automated testing for the command line stuff).

aramallo commented 3 years ago

It is a tolerable workaround for us. So you can close this one if you want.

I see if I can manage to find some time to make the change and test it. BTW another improvement might be to move from mochijson to jsone or similar.

martincox commented 3 years ago

In #1051 I've added tr to truncate whitespace from the props string - I don't think any key/values legitimately need whitespace, so should be safe. Don't think it can be added in the regex in /usr/sbin/riak because it would strip whitespace between the command and args also.

martincox commented 3 years ago

Fixed #1051

aramallo commented 3 years ago

Thanks a lot @martincox !