Invalid JSON when trying to create a bucket type with custom properties

jorgerdc commented 8 months ago

In Riak 3.2.0 I'm trying to create the following bucket type:

riak admin bucket-type create sessions_v1 '{"props":{"n_val":3 }}'

I'm getting this error

Cannot create bucket type sessions_v1: invalid json error

As you can see, JSON is valid, not suere what's going on..

martinsumner commented 8 months ago

Which OS is this on?

Just done this on Ubuntu:

martin_sumner@riak-ubuntu20-arm:~/riak$ rel/riak/bin/riak daemon
martin_sumner@riak-ubuntu20-arm:~/riak$ rel/riak/bin/riak admin bucket-type create sessions_v1 '{"props":{"n_val":3 }}'
sessions_v1 created

WARNING: After activating sessions_v1, nodes in this cluster
can no longer be downgraded to a version of Riak prior to 2.0
ok
martin_sumner@riak-ubuntu20-arm:~/riak$

However, I know we had issues with riak admin commands for bucket creation (with JSON) on OSX in test. Is this related?

martinsumner commented 8 months ago

Just double-checked and the problem is that the riak admin script relies on this escape function:

https://github.com/basho/riak/blob/812ded6d64b080beeb068ba92f508556e6916d97/rel/files/riak-admin#L557-L559

The echo -n option is not supported consistently in different shells - including in OSX. Consequently the downstream code is trying to parse a json that looks like -n {"props":{"n_val":3 }} not {"props":{"n_val":3 }}.

There are a couple of workarounds. Use a bash shell, or you can set the bucket properties directly from riak remote_console (i.e. the erlang shell):

riak_kv_console:bucket_type_create(["sessions_v1", "{\"props\":{\"n_val\":3 }}"]).

martinsumner commented 8 months ago

The long term answer might be to swap echo -n for printf.

@hmmr - I think though that there is some history here about why echo-n is used - were there some challenges resolving this i.e. something else which broke if we changed it?

martinsumner commented 8 months ago

This all seemed familiar ... and now I remember why ... and its because a PR was raised by @aef- for this last year:

https://github.com/basho/riak/pull/1134

I approved it at the time, but deferred merging as we were not due to release a new version of 3.2, as the focus was on 3.0 updates.

@nsaadouni, @bet365-bspencer - as this has come up multiple times now, could you look at the PR?

hmmr commented 8 months ago

Yes -- printf instead of echo -n was precisely about -n not being available outside gnu in Linux. The proposed replacement seems to be sufficient.

jorgerdc commented 8 months ago

Which OS is this on?

Just done this on Ubuntu:
martin_sumner@riak-ubuntu20-arm:~/riak$ rel/riak/bin/riak daemon
martin_sumner@riak-ubuntu20-arm:~/riak$ rel/riak/bin/riak admin bucket-type create sessions_v1 '{"props":{"n_val":3 }}'
sessions_v1 created

WARNING: After activating sessions_v1, nodes in this cluster
can no longer be downgraded to a version of Riak prior to 2.0
ok
martin_sumner@riak-ubuntu20-arm:~/riak$
However, I know we had issues with riak admin commands for bucket creation (with JSON) on OSX in test. Is this related?

Hi, I'm using a docker container with ubuntu:jammy Here the details:

PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
root@d-riak-coord-jrc:/#

basho / riak_kv

Invalid JSON when trying to create a bucket type with custom properties #1885