basho / riak

Riak is a decentralized datastore from Basho Technologies.
http://docs.basho.com
Apache License 2.0
3.92k stars 534 forks source link

will riak python client 2.7.0 work for Riak 3.0 version #1041

Closed macmarcdhas closed 3 years ago

macmarcdhas commented 3 years ago

HI All,

I am testing riak 3.0 using python client for riak (v2.7.0), the basic CRUD operation, it works fine with Riak 2.2.3.

These are my commands on python shell (python riak client 2.7.0 using Riak 3.0 permanent)

myClient = riak.RiakClient(host='localhost', pb_port=8087, protocol='pbc') myBucket = myClient.bucket('test') val1 = 1 key1 = myBucket.new('one', data=val1) key1.store()

Errors out: ValueError: not enough values to unpack (expected 3, got 2)

Should we give the bucket_type as the 3rd value, if yes, could you please let me know how should I provide it on python shell.

martinsumner commented 3 years ago

When Riak went open source, the decision was made to update the erlang clients only going forward as we make releases - there wasn't the bandwidth to keep individual language clients up to date. Most customers were either using the erlang client or had drifted towards using the PB or HTTP API's directly.

I'm still surprised this fails though, I don't think you're doing anything wrong. I would expect new features to be unavailable via the python client, not for it to crash out in this basic way in 3.0. I will try and recreate this today or tomorrow.

martinsumner commented 3 years ago

The python client is failing as it does a check of server_version when first connecting to Riak - and the response 3.0 isn't handled. I think it does this to get the capabilities to make sure it doesn't try using a feature that's not supported yet.

There might be an easy update to the python client to add support for the new server version in. I will try and look at this later.

martinsumner commented 3 years ago

ah, it may be even simpler. I think it can cope with the server version going up - but the latest release of Riak is returning 3.0 as the server version - and it is expecting a version of the form a.b.c - this is the failure to unpack (it expects 3 gets 2 when it tries to do this):

server_info = self.get_server_info()
ver = server_info['server_version']
(maj, min, patch) = [int(v) for v in ver.split('.')]
martinsumner commented 3 years ago

If you're building from source, you can correct this straight away. There is a line in the rebar.config file in the root of the riak folder:

https://github.com/basho/riak/blob/develop-3.0/rebar.config#L48

This line sets the version to "3.0". If you change that to "3.0.1" and make rel riak again - the python client will work again as before.

@Bob-The-Marauder @martincox - this is another gotcha to watch out for as we package/release.

macmarcdhas commented 3 years ago

Thank you Martin, This was quick one. We are using a Centos 7 based rpm for riak. There are few errors during the compiling from source, which I will check on.

macmarcdhas commented 3 years ago

@martinsumner Can we get an rpm with this update? so that we can deploy on our Centos 7 instance.

martinsumner commented 3 years ago

I've created an RPM, but rather than work out a way of sending it through to you, you can just follow a script and do it yourself. It only takes about 30 minutes.

To create an RPM I just started a vanilla Centos 7 VM on GCP (I assume this would work the same with other cloud providers) - and then went through this script:

sudo yum install gcc gcc-c++ glibc-devel make ncurses-devel openssl-devel automake autoconf java-1.8.0-openjdk-devel git rpm-build pam-devel

curl -O https://raw.githubusercontent.com/kerl/kerl/master/kerl
chmod a+x kerl
./kerl update releases
./kerl build 22.3.4 22.3.4

[this step takes some time]

./kerl install 22.3.4 ~/erlang/22.3.4
. ~/erlang/22.3.4/activate

[if you prefer to use a different OTP 20+ erlang version for this package, change the version in the last 3 steps]

git clone https://github.com/basho/riak.git
cd riak
git checkout mas-i1041-releasetag

[this branch is develop-3.0 but with the change to the rebar.config release version]

git tag 3.0.1u

[created a tag with a ‘u’ added to indicate user-specific]

make package

ls rel/pkg/out/packages/
martinsumner commented 3 years ago

As of release 3.0.2 the root cause of this is resolved, Riak version are now of the form 3.a.b again.