invalid_bucket_props error with default settings

[allnightlong]

I'm trying to set single-node riak-cs instance. For test purpose, I use vagrant and ubuntu 14.04 LTS. After creating brand new virtual box, installing chef-dk, I've created Berksfile:

source "https://supermarket.getchef.com"

cookbook 'riak-cs'

then installing riak-cs cookbook with deps

vagrant@vagrant-ubuntu-trusty-64:/vagrant/chef-riak$ berks vendor cookbooks
Resolving cookbook dependencies...
Using apt (2.3.10)
Using build-essential (2.0.6)
Using chef_handler (1.1.6)
Using dmg (2.2.2)
Using erlang (1.5.6)
Using git (3.1.0)
Using packagecloud (0.0.16)
Using riak (2.4.18)
Using riak-cs (2.2.10)
Using runit (1.5.10)
Using sysctl (0.4.0)
Using ulimit (0.3.2)
Using windows (1.34.8)
Using yum (3.5.1)
Using yum-epel (0.5.3)
Using yum-erlang_solutions (0.2.0)
Vendoring apt (2.3.10) to cookbooks/apt
Vendoring build-essential (2.0.6) to cookbooks/build-essential
Vendoring chef_handler (1.1.6) to cookbooks/chef_handler
Vendoring dmg (2.2.2) to cookbooks/dmg
Vendoring erlang (1.5.6) to cookbooks/erlang
Vendoring git (3.1.0) to cookbooks/git
Vendoring packagecloud (0.0.16) to cookbooks/packagecloud
Vendoring riak (2.4.18) to cookbooks/riak
Vendoring riak-cs (2.2.10) to cookbooks/riak-cs
Vendoring runit (1.5.10) to cookbooks/runit
Vendoring sysctl (0.4.0) to cookbooks/sysctl
Vendoring ulimit (0.3.2) to cookbooks/ulimit
Vendoring windows (1.34.8) to cookbooks/windows
Vendoring yum (3.5.1) to cookbooks/yum
Vendoring yum-epel (0.5.3) to cookbooks/yum-epel
Vendoring yum-erlang_solutions (0.2.0) to cookbooks/yum-erlang_solutions

and finally run chef-client in local-mode (run-list was taken from docs)

sudo chef-client -z -r "recipe[riak-cs::package]","recipe[riak]","recipe[riak-cs]","recipe[riak-cs::stanchion]"

Chef run is successful, but riak-cs is not running:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/chef-riak$ sudo service riak status
pong
riak is running
vagrant@vagrant-ubuntu-trusty-64:/vagrant/chef-riak$ sudo service riak-cs status
Node 'riak-cs@10.0.2.15' not responding to pings.
riak-cs is stopped
vagrant@vagrant-ubuntu-trusty-64:/vagrant/chef-riak$ sudo service stanchion status
pong
stanchion is running

Here is console output:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/chef-riak$ sudo riak-cs console
Node 'riak-cs@10.0.2.15' not responding to pings.
config is OK
!!!!
!!!! WARNING: ulimit -n is 1024; 4096 is the recommended minimum.
!!!!
Exec: /usr/lib/riak-cs/erts-5.9.1/bin/erlexec -boot /usr/lib/riak-cs/releases/1.5.2/riak-cs              -config /etc/riak-cs/app.config             -pa /usr/lib/riak-cs/lib/basho-patches             -args_file /etc/riak-cs/vm.args -- console
Root: /usr/lib/riak-cs
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [async-threads:64] [kernel-poll:true]

=INFO REPORT==== 25-Nov-2014::16:30:50 UTC ===
opening log file: "/var/log/riak-cs/access.log.2014_11_25_16"
Eshell V5.9.1  (abort with ^G)
(riak-cs@10.0.2.15)1> /usr/lib/riak-cs/lib/os_mon-2.2.9/priv/bin/memsup: Erlang has closed.
                                                                                            Erlang has closed
                                                                                                             {"Kernel pid terminated",application_controller,"{application_start_failure,riak_cs,{invalid_bucket_props,{riak_cs_app,start,[normal,[]]}}}"}

Crash dump was written to: /var/log/riak-cs/erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,riak_cs,{invalid_bucket_props,{riak_cs_app,start,[normal,[]]}}})

And here is /var/log/riak-cs/erl_crash.dump (22k+ lines of code - can slow down your browser).

I've googled, but didn't find anything similar.

[cheeseplus]

@allnightlong I'm willing to wager that the riak configuration file is missing:

under riak_core {default_bucket_props, [{allow_mult, true}]}

If you notice in the .kitchen.yml

      config:
        riak_kv:
          storage_backend: riak_cs_kv_multi_backend

The above triggers this block in the riak-chef-cookbook attributes which sets some attributes that are needed for riak-cs operation: https://github.com/basho/riak-chef-cookbook/blob/riak-1.x/attributes/default.rb#L124

[allnightlong]

Yeah, it seems, you are right. After providing:

default['riak']['config']['riak_kv']['storage_backend'] = "riak_cs_kv_multi_backend"

riak-cs is up and running.

My fault, missed this item in the docs (relied on defaults).
But shouldn't the error message be more clear? Spend some time, to resolve this.

[cheeseplus]

The docs note the setting here: http://docs.basho.com/riakcs/latest/cookbooks/configuration/Configuring-Riak/#Allowing-for-Sibling-Creation

I'm all for better error messages but you'd have to take that up on the https://github.com/basho/riak_cs repo. As this isn't a cookbook issue specifically but an upstream documentation one I'm closing the issue.

[boddumanohar]

I am getting the same issue in the version of riak 2.2.0 I've followed the documentatation ./riak start -> works ./stanchion start -> works ./riak-cs start -> shows an error

config is OK
-config /usr/local/Cellar/stanchion-2.1.1/data/generated.configs/app.2017.01.12.19.48.19.config -args_file /usr/local/Cellar/stanchion-2.1.1/data/generated.configs/vm.2017.01.12.19.48.19.args -vm_args /usr/local/Cellar/stanchion-2.1.1/data/generated.configs/vm.2017.01.12.19.48.19.args
Exec:  /usr/local/Cellar/stanchion-2.1.1/bin/../erts-5.10.3/bin/erlexec -boot /usr/local/Cellar/stanchion-2.1.1/bin/../releases/2.1.0/stanchion               -config /usr/local/Cellar/stanchion-2.1.1/data/generated.configs/app.2017.01.12.19.48.19.config -args_file /usr/local/Cellar/stanchion-2.1.1/data/generated.configs/vm.2017.01.12.19.48.19.args -vm_args /usr/local/Cellar/stanchion-2.1.1/data/generated.configs/vm.2017.01.12.19.48.19.args              -pa /usr/local/Cellar/stanchion-2.1.1/bin/../lib/basho-patches -- console
Root: /usr/local/Cellar/stanchion-2.1.1/bin/..
Erlang R16B02_basho8 (erts-5.10.3) [source] [64-bit] [smp:4:4] [async-threads:64] [kernel-poll:true] [frame-pointer] [dtrace]

=INFO REPORT==== 12-Jan-2017::19:48:20 ===
    alarm_handler: {set,{{disk_almost_full,"/"},[]}}
Setup running ...
Directories verified. Res = {[ok],[]}
Setup finished processing hooks ...
Eshell V5.10.3  (abort with ^G)
(stanchion@127.0.0.1)1> 
=INFO REPORT==== 12-Jan-2017::19:48:20 ===
    alarm_handler: {clear,{disk_almost_full,"/"}}
{"Kernel pid terminated",application_controller,"{application_start_failure,stanchion,{{riak_connect_failed,{{tcp,econnrefused},\"127.0.0.1\",8087}},{stanchion_app,start,[normal,[]]}}}"}

Crash dump was written to: ./log/erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,stanchion,{{riak_connect_failed,{{tcp,econnrefused},"127.0.0.1",8087}},{stanchion_app,start,[normal,[]]}}})

riak-cs version 2.1.1

it looks like the problem is problem is disk_almost_full what is this error? and how do I overcome this error? My system configuration: OS : macOS 10.12.2 RAM : 4 GB