datastax / cstar_perf

Apache Cassandra performance testing platform
Apache License 2.0
72 stars 34 forks source link

Allow setting cdc directories through cluster_config.json #192

Closed nastra closed 8 years ago

nastra commented 8 years ago

This is for CASSANDRA-8844. We handle the cdc directories the same way as we do the commitlog_directory (e.g. not allowed to set them through the cstar_perf UI).

In case the cdc directories change with newer CASSANDRA-8844, we can easily reflect it in this PR.

mambocab commented 8 years ago

I can't get a test environment running in docker on openstack. When I try to launch the backend, I get the following error:

(env)automaton@adocker-d59186a:~/git/cstar_perf$ cstar_docker launch test_cluster 1 -m
INFO:cstar_docker:Launching a 2 node cluster with a separate client node ...
DEBUG:cstar_docker:docker run --ulimit memlock=100000000:100000000 --privileged --label cstar_node=true --label cluster_name=test_cluster --label cluster_type=cluster --label node=0  -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/bin/docker -d -m 2G --name=test_cluster_00  -h test_cluster_00 -v /home/automaton/git/cstar_perf:/home/cstar/git/cstar_perf datastax/cstar_docker
Traceback (most recent call last):
  File "/home/automaton/git/cstar_perf/env/bin/cstar_docker", line 9, in <module>
    load_entry_point('cstar-perf.tool==1.0', 'console_scripts', 'cstar_docker')()
  File "/home/automaton/git/cstar_perf/tool/cstar_perf/docker/cstar_docker.py", line 819, in main
    execute_cmd(args.command, args)
  File "/home/automaton/git/cstar_perf/tool/cstar_perf/docker/cstar_docker.py", line 695, in execute_cmd
    client_double_duty=args.client_double_duty)
  File "/home/automaton/git/cstar_perf/tool/cstar_perf/docker/cstar_docker.py", line 361, in launch
    node_data[node_name] = get_container_data(container_id)
  File "/home/automaton/git/cstar_perf/tool/cstar_perf/docker/cstar_docker.py", line 209, in get_container_data
    return json.loads(p.communicate()[0])[0]
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
(env)automaton@adocker-d59186a:~/git/cstar_perf$ 

The docker daemon does seem to be running. Other than that, I don't know enough about docker to debug this further. So, @mshuler, if you have time and a test environment, could you smoke test this please?

EnigmaCurry commented 8 years ago

Removing the stderr=subprocess.PIPE might elucidate what the error is. Can't see any reason we needed to pipe stderr

nastra commented 8 years ago

@mambocab so the problem you're seeing is because newer docker versions don't support _ in hostnames. See also https://github.com/docker/docker/issues/22057

$ docker run --ulimit memlock=100000000:100000000 --privileged --label cstar_node=true --label cluster_name=test_cluster --label cluster_type=cluster --label node=0  -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/bin/docker -d -m 2G --name=test_cluster_00  -h test_cluster_00 -v /home/automaton/git/cstar_perf:/home/cstar/git/cstar_perf datastax/cstar_docker
docker: invalid hostname format for --hostname: test_cluster_00.
See 'docker run --help'.

I will fix that and provide a PR

PS: I removed piping stderr in #195