Closed Roycohen closed 8 years ago
codis-config
has been removed from codis3.x. Please use codis-dashboard
and codis-fe
instead.
codis-dashboard
is equivalent to codis-config --dashboard
and codis-fe
is the frond end of codis-dashboad
.
You can find a bash script named demo.sh
under folder scripts
. It will setup a full featured codis-cluster for demo purpose.
Thanks you very much for the detailed answer. I will go over it and Let You Know how it is working for me.
Where I can find English Documentation for the new release? I'm eager to use codis for a long time and now is the time for me to make the switch :)
Do I have to install ZooKeeper and etcd?
You can choose either one of them.
I prefer etcd for demo purpose. You can clone it from github, and compile it with script build.sh
(can be found under its repo). Then the binary files etcd
and etcdctl
will be generated to bin
folder. You can copy them to any where you want (e.g. $GOPATH/bin
).
Great. I already manage to solve this. Now, What is the dashboard port. I can log in to several ports but I only see JSON.
In addition, what the demo config? And what is the use of etcd in this case? To manage to configurations?
sorry, these's no english document for this release yet. so I wrote some comments for you.
#!/bin/bash
PATH=$PATH:`realpath ../bin`
# preapre step 1.
# choose etcd as default coordinator
for x in etcd codis-dashboard codis-proxy codis-admin codis-server; do
which $x >/dev/null
if [ $? -ne 0 ]; then
echo "missing $x"
exit 1
fi
done
# prepare step 2.
# create a temp folder for demo (logs & configs)
rm -rf tmp; mkdir -p tmp && pushd tmp
if [ $? -ne 0 ]; then
echo "pushd failed"
exit 1
fi
trap "kill 0" EXIT SIGQUIT SIGKILL SIGTERM
# setup 1.
# run etcd with default configuration (listen on port 2379)
nohup etcd --name=codis-test &>etcd.log &
lastpid=$!
pidlist=$lastpid
echo "etcd.pid=$lastpid"
# setup 2.
# create 8 codis-server instances
# + listen on port 16379 ~ 16386
for ((i=0;i<8;i++)); do
let p="16379+i"
nohup codis-server --port ${p} &>redis-${p}.log &
lastpid=$!
pidlist="$pidlist $lastpid"
echo "codis-server-${p}.pid=$lastpid"
done
# setup 3.
# create 4 codis-proxy instances with default config
# + listen on port 19000 ~ 19003 for redis-client
# + listen on port 11080 ~ 11083 for codis-cluster (restful api)
# - can be visited by codis-dashboard or codis-admin
# + run `codis-proxy -h` for help
# - default config can be generated by `codis-proxy --default-config`
for ((i=0;i<4;i++)); do
let p1="11080+i"
let p2="19000+i"
cat > ${p1}.toml <<EOF
product_name = "codis-test"
product_auth = ""
proto_type = "tcp4"
admin_addr = "0.0.0.0:${p1}"
proxy_addr = "0.0.0.0:${p2}"
EOF
nohup codis-proxy -c ${p1}.toml &>${p1}.log &
lastpid=$!
pidlist="$pidlist $lastpid"
echo "proxy-${p1}x${p2}.pid=$lastpid"
done
# setup 4.
# create 1 codis-dashboard instance with default config (with etcd)
# + listen on port 18080 for codis-fe
cat > dashboard.toml <<EOF
coordinator_name = "etcd"
coordinator_addr = "127.0.0.1:2379"
product_name = "codis-test"
product_auth = ""
admin_addr = "0.0.0.0:18080"
EOF
nohup codis-dashboard -c dashboard.toml &> dashboard.log &
lastpid=$!
pidlist="$pidlist $lastpid"
echo "dashboard.pid=$lastpid"
# setup 5.
# setup codis-fe for codis-dashboard
cat > codis.json <<EOF
[
{
"name": "codis-test",
"dashboard": "127.0.0.1:18080"
}
]
EOF
nohup ../../bin/codis-fe -d codis.json --listen 0.0.0.0:8080 &> fe.log &
lastpid=$!
pidlist="$pidlist $lastpid"
echo "fe.pid=$lastpid"
sleep 3
for pid in $pidlist; do
ps -p $pid >/dev/null
if [ $? -ne 0 ]; then
echo "pid=$pid not found"
exit 1
fi
done
codis_admin() {
codis-admin --dashboard=127.0.0.1:18080 $@
if [ $? -ne 0 ]; then
echo "codis-admin error: $@"
exit 1
fi
}
# setup 6.
# create 4 groups (group-1 ~ group-4)
for ((i=0;i<4;i++)); do
let g="i+1"
codis_admin --create-group --gid $g
done
# setup 7.
# 2 codis-server for each group (1master + 1slave)
for ((i=0;i<8;i++)); do
let p="16379+i"
let g="i/2+1"
codis_admin --group-add --gid $g -x 127.0.0.1:${p}
done
# setup 8.
# add codis-proxy instances to codis-cluster
for ((i=0;i<4;i++)); do
let p1="11080+i"
codis_admin --create-proxy -x 127.0.0.1:${p1}
done
# setup 9.
# setup slots (auto-rebalance 1024 slots into 4 groups)
codis_admin --slot-action --interval=100
codis_admin --rebalance --confirm
echo done
# setup 10.
# all done!!
# please visit codis-fe (localhost:8080) with your browser
# + once the auto-rebalace phase has been completed, you can run redis-benchmark or anything else you want
while true; do
date
sleep 60
done
here's the architecture of codis3, https://raw.githubusercontent.com/CodisLabs/codis/release3.0/doc/pictures/architecture.png
we use zookeeper & etcd as external storage to save the configs of codis-clusters (such as slot mappings, proxy instances and group servers, etc .,)
@spinlock THANKS YOU VERY MUCH!!! You are much appreciated. I will continue with my tests and report if I have questions or issues.
Questions that pops out are:
NOTE: I've just tried to use rename method from PHP client and I'm getting an error. "read error on connection" Is rename method is supported?
I'm getting this error when trying to use: codis-admin --dashboard=127.0.0.1:18080 --create-group --gid 1 2016/07/06 17:50:12 dashboard.go:394: [PANIC] call rpc create-group to dashboard 127.0.0.1:18080 failed
It seems like everything is fine but the codis-admin command doesn't work.
When trying to log in to web I get 403 forbidden. netstat -nap | grep 8080 tcp 0 0 :::18080 :::* LISTEN 21134/codis-dashboa tcp 0 0 :::8080 :::* LISTEN 21159/codis-fe tcp 0 0 ::ffff:192.168.56.4:8080 ::ffff:192.168.56.2:49978 ESTABLISHED 21159/codis-fe
Hi @Roycohen
Q1: Yep, you can choose any client that implements the RESP protocol. And here is the list of the recommended clients from the official website. Please note that, codis only supports a subset of redis commands, here is the blacklist.
Q2: codis-proxy will open 2 different tcp ports, you can find their definitions from its config file: one for your client program; and the second one, namely admin_addr
, can be used to communicate with codis-dashboard or codis-admin to share proxy's status or sync slot mappings, group servers, etc.,
Q3: codis-cluster means the codis-dashbard + codis-proxy(s) + codis-server(s). If you want to use multiple proxies, jodis may be your best choice. And we'll publish a go version of jodis as soon as possible.
Q4: please check the log of codis-dashboard and the detail of the panic message, it'll tell you some reasons. I think it was probably caused by 'group-1 already exists'.
@Roycohen codis supports most of the redis commands, including pipeline. but it doesn't support transactions or any other command which will block the tcp connection. once the codis-proxy detects any invalid commands from client or any failure from backend codis-server, it'll reset the client's connection. please check your proxy's log file, it will tell you more information.
I have issues reloading dashboard if I don't delete the old data in etcd.
2016/07/10 11:23:35 topom.go:121: [ERROR] store: acquire lock of aniview-realtime-analytics failed
[error]: 105: Key already exists (/codis3/aniview-realtime-analytics/topom) [8]
4 /usr/src/go/src/github.com/CodisLabs/codis/pkg/models/etcd/etcdclient.go:133
github.com/CodisLabs/codis/pkg/models/etcd.(*EtcdClient).Create
3 /usr/src/go/src/github.com/CodisLabs/codis/pkg/models/store.go:63
github.com/CodisLabs/codis/pkg/models.(*Store).Acquire
2 /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:120
github.com/CodisLabs/codis/pkg/topom.(*Topom).setup
1 /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:95
github.com/CodisLabs/codis/pkg/topom.New
0 /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:116
main.main
... ...
[stack]:
2 /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:121
github.com/CodisLabs/codis/pkg/topom.(*Topom).setup
1 /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:95
github.com/CodisLabs/codis/pkg/topom.New
0 /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:116
main.main
... ...
2016/07/10 11:23:35 main.go:118: [PANIC] create topom with config file failed
coordinator_name = "etcd"
coordinator_addr = "127.0.0.1:2379"
admin_addr = "0.0.0.0:18080"
product_name = "aniview-realtime-analytics"
product_auth = ""
[error]: store: acquire lock of aniview-realtime-analytics failed
2 /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:122
github.com/CodisLabs/codis/pkg/topom.(*Topom).setup
1 /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:95
github.com/CodisLabs/codis/pkg/topom.New
0 /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:116
main.main
... ...
[stack]:
0 /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:118
main.main
... ...
@Roycohen you can remove the lock node manually with command codis-admin --remove-lock
.
@spinlock thank you very much. Do you know why I might have 403 forbidden? From the logs it seems like everything was initialized as it should.
@Roycohen what's your error message?
It may be caused by some failures while loading the css resources.
Please make sure that codis-fe
and the folder assets
are under the same directory and try again.
@spinlock, I think this is the issue. OK, After using the codis-fe in the binary the page loads.
Basically what I'm doing is: After compiling Codis I'm taking the binaries to the /usr/local/bin so I'm not using the codis-fe from the /usr/local/bin directory. So basically, all the binaries that I'm using are not in the bin folder under the Codis library. Is that something that might make things wrong?
Is there a way to make it work? In addition another several questions:
BTW, after this long thread I will probably make a PR with Q&A So you guys will be able to use :)
@Roycohen, I think this problem may be caused by running codis-fe
without any path prefix.
Here's the source code and some comments for you:
// calculate the location with your command line arguments
binpath, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.PanicErrorf(err, "get path of binary failed")
}
// calculate the absolute path of `assets`
assets := filepath.Join(binpath, "assets")
fi, err := os.Stat(assets)
if err != nil {
log.PanicErrorf(err, "get stat of %s failed", assets)
}
After putting those binary files to /usr/local/bin
, you can run codis-fe
without any path prefix. But the calculated value of binpath
should be always the _current path_ which is wrong.
So I suggest to you guys to maybe add a '--assets-dir' argument where you will be able to decide the assets directory :) - It is pretty easy to add this.
Can yo please answer my questions above.. How can you iterate over keys. It is something that is really important in most applications.
Thank you for your suggestions, I'll add this argument as soon as possible. 😄
@Roycohen, SCAN
is not supported by codis-proxy.
@spinlock So how do you suggest to iterate over keys? Is there a way or something you know.
It can't be that companies are using Codis without the ability to iterate over keys. How do you suggest doing so?
It will be great if you can share with me a thought on how to achieve this. I must iterate over the keys...
@spinlock, any comment? 👍
@springlie, thank you very much. If someone implemented it, why you are not adding it to the official release? How and where should i put it? And probably I need to recompile Codis again?
@springlie, please help me to integrate it. It is a must in our env. And you could please answer the above.
@Roycohen it's very difficult to implement such command in codis. I'll consider about your requirements and suggestions, and I'll try to solve this problem in next release of codis3, but I can't give you any promise.
@spinlock thanks for the reply. I understand that you can't promise this kind of thing. But it is really a must feature in your amazing project. Seems like this is not a big issue for someone who knows the Codis system. Basically you need to iterate over the slots and then over the relevant redis's. Or you can just iterate over the redis servers and keeping a pointer in Codis where you stopped and from where to continue.. 💯
Will be awesome if you could add it to the next release. Even something that is not 100% optimized but stable will be enough for now.
BTW, You guys are doing a great work!
Hi @Roycohen ,
Thank you very much for your suggestions.
Now Codis3.1 implemented a new command named 'SLOTSSCAN' which meets your requirements. This command has a similar style to 'SCAN': slotsscan slotnum cursor [COUNT count]
. Here is the patch file 0001-slotsscan.patch, you can find more detail in it.
Please notice that dev3.1 is an unstable branch, and it will be released ASAP.
Hi Guys,
I'm installing codis with GO 1.7 and I've successfully compiled it with all dependencies. First there are issues resolving something in the Makefile but I could managed to resolve it.
Now after everything is compiled I have the follwoing files under bin directory: drwxr-xr-x. 4 root root 4096 Jul 4 16:10 assets -rwxr-xr-x. 1 root root 15041943 Jul 4 16:10 codis-admin -rwxr-xr-x. 1 root root 16271124 Jul 4 16:10 codis-dashboard -rwxr-xr-x. 1 root root 8843404 Jul 4 16:10 codis-fe -rwxr-xr-x. 1 root root 8620289 Jul 4 16:10 codis-ha -rwxr-xr-x. 1 root root 10079739 Jul 4 16:10 codis-proxy -rwxr-xr-x. 1 root root 6312957 Jul 4 16:08 codis-server -rw-r--r--. 1 root root 151 Jul 4 16:08 version
Where and how I can compile the codis-config. Do I need it in this version?