search

SeldonIO / seldon-server

Machine Learning Platform and Recommendation Engine built on Kubernetes
https://www.seldon.io/
Apache License 2.0
1.47k stars 299 forks source link

start start-microservice raise MicroserviceError("failed to run seldon cli to create conf"+name) #60

Open mullerhai opened 6 years ago

mullerhai commented 6 years ago

when I command

start-microservice --type prediction --client test -i iris-xgboost seldonio/iris_xgboost:2.1 rest 1.0

raise Exception
maybe zookeeper can not normal work

Traceback (most recent call last): File "/opt/conda/bin/seldon-cli", line 4, in connecting to zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181 [SUCCEEDED] import('pkg_resources').run_script('seldon==2.2.6', 'seldon-cli') File "/opt/conda/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/init.py", line 719, in run_script File "/opt/conda/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/pkg_resources/init.py", line 1504, in run_script File "/opt/conda/lib/python2.7/site-packages/seldon-2.2.6-py2.7.egg/EGG-INFO/scripts/seldon-cli", line 5, in seldon.cli.start_seldoncli() File "/opt/conda/lib/python2.7/site-packages/seldon-2.2.6-py2.7.egg/seldon/cli/init.py", line 3, in start_seldoncli cli_main.main() File "/opt/conda/lib/python2.7/site-packages/seldon-2.2.6-py2.7.egg/seldon/cli/cli_main.py", line 373, in main cmds[cmd](opts,command_data, command_args) File "/opt/conda/lib/python2.7/site-packages/seldon-2.2.6-py2.7.egg/seldon/cli/cmd_pred.py", line 203, in cmd_pred actions[action](command_data, opts) File "/opt/conda/lib/python2.7/site-packages/seldon-2.2.6-py2.7.egg/seldon/cli/cmd_pred.py", line 158, in action_create if not is_existing_client(zkroot, client_name): File "/opt/conda/lib/python2.7/site-packages/seldon-2.2.6-py2.7.egg/seldon/cli/cmd_pred.py", line 64, in is_existing_client client_names = os.listdir(zkroot + gdata["all_clients_node_path"]) OSError: [Errno 2] No such file or directory: '/seldon-data/conf/zkroot/all_clients' command terminated with exit code 1 Traceback (most recent call last): File "/home/muller/Documents/github/seldon-server/kubernetes/bin/start-microservice", line 313, in runner.run(args.type,args.client,args.i + args.p) File "/home/muller/Documents/github/seldon-server/kubernetes/bin/start-microservice", line 214, in run self.predictCreate(client,services) File "/home/muller/Documents/github/seldon-server/kubernetes/bin/start-microservice", line 203, in predictCreate raise MicroserviceError("failed to run seldon cli to create conf"+name) NameError: global name 'name' is not defined

NAME READY STATUS RESTARTS AGE influxdb-grafana-65fd8f899-x6r2f 2/2 Running 0 21m kafka-controller-94b9896d9-h8gzv 1/1 Running 0 17m memcached1-5ccc54f47d-6s99q 1/1 Running 0 21m memcached2-8d984bb95-52xlw 1/1 Running 0 21m mysql-c74858654-mqq52 1/1 Running 0 21m redis-74cf78b489-xlsfq 1/1 Running 0 21m seldon-control-7594c78659-7pz78 1/1 Running 0 21m spark-master-controller-7cfb76b44d-hdgvj 0/1 Pending 0 4m td-agent-server-54bd8d4f58-9xvn5 1/1 Running 0 17m zookeeper1-7769dc5c48-qvnsv 1/1 Running 0 21m zookeeper2-6754ddb54b-4qhxc 1/1 Running 0 21m zookeeper3-6749cd76d4-wd5sk 1/1 Running 0 21m

[root@localhost docker-compost]# kubectl get po NAME READY STATUS RESTARTS AGE influxdb-grafana-65fd8f899-x6r2f 2/2 Running 0 26m iris-xgboost-6ff4567f58-krsnz 1/1 Running 0 5m kafka-controller-94b9896d9-h8gzv 1/1 Running 0 22m memcached1-5ccc54f47d-6s99q 1/1 Running 0 26m memcached2-8d984bb95-52xlw 1/1 Running 0 26m mysql-c74858654-mqq52 1/1 Running 0 26m redis-74cf78b489-xlsfq 1/1 Running 0 26m seldon-control-7594c78659-7pz78 1/1 Running 0 26m spark-master-controller-7cfb76b44d-hdgvj 0/1 Pending 0 9m td-agent-server-54bd8d4f58-9xvn5 1/1 Running 0 22m zookeeper1-7769dc5c48-qvnsv 1/1 Running 0 26m zookeeper2-6754ddb54b-4qhxc 1/1 Running 0 26m zookeeper3-6749cd76d4-wd5sk 1/1 Running 0 26m

[root@localhost docker-compost]# kubectl get po NAME READY STATUS RESTARTS AGE influxdb-grafana-65fd8f899-x6r2f 2/2 Running 0 26m iris-xgboost-6ff4567f58-krsnz 1/1 Running 0 5m kafka-controller-94b9896d9-h8gzv 1/1 Running 0 22m memcached1-5ccc54f47d-6s99q 1/1 Running 0 26m memcached2-8d984bb95-52xlw 1/1 Running 0 26m mysql-c74858654-mqq52 1/1 Running 0 26m redis-74cf78b489-xlsfq 1/1 Running 0 26m seldon-control-7594c78659-7pz78 1/1 Running 0 26m spark-master-controller-7cfb76b44d-hdgvj 0/1 Pending 0 9m td-agent-server-54bd8d4f58-9xvn5 1/1 Running 0 22m zookeeper1-7769dc5c48-qvnsv 1/1 Running 0 26m zookeeper2-6754ddb54b-4qhxc 1/1 Running 0 26m zookeeper3-6749cd76d4-wd5sk 1/1 Running 0 26m

ukclivecox commented 6 years ago

Hi. It looks like the shared disk was not mounted. Are you running a cluster of multiple nodes? If so you would need a shared filesystem like glusterFS. BTW: this project is dormant, you might want to look at https://github.com/SeldonIO/seldon-core and see if this fits your needs?