kramasamy
commented
6 years ago @cckellogg - can you check this?
Open sebastienpattyn93 opened 6 years ago
kramasamy
commented
6 years ago @cckellogg - can you check this?
cckellogg
commented
6 years ago @sebastienpattyn93 what version for heron are you running? What is the yaml for apiserver deployment? It looks like the heron-apiserver is configured wrong. I don't think there has been a release since the packaging has changed. This class does not exist org.apache.heron.uploader.dlog.DLUploader. If you change the org.apache to com.twitter it should work.
sebastienpattyn93
commented
6 years ago Hi @cckellogg I'm using the yaml from this [turotial] (https://apache.github.io/incubator-heron/docs/operators/deployment/schedulers/kubernetes/#general-kubernetes-clusters)
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
##
## Heron API server deployment
##
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: heron-apiserver
name: heron-apiserver
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: heron-apiserver
labels:
app: heron-apiserver
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: heron-apiserver
namespace: default
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: heron-apiserver
labels:
app: heron-apiserver
namespace: default
spec:
selector:
matchLabels:
app: heron-apiserver
replicas: 1
template:
metadata:
labels:
app: heron-apiserver
spec:
serviceAccountName: heron-apiserver
terminationGracePeriodSeconds: 0
tolerations:
- key: "node.kubernetes.io/not-ready"
operator: "Equal"
effect: "NoExecute"
tolerationSeconds: 10
- key: "node.alpha.kubernetes.io/notReady"
operator: "Equal"
effect: "NoExecute"
tolerationSeconds: 10
- key: "node.alpha.kubernetes.io/unreachable"
operator: "Equal"
effect: "NoExecute"
tolerationSeconds: 10
containers:
- name: heron-apiserver
image: heron/heron:latest
command: ["sh", "-c"]
args:
- >-
heron-apiserver
--base-template kubernetes
--cluster kubernetes
-D heron.statemgr.connection.string=zookeeper:2181
-D heron.kubernetes.scheduler.uri=http://localhost:8001
-D heron.executor.docker.image=heron/heron:latest
-D heron.class.uploader=org.apache.heron.uploader.dlog.DLUploader
-D heron.uploader.dlog.topologies.namespace.uri=distributedlog://zookeeper:2181/distributedlog
- name: kubectl-proxy
image: heron/kubectl:latest
command: ["sh", "-c"]
args:
- >
kubectl proxy -p 8001
---
apiVersion: v1
kind: Service
metadata:
name: heron-apiserver
spec:
selector:
app: heron-apiserver
ports:
- protocol: TCP
port: 9000
targetPort: 9000
type: NodePort
and this is my response when I curl the api server:
{
"heron.build.git.revision" : "ddbb98bbf173fb082c6fd575caaa35205abe34df",
"heron.build.git.status" : "Clean",
"heron.build.host" : "ci-server-01",
"heron.build.time" : "Sat Mar 31 09:27:19 UTC 2018",
"heron.build.timestamp" : "1522488439000",
"heron.build.user" : "release-agent",
"heron.build.version" : "0.17.8"
}@cckellogg any updates on this?
cristobalcl
commented
6 years ago Last week I posted an issue very similar to this one: #2919
As this issue included an error message in the CLI side like this Failed to instantiate uploader class 'org.apache.heron.uploader.dlog.DLUploader' and I don't, I thought at that moment they should be different problems. But now I'm thinking they are related and maybe we are using different CLI versions, because the errors in the server side are similar. I posted there a full server log.
Any progress on this? Did you get over somehow?
sprasad09
commented
6 years ago Getting the same error :(
[2018-08-20 22:55:33 -0700] [INFO]: Launching topology: 'WindowedWordCount' [2018-08-20 22:55:33 -0700] [INFO]: {'config_property': [], 'topology-file-name': '/Users/sprasad/.heron/examples/heron-streamlet-examples.jar', 'verbose': False, 'subcommand': 'submit', 'deploy_deactivated': False, 'submit_user': 'sprasad', 'cluster': 'kubernetes', 'extra_launch_classpath': '', 'deploy_mode': 'server', 'role': 'sprasad', 'dry_run_format': 'colored_table', 'dry_run': False, 'config_path': '/usr/local/heron/conf', 'topology_main_jvm_property': [], 'environ': 'default', 'service_url': 'http://localhost:8001/api/v1/proxy/namespaces/default/services/heron-apiserver:9000', 'topology-class-name': 'com.twitter.heron.examples.streamlet.WindowedWordCountTopology'} [2018-08-20 22:55:33 -0700] [ERROR]: Failed to instantiate uploader class 'org.apache.heron.uploader.dlog.DLUploader' [2018-08-20 22:55:33 -0700] [ERROR]: Failed to launch topology 'WindowedWordCount'
xiaoyao1991
commented
5 years ago I had similar issues trying to run it on minikube locally. Turns out that the docker image on docker hub is outdated. The artifact in the docker image have com.twitter.... as namespaces instead of org.apache.....
When following the instructions on https://apache.github.io/incubator-heron/docs/operators/deployment/schedulers/kubernetes/#minikube:
$ kubectl create -f https://raw.githubusercontent.com/twitter/heron/master/deploy/kubernetes/minikube/apiserver.yaml heron/deploy/kubernetes/minikube/apiserver.yaml -D heron.class.uploader=org.apache.heron.uploader.dlog.DLUploader to -D heron.class.uploader=com.twitter.heron.uploader.dlog.DLUploader. $ kubectl create -f <path_to_your_source>/heron/deploy/kubernetes/minikube/apiserver.yaml. xiaoyao1991
commented
5 years ago I also happened to find that the service_url on the doc does not work for my minikube setting on mac. I ended up using http://localhost:8001/api/v1/namespaces/default/services/heron-apiserver/proxy as my service_url
zhangcheng
commented
5 years ago Indeed this help, thanks @xiaoyao1991 for the tips
I have a Kubernetes cluster with 3 nodes where Heron is deployed. I am able to visit the heron ui and to curl the heron api-server.
When I try to submit a topology I keep getting the same error:
Looking in my Kubernetes POD logs I found the following error:
I tried with other example topologies from heron but got the same error.