search

sitewhere / sitewhere-k8s

SiteWhere / Kubernetes integration including Helm Charts
18 stars 23 forks source link

Installation: "": no matches for kind "Deployment" in version "extensions/v1beta1" #154

Closed aravindj76 closed 3 years ago

aravindj76 commented 4 years ago

Describe the bug I am installing sitewhere and I am getting this error during install.

Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

Provide Information Output of helm version: Client: &version.Version{SemVer:"v2.16.9", GitCommit:"8ad7037828e5a0fca1009dabe290130da6368e39", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.16.9", GitCommit:"8ad7037828e5a0fca1009dabe290130da6368e39", GitTreeState:"clean"}

Output of kubectl version: Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:43:34Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Output of helm ls sitewhere None.

But I have already followed the steps provided in the development guide link and please find below the folders.

$ ls build.gradle LICENSE.txt service-device-registration service-label-generation sitewhere-cassandra sitewhere-influxdb CODE_OF_CONDUCT.md package-lock.json service-device-state service-outbound-connectors sitewhere-communication sitewhere-jdk gradle README.md service-event-management service-rule-processing sitewhere-configuration sitewhere-microservice gradle.properties service-asset-management service-event-search service-schedule-management sitewhere-core sitewhere-mongodb gradlew service-batch-operations service-event-sources service-streaming-media sitewhere-core-api sitewhere-solr gradlew.bat service-command-delivery service-inbound-processing service-web-rest sitewhere-core-lifecycle HEADER service-device-management service-instance-management settings.gradle sitewhere-grpc-client ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere$ pwd /home/ceyarkadmin/siteWhere/sitewhere

and please find my docker images.

$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE sitewhere/service-web-rest 2.1.1 7a390009cc94 5 days ago 160MB sitewhere/service-streaming-media 2.1.1 19726c38bef8 5 days ago 146MB sitewhere/service-schedule-management 2.1.1 bc1d80f9239d 5 days ago 143MB sitewhere/service-rule-processing 2.1.1 8ee1e57e6dc3 5 days ago 140MB sitewhere/service-outbound-connectors 2.1.1 d8eee561b66c 5 days ago 148MB sitewhere/service-label-generation 2.1.1 a2fea2e8cf35 5 days ago 141MB sitewhere/service-instance-management 2.1.1 1f11247fc559 5 days ago 145MB sitewhere/service-inbound-processing 2.1.1 5857ee1ccc7d 5 days ago 140MB sitewhere/service-event-sources 2.1.1 410e9b479e82 5 days ago 152MB sitewhere/service-event-search 2.1.1 e823bba56eba 5 days ago 142MB sitewhere/service-event-management 2.1.1 0e9d92bc2a53 5 days ago 146MB sitewhere/service-device-state 2.1.1 c2a38c413a2c 5 days ago 142MB sitewhere/service-device-registration 2.1.1 3254f04e4c4b 5 days ago 140MB sitewhere/service-device-management 2.1.1 9168243d2b88 5 days ago 142MB sitewhere/service-command-delivery 2.1.1 bb21068ac93d 5 days ago 143MB sitewhere/service-batch-operations 2.1.1 ac566456ff8e 5 days ago 142MB sitewhere/service-asset-management 2.1.1 04fcbab910ca 5 days ago 143MB gcr.io/k8s-minikube/kicbase v0.0.10 e6bc41c39dc4 3 months ago 974MB analyticsserverdocker_ui latest 036cda8e4821 4 months ago 134MB

48da91e3d8f5 4 months ago 1.09GB analyticsserverdocker_apis latest d3ea7ceca838 4 months ago 941MB analyticsserverdocker_python-tracker-module latest 82ee427f0aaf 4 months ago 1.71GB analyticsserverdocker_logstash latest 65cbcc4b8114 4 months ago 682MB analyticsserverdocker_cassandra latest 2af89bb319a3 4 months ago 323MB analyticsserverdocker_kafka latest 5227b2f0fac1 4 months ago 332MB nginx latest 6678c7c2e56c 4 months ago 127MB python 3.6 1daf62e8cab5 5 months ago 914MB nvcr.io/nvidia/digits 20.02-tensorflow-py3 2c156780f489 5 months ago 10.5GB node 8 8eeadf3757f4 7 months ago 901MB nvcr.io/partners/omnisci-os cuda10-4.8.1 c676c23375eb 7 months ago 2.55GB nvcr.io/nvidia/tlt-streamanalytics v1.0.1_py2 5e2cce534d6e 8 months ago 7.83GB nvcr.io/nvidia/deepstream_360d 4.0.1-19.11 b1c139ad184a 8 months ago 3.63GB openjdk 8-jre-alpine f7a292bbb70c 14 months ago 84.9MB wurstmeister/zookeeper latest 3f43f72cb283 18 months ago 510MB openjdk 8u171-jre-alpine 0fe3f0d1ee48 22 months ago 83MB docker.elastic.co/logstash/logstash-oss 6.4.0 15e297c65be6 23 months ago 669MB docker.elastic.co/kibana/kibana-oss 6.4.0 c8986fcd3caa 23 months ago 493MB analyticsserverdocker_kibana latest c8986fcd3caa 23 months ago 493MB docker.elastic.co/elasticsearch/elasticsearch-oss 6.4.0 a13ec2fa6275 23 months ago 678MB analyticsserverdocker_elasticsearch latest a13ec2fa6275 23 months ago 678MB cassandra 3.11.2 1d46448d0e52 2 years ago 323MB gettyimages/spark 2.2.0-hadoop-2.7 cdc6bc372162 2 years ago 751MB ceyarkadmin@ceyark-macserv1:~$ pwd /home/ceyarkadmin Cloud Provider/Platform (AKS, GKE, Minikube etc.): workstation running minikube $ minikube start 😄 minikube v1.12.1 on Ubuntu 18.04 ✨ Using the docker driver based on existing profile 👍 Starting control plane node minikube in cluster minikube 🔄 Restarting existing docker container for "minikube" ... 🐳 Preparing Kubernetes v1.18.3 on Docker 19.03.2 ... 🔎 Verifying Kubernetes components... 🌟 Enabled addons: dashboard, default-storageclass, storage-provisioner 🏄 Done! kubectl is now configured to use "minikube" SiteWhere App Version: 2.1.1 SiteWhere Chart Version: $ helm dep list NAME VERSION REPOSITORY STATUS sitewhere-infra-core 0.2.7 https://sitewhere.io/helm-charts ok sitewhere-infra-database 0.2.12 https://sitewhere.io/helm-charts ok **To Reproduce** 1. followed the steps provided in Sitewhere development guide 2. then followed the steps provided in installing a Sitewhere instance link to install necessary software tools including kubernetes, docker, minkube, metallb, helm, istio 3. then followed the steps provided in deployment guide. 4. got the below error during one of the steps. ~/kubernetes/sitewhere/sitewhere-k8s/charts$ helm install --name sitewhere -f ./sitewhere/values-dev.yaml --set sitewhere-infra-database.mongodb.persistence.storageClass=hostpath --set sitewhere-infra-core.kafka.persistence.storageClass=hostpath --set sitewhere-infra-core.kafka.zookeeper.persistence.storageClass=hostpath ./sitewhere Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" then executed below command ~/kubernetes/sitewhere/sitewhere-k8s/charts$ helm install --name sitewhere -f ./sitewhere/values-dev.yaml sitewhere Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" Same error. **Expected behavior** if i execute the command "kubectl get po", then I want to see the microservices running. **Additional context** Please help to solve this issue.
jorgevillaverde-sitewhere commented 4 years ago

Can you try using:

helm install --name sitewhere \
 -f ./sitewhere/values-dev.yaml \
  --set sitewhere-infra-database.mongodb.persistence.storageClass=hostpath \
  --set sitewhere-infra-core.kafka.persistence.storageClass=hostpath \
  --set sitewhere-infra-core.kafka.zookeeper.persistence.storageClass=hostpath \
  sitewhere/sitewhere
aravindj76 commented 4 years ago

Hi Jorge,

Thanks for your reply. I ran the command and got the below error.

$ helm install --name sitewhere -f ./sitewhere/values-dev.yaml --set sitewhere-infra-database.mongodb.persistence.storageClass=hostpath --set sitewhere-infra-core.kafka.persistence.storageClass=hostpath --set sitewhere-infra-core.kafka.zookeeper.persistence.storageClass=hostpath sitewhere/sitewhere Error: failed to download "sitewhere/sitewhere" (hint: running helm repo update may help)

$ helm repo update Hang tight while we grab the latest from your chart repositories... ...Skip local chart repository ...Successfully got an update from the "stable" chart repository Update Complete.

Then I ran the install and I got the same error.

Am I having wrong file structures for the software to work.

My compiled development code exist in $pwd /home/ceyarkadmin/siteWhere/sitewhere ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere$ ls build.gradle service-instance-management CODE_OF_CONDUCT.md service-label-generation gradle service-outbound-connectors gradle.properties service-rule-processing gradlew service-schedule-management gradlew.bat service-streaming-media HEADER service-web-rest LICENSE.txt settings.gradle package-lock.json sitewhere-cassandra README.md sitewhere-communication service-asset-management sitewhere-configuration service-batch-operations sitewhere-core service-command-delivery sitewhere-core-api service-device-management sitewhere-core-lifecycle service-device-registration sitewhere-grpc-client service-device-state sitewhere-influxdb service-event-management sitewhere-jdk service-event-search sitewhere-microservice service-event-sources sitewhere-mongodb service-inbound-processing sitewhere-solr

The K8s charts is in the following path pwd /home/ceyarkadmin/siteWhere/sitewhere-k8s/charts ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ ls common-issues.md sitewhere sitewhere-infra-database README.md sitewhere-infra-core warp10

Can you please guide here If I have restructure the files path to resolve this issue.

Thanks.

jorgevillaverde-sitewhere commented 4 years ago

From this: 

helm repo update
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.

It seems that you don't have sitewhere helm repository. Please, add it, update and install again:

helm repo add sitewhere https://sitewhere.io/helm-charts
helm repo update
helm install --name sitewhere \
 -f ./sitewhere/values-dev.yaml \
  --set sitewhere-infra-database.mongodb.persistence.storageClass=hostpath \
  --set sitewhere-infra-core.kafka.persistence.storageClass=hostpath \
  --set sitewhere-infra-core.kafka.zookeeper.persistence.storageClass=hostpath \
  sitewhere/sitewhere
aravindj76 commented 4 years ago

Thanks for your reply. I executed the two commands you gave me. But still have the same error. Please find my commands executed. ---------------start of logs--------------------------------------------------------------------- ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ kubectl get svc -n istio-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-ingressgateway LoadBalancer 10.110.93.72 192.168.99.96 15021:31586/TCP,80:30801/TCP,443:31464/TCP,15443:30166/TCP 5d10h istiod ClusterIP 10.107.41.77 15010/TCP,15012/TCP,443/TCP,15014/TCP,853/TCP 5d10h prometheus ClusterIP 10.99.66.25 9090/TCP 5d10h ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 443/TCP 5d10h ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ helm install --name sitewhere -f ./sitewhere/values-dev.yaml sitewhere Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1" ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ helm install --name sitewhere sitewhere/sitewhere

NAME: sitewhere LAST DEPLOYED: Sun Aug 2 20:41:29 2020 NAMESPACE: default STATUS: DEPLOYED

RESOURCES: ==> v1/ConfigMap NAME DATA AGE sitewhere-influxdb 1 12s sitewhere-syncope-environment-config 10 11s sitewhere-warp10 1 11s

==> v1/Deployment NAME READY UP-TO-DATE AVAILABLE AGE sitewhere-asset-management 0/1 1 0 15s sitewhere-batch-operations 0/1 1 0 13s sitewhere-command-delivery 0/1 1 0 13s sitewhere-device-management 0/1 1 0 14s sitewhere-device-registration 0/1 1 0 14s sitewhere-device-state 0/1 1 0 14s sitewhere-event-management 0/1 1 0 14s sitewhere-event-search 0/1 1 0 14s sitewhere-event-sources 0/1 1 0 14s sitewhere-inbound-processing 0/1 1 0 14s sitewhere-instance-management 0/1 1 0 14s sitewhere-label-generation 0/1 1 0 14s sitewhere-mosquitto 0/1 1 0 14s sitewhere-outbound-connectors 0/1 1 0 14s sitewhere-rule-processing 0/1 1 0 15s sitewhere-schedule-management 0/1 1 0 15s sitewhere-streaming-media 0/1 1 0 15s sitewhere-syncope 0/1 1 0 15s sitewhere-syncope-console 0/1 1 0 15s sitewhere-syncope-enduser 0/1 1 0 15s sitewhere-web-rest 0/1 1 0 13s

==> v1/Pod(related) NAME READY STATUS RESTARTS AGE sitewhere-asset-management-6f4f47844b-jr9jm 0/2 Init:0/1 0 20s sitewhere-batch-operations-765b7b7df4-7vv4b 0/2 Init:0/1 0 19s sitewhere-cassandra-0 0/1 ContainerCreating 0 32s sitewhere-command-delivery-5ff76b6797-q9snx 0/2 Init:0/1 0 22s sitewhere-cp-kafka-0 0/1 ContainerCreating 0 33s sitewhere-cp-kafka-1 0/1 ContainerCreating 0 32s sitewhere-cp-kafka-2 0/1 ContainerCreating 0 31s sitewhere-cp-zookeeper-0 0/1 ContainerCreating 0 30s sitewhere-cp-zookeeper-1 0/1 ContainerCreating 0 29s sitewhere-cp-zookeeper-2 0/1 ContainerCreating 0 28s sitewhere-device-management-677fc47656-zdvbq 0/2 Init:0/1 0 23s sitewhere-device-registration-6d95f76f47-gdwmf 0/2 Init:0/1 0 20s sitewhere-device-state-6f9877f9f4-l5rdz 0/2 Init:0/1 0 22s sitewhere-event-management-6b65bd7b46-2pdjf 0/2 Init:0/1 0 23s sitewhere-event-search-64695685c8-jmczm 0/2 Init:0/1 0 22s sitewhere-event-sources-b498b76d7-8npbx 0/2 Init:0/1 0 26s sitewhere-inbound-processing-76f4797697-wxqpn 0/2 Init:0/1 0 24s sitewhere-influxdb-0 0/1 Running 0 35s sitewhere-instance-management-54bf57d5f4-9j9n6 0/2 Init:0/1 0 28s sitewhere-label-generation-78f8b6bb94-c6lg4 0/2 Init:0/1 0 28s sitewhere-mongodb-arbiter-0 0/1 ContainerCreating 0 36s sitewhere-mongodb-primary-0 0/1 ContainerCreating 0 32s sitewhere-mongodb-secondary-0 0/1 ContainerCreating 0 35s sitewhere-mosquitto-84444d4fb8-8v92c 0/1 ContainerCreating 0 30s sitewhere-outbound-connectors-75d5c6b695-szk7q 0/2 Init:0/1 0 25s sitewhere-postgresql-0 0/1 ContainerCreating 0 32s sitewhere-rule-processing-5b57ccf8cf-nkqcg 0/2 Init:0/1 0 28s sitewhere-schedule-management-6b6c788d84-b8wff 0/2 Init:0/1 0 30s sitewhere-streaming-media-79b59c5856-ktm2q 0/2 Init:0/1 0 29s sitewhere-syncope-7fc989f5f6-pb4j9 0/1 Init:0/1 0 21s sitewhere-syncope-console-66c66ff696-5pjrr 0/1 ContainerCreating 0 21s sitewhere-syncope-enduser-56877b8875-sxqfx 0/1 ContainerCreating 0 22s sitewhere-tenantsdb-0 0/1 ContainerCreating 0 35s sitewhere-warp10-0 0/1 ContainerCreating 0 31s sitewhere-web-rest-859b684955-rkbk8 0/2 Init:0/1 0 20s

==> v1/Secret NAME TYPE DATA AGE sitewhere-tenantsdb Opaque 1 11s syncope Opaque 1 11s

==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE sitewhere-asset-management-svc ClusterIP None 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-batch-operations-svc ClusterIP None 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-cassandra ClusterIP None 7000/TCP,7001/TCP,7199/TCP,9042/TCP,9160/TCP 12s sitewhere-command-delivery-svc ClusterIP 10.98.29.203 9001/TCP,9090/TCP 13s sitewhere-cp-kafka ClusterIP 10.111.211.66 9092/TCP 11s sitewhere-cp-kafka-headless ClusterIP None 9092/TCP 11s sitewhere-cp-zookeeper ClusterIP 10.110.117.222 2181/TCP 11s sitewhere-cp-zookeeper-headless ClusterIP None 2888/TCP,3888/TCP 11s sitewhere-device-management-svc ClusterIP None 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-device-registration-svc ClusterIP 10.110.235.72 9001/TCP,9090/TCP 12s sitewhere-device-state-svc ClusterIP 10.110.182.234 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-event-management-svc ClusterIP None 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-event-search-svc ClusterIP 10.108.67.71 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-event-sources-svc ClusterIP 10.104.235.14 9001/TCP,9090/TCP 13s sitewhere-inbound-processing-svc ClusterIP 10.106.185.47 9001/TCP,9090/TCP 14s sitewhere-influxdb ClusterIP 10.97.69.85 8086/TCP,8088/TCP 12s sitewhere-instance-management-svc ClusterIP 10.99.53.113 9000/TCP,9001/TCP,9004/TCP,9005/TCP,9090/TCP 13s sitewhere-label-generation-svc ClusterIP 10.106.243.233 9000/TCP,9001/TCP,9090/TCP 14s sitewhere-mongodb ClusterIP 10.101.252.22 27017/TCP 11s sitewhere-mongodb-headless ClusterIP None 27017/TCP 12s sitewhere-mosquitto-svc LoadBalancer 10.106.29.247 192.168.99.97 1883:32655/TCP 13s sitewhere-outbound-connectors-svc ClusterIP 10.109.205.185 9001/TCP,9090/TCP 13s sitewhere-postgresql ClusterIP 10.96.105.19 5432/TCP 10s sitewhere-postgresql-headless ClusterIP None 5432/TCP 11s sitewhere-rule-processing-svc ClusterIP 10.96.51.33 9001/TCP,9090/TCP 14s sitewhere-schedule-management-svc ClusterIP None 9000/TCP,9001/TCP,9090/TCP 14s sitewhere-streaming-media-svc ClusterIP 10.104.55.29 9000/TCP,9001/TCP,9090/TCP 13s sitewhere-syncope ClusterIP 10.108.58.55 8080/TCP 12s sitewhere-syncope-console ClusterIP 10.106.42.84 8080/TCP 11s sitewhere-syncope-enduser ClusterIP 10.96.198.155 8080/TCP 12s sitewhere-tenantsdb ClusterIP 10.100.180.120 5432/TCP 11s sitewhere-tenantsdb-headless ClusterIP None 5432/TCP 12s sitewhere-warp10 ClusterIP 10.102.109.2 8080/TCP,8081/TCP 13s sitewhere-warp10-headless ClusterIP None 8080/TCP,8081/TCP 12s sitewhere-web-rest-grpc ClusterIP 10.100.164.237 9001/TCP,9090/TCP 11s sitewhere-web-rest-http ClusterIP 10.105.154.48 8080/TCP 13s

==> v1/StatefulSet NAME READY AGE sitewhere-cassandra 0/3 16s sitewhere-cp-kafka 0/3 15s sitewhere-cp-zookeeper 0/3 14s sitewhere-influxdb 0/1 16s sitewhere-mongodb-arbiter 0/1 16s sitewhere-mongodb-primary 0/1 15s sitewhere-mongodb-secondary 0/1 15s sitewhere-postgresql 0/1 14s sitewhere-tenantsdb 0/1 15s sitewhere-warp10 0/1 15s

==> v1alpha3/Gateway NAME AGE sitewhere-gateway 16s

==> v1alpha3/VirtualService NAME GATEWAYS HOSTS AGE sitewhere-web-rest [sitewhere-gateway] [*] 16s

==> v1beta1/PodDisruptionBudget NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE sitewhere-cp-zookeeper-pdb N/A 1 0 11s sitewhere-mongodb-arbiter 1 N/A 0 11s sitewhere-mongodb-primary 1 N/A 0 11s sitewhere-mongodb-secondary 1 N/A 0 10s

NOTES:

  1. Get SiteWhere application URL by running these commands: export ISTIO_INGRESS_IP=$(kubectl get svc --namespace istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') To login to SiteWhere Admin Console use:
    • Username: admin
    • Password: password
    • protocol: http
    • Server: $ISTIO_INGRESS_IP
    • Port: 80

ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ helm ls --all sitewhere NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE sitewhere 1 Sun Aug 2 20:41:29 2020 DEPLOYED sitewhere-0.2.9 2.2 default
ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ ceyarkadmin@ceyark-macserv1:~/siteWhere/sitewhere-k8s/charts$ helm install --name sitewhere -f ./sitewhere/values-dev.yaml --set sitewhere-infra-database.mongodb.persistence.storageClass=hostpath --set sitewhere-infra-core.kafka.persistence.storageClass=hostpath --set sitewhere-infra-core.kafka.zookeeper.persistence.storageClass=hostpath sitewhere/sitewhere Error: a release named sitewhere already exists. Run: helm ls --all sitewhere; to check the status of the release Or run: helm del --purge sitewhere; to delete it

-------------------------------------end of logs---------------------- So i have a query. Why am I getting an error when I execute the command helm install --name sitewhere -f ./sitewhere/values-dev.yaml sitewhere

whereas I don't get error when I execute "helm install --name sitewhere sitewhere/sitewhere". Can you please clarify.

Is this sufficient to continue to work on this software package. But I would prefer to work on the developer instance running to save the resources on the machine. Can you please guide.

Thanks.

aravindj76 commented 3 years ago

The issue is solved. Thanks for the support.