search

labring / sealos

Sealos is a production-ready Kubernetes distribution. You can run any Docker image on sealos, start high availability databases like mysql/pgsql/redis/mongo, develop applications using any Programming language.
https://cloud.sealos.io
Apache License 2.0
14.09k stars 2.08k forks source link

BUG: sealos run with `error converting YAML to JSON` #1454

Closed willzhang closed 2 years ago

willzhang commented 2 years ago

Which command or component

 sealos apply -f Clusterfile

Clusterfile

apiVersion: apps.sealos.io/v1beta1
kind: Cluster
metadata:
  creationTimestamp: null
  name: default
spec:
  hosts:
  - ips:
    - 192.168.72.50:22
    roles:
    - master
    - amd64
  - ips:
    - 192.168.72.51:22
    - 192.168.72.52:22
    roles:
    - node
    - amd64
  image:
  - labring/kubernetes:v1.24.3
  - labring/helm:v3.8.2
  - registry.cn-shenzhen.aliyuncs.com/cnmirror/cilium:v1.12.0
  ssh:
    passwd: "123456"
    pk: /root/.ssh/id_rsa
    port: 22
    user: root
status: {}
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
skipPhases:
- addon/kube-proxy

The Description of the bug

Describe the bug.

How to reproduce（pictures can be attached if necessary）

#  sealos apply -f Clusterfile
......................
2022-08-02 19:16:49 [INFO] Executing pipeline RunGuest in CreateProcessor.
2022-08-02 19:16:49 [INFO] guest cmd is cp opt/helm /usr/bin/
2022-08-02 19:16:50 [INFO] guest cmd is cp opt/* /usr/bin/
2022-08-02 19:16:57 [INFO] guest cmd is helm install cilium charts/cilium --namespace=kube-system --set kubeProxyReplacement=strict --set k8sServiceHost=apiserver.cluster.local --set k8sServicePort=6443
Error: INSTALLATION FAILED: cannot load values.yaml: error converting YAML to JSON: yaml: line 2119: could not find expected ':'
2022-08-02 19:16:57 [EROR] Applied to cluster error: exit status 1
2022-08-02 19:16:58 [INFO] 
      ___           ___           ___           ___       ___           ___
     /\  \         /\  \         /\  \         /\__\     /\  \         /\  \
    /::\  \       /::\  \       /::\  \       /:/  /    /::\  \       /::\  \
   /:/\ \  \     /:/\:\  \     /:/\:\  \     /:/  /    /:/\:\  \     /:/\ \  \
  _\:\~\ \  \   /::\~\:\  \   /::\~\:\  \   /:/  /    /:/  \:\  \   _\:\~\ \  \
 /\ \:\ \ \__\ /:/\:\ \:\__\ /:/\:\ \:\__\ /:/__/    /:/__/ \:\__\ /\ \:\ \ \__\
 \:\ \:\ \/__/ \:\~\:\ \/__/ \/__\:\/:/  / \:\  \    \:\  \ /:/  / \:\ \:\ \/__/
  \:\ \:\__\    \:\ \:\__\        \::/  /   \:\  \    \:\  /:/  /   \:\ \:\__\
   \:\/:/  /     \:\ \/__/        /:/  /     \:\  \    \:\/:/  /     \:\/:/  /
    \::/  /       \:\__\         /:/  /       \:\__\    \::/  /       \::/  /
     \/__/         \/__/         \/__/         \/__/     \/__/         \/__/

                  Website :https://www.sealos.io/
                  Address :github.com/labring/sealos

why the default values.yaml changed

root@node1:~# cat /var/lib/sealos/data/default/rootfs/charts/cilium/values.yaml
..........................
dnsProxy:
  # -- DNS response code for rejecting DNS requests, available options are '[nameError refused]'.
  dnsRejectResponseCode: refused
  # -- Allow the DNS proxy to compress responses to endpoints that are larger than 512 Bytes or the EDNS0 option, if present.
  enableDnsCompression: true
  # -- Maximum number of IPs to maintain per FQDN name for each endpoint.
  endpointMaxIpPerHostname: 50
  # -- Time during which idle but previously active connections with expired DNS lookups are still considered alive.
  idleConnectionGracePeriod: 0s
  # -- Maximum number of IPs to retain for expired DNS lookups with still-active connections.
  maxDeferredConnectionDeletes: 10000
  # -- The minimum time, in seconds, to use DNS data for toFQDNs policies.
  minTtl: 3600
  # -- DNS cache data at this path is preloaded on agent startup.
  preCache: ""
  # -- Global port on which the in-agent DNS proxy should listen. Default 0 is a OS-assigned port.
  proxyPort: 0
  # -- The maximum time the DNS proxy holds an allowed DNS response before sending it along. Responses are sent as soon as the datapath is updated with the new IP information.
  proxyResponseMaxDelay: 100ms
nections with expired DNS lookups are still considered alive.
  idleConnectionGracePeriod: 0s
  # -- Maximum number of IPs to retain for expired DNS lookups with still-active connections.
  maxDeferredConnectionDeletes: 10000
  # -- The minimum time, in seconds, to use DNS data for toFQDNs policies.
  minTtl: 3600
  # -- DNS cache data at this path is preloaded on agent startup.
  preCache: ""
  # -- Global port on which the in-agent DNS proxy should listen. Default 0 is a OS-assigned port.
  proxyPort: 0
  # -- The maximum time the DNS proxy holds an allowed DNS response before sending it along. Responses are sent as soon as the datapath is updated with the new IP information.
  proxyResponseMaxDelay: 100ms

at the end Added multiple lines of content

nections with expired DNS lookups are still considered alive.
  idleConnectionGracePeriod: 0s
  # -- Maximum number of IPs to retain for expired DNS lookups with still-active connections.
  maxDeferredConnectionDeletes: 10000
  # -- The minimum time, in seconds, to use DNS data for toFQDNs policies.
  minTtl: 3600
  # -- DNS cache data at this path is preloaded on agent startup.
  preCache: ""
  # -- Global port on which the in-agent DNS proxy should listen. Default 0 is a OS-assigned port.
  proxyPort: 0
  # -- The maximum time the DNS proxy holds an allowed DNS response before sending it along. Responses are sent as soon as the datapath is updated with the new IP information.
  proxyResponseMaxDelay: 100ms

and the origin just have this

dnsProxy:
  # -- DNS response code for rejecting DNS requests, available options are '[nameError refused]'.
  dnsRejectResponseCode: refused
  # -- Allow the DNS proxy to compress responses to endpoints that are larger than 512 Bytes or the EDNS0 option, if present.
  enableDnsCompression: true
  # -- Maximum number of IPs to maintain per FQDN name for each endpoint.
  endpointMaxIpPerHostname: 50
  # -- Time during which idle but previously active connections with expired DNS lookups are still considered alive.
  idleConnectionGracePeriod: 0s
  # -- Maximum number of IPs to retain for expired DNS lookups with still-active connections.
  maxDeferredConnectionDeletes: 10000
  # -- The minimum time, in seconds, to use DNS data for toFQDNs policies.
  minTtl: 3600
  # -- DNS cache data at this path is preloaded on agent startup.
  preCache: ""
  # -- Global port on which the in-agent DNS proxy should listen. Default 0 is a OS-assigned port.
  proxyPort: 0
  # -- The maximum time the DNS proxy holds an allowed DNS response before sending it along. Responses are sent as soon as the datapath is updated with the new IP information.
  proxyResponseMaxDelay: 100ms

Test Result

What you expected to happen

Expected to xxx

Operating environment

cuisongliu commented 2 years ago

rename values.yaml.tmpl to values.yaml.tmpl.bak . Because the .tmpl suffix file will be rendered by default.

fanux commented 2 years ago

So this issue is already fixed?

@willzhang seems we can add cilium on our homepage

willzhang commented 2 years ago

So this issue is already fixed?

@willzhang seems we can add cilium on our homepage

yes, there have a values.yaml.tmpl

root@ubuntu:/data/git/cluster-image/applications/cilium/v1.12.0# ll charts/cilium/
total 244
drwxr-xr-x  4 root root   176 Aug  3 13:20 ./
drwxr-xr-x  3 root root    20 Aug  3 13:19 ../
-rw-r--r--  1 root root  5201 Aug  3 13:19 Chart.yaml
drwxr-xr-x  3 root root    22 Aug  3 13:19 files/
-rw-r--r--  1 root root   349 Aug  3 13:19 .helmignore
-rw-r--r--  1 root root 11350 Aug  3 13:19 LICENSE
-rw-r--r--  1 root root 65663 Aug  3 13:19 README.md
-rw-r--r--  1 root root  2825 Aug  3 13:19 README.md.gotmpl
drwxr-xr-x 12 root root  4096 Aug  3 13:19 templates/
-rw-r--r--  1 root root 72990 Aug  3 13:19 values.yaml
-rw-r--r--  1 root root 72272 Aug  3 13:19 values.yaml.tmpl

render logs


root@node1:~# sealos run -f registry.cn-shenzhen.aliyuncs.com/cnmirror/cilium:v1.12.0 --debug
2022-08-02 18:57:52 [INFO] start to install app in this cluster
2022-08-02 18:57:52 [DEBG] cmd for sh in host:  type buildah
2022-08-02 18:57:52 [DEBG] cmd for sh in host:  type buildah
2022-08-02 18:57:52 [DEBG] cmd for sh in host:  type buildah
2022-08-02 18:57:52 [DEBG] cmd for sh in host:  type buildah
are you sure to override these app? Yes [y/yes], No [n/no] : y
2022-08-02 18:57:53 [DEBG] sync cluster status is: apiVersion: apps.sealos.io/v1beta1
kind: Cluster
metadata:
  creationTimestamp: "2022-08-02T10:43:42Z"
  name: default
spec:
  hosts:
  - ips:
    - 192.168.72.50:22
    roles:
    - master
    - amd64
  - ips:
    - 192.168.72.51:22
    - 192.168.72.52:22
    roles:
    - node
    - amd64
  image:
  - labring/kubernetes:v1.24.3
  - labring/helm:v3.8.2
  - registry.cn-shenzhen.aliyuncs.com/cnmirror/cilium:v1.12.0
  ssh:
    passwd: "123456"
    pk: /root/.ssh/id_rsa
    port: 22
    user: root
status:
  conditions:
  - lastHeartbeatTime: "2022-08-02T10:53:35Z"
    message: exit status 1
    reason: ApplyClusterError
    status: "False"
    type: ApplyClusterSuccess
  mounts:
  - env:
      criData: /var/lib/containerd
      registryConfig: /etc/registry
      registryData: /var/lib/registry
      registryDomain: sealos.hub
      registryPassword: passw0rd
      registryPort: "5000"
      registryUsername: admin
    imageName: labring/kubernetes:v1.24.3
    labels:
      auth: auth.sh
      check: check.sh
      clean: clean.sh $criData
      clean-registry: clean-registry.sh $registryData $registryConfig
      image: ghcr.io/labring/lvscare:v4.1.0-rc1
      init: init.sh $criData $registryDomain $registryPort $registryUsername $registryPassword
      init-registry: init-registry.sh $registryPort $registryData $registryConfig
      io.buildah.version: 1.25.0-dev
      sealos.io.type: rootfs
      version: v1.24.3
    mountPoint: /var/lib/containers/storage/overlay/b5410fdf9b143fab2847fa391cd3b35aa62e443895aff418f1e0d76b8278695f/merged
    name: default-2etgjy1t
    type: rootfs
  - cmd:
    - cp opt/helm /usr/bin/
    imageName: labring/helm:v3.8.2
    labels:
      io.buildah.version: 1.25.0-dev
    mountPoint: /var/lib/containers/storage/overlay/ef045f9434d903313388d2219c9bdf17c98505d49b315da29b433ae57a749d76/merged
    name: default-azakzg4f
    type: application
  - cmd:
    - cp opt/* /usr/bin/
    - helm install cilium charts/cilium --namespace=kube-system --set kubeProxyReplacement=strict
      --set k8sServiceHost=apiserver.cluster.local --set k8sServicePort=6443
    imageName: registry.cn-shenzhen.aliyuncs.com/cnmirror/cilium:v1.12.0
    labels:
      io.buildah.version: 1.25.0-dev
    mountPoint: /var/lib/containers/storage/overlay/3a38817cdb697e2f3af5a1e58c6b03870fd621a03c87991b1ce40fdde4a548a6/merged
    name: default-j0n5ou7i
    type: application
  phase: ClusterFailed

21e7c5b3cedbed810b1f2f0c3d6725ca03ccbd02f0b30eae7bcea075b8a70379
2022-08-02 18:57:54 [DEBG] current container names is: default-j0n5ou7i
default-j0n5ou7i
2022-08-02 18:57:54 [DEBG] clusterfile config is empty!
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/etc
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/charts
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/manifests
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/etc
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/charts
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/manifests
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/etc
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/charts
2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/manifests
2022-08-02 18:57:54 [DEBG] cmd for bash in host:  cd /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged && chmod -R 0755 *
2022-08-02 18:57:54 [DEBG] send  app images ,ip: 192.168.72.50 , init flag: false, app flag: true,image name: registry.cn-shenzhen.aliyuncs.com/cnmirror/cilium:v1.12.0, image type: application
2022-08-02 18:57:54 [DEBG] local copy files src /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged to dst /var/lib/sealos/data/default/rootfs
2022-08-02 18:57:55 [INFO] guest cmd is cp opt/* /usr/bin/
2022-08-02 18:57:55 [DEBG] cmd for pipe in host:  bash -c cd /var/lib/sealos/data/default/rootfs && cp opt/* /usr/bin/
2022-08-02 18:57:56 [INFO] guest cmd is helm install cilium charts/cilium --namespace=kube-system --set kubeProxyReplacement=strict --set k8sServiceHost=apiserver.cluster.local --set k8sServicePort=6443
2022-08-02 18:57:56 [DEBG] cmd for pipe in host:  bash -c cd /var/lib/sealos/data/default/rootfs && helm install cilium charts/cilium --namespace=kube-system --set kubeProxyReplacement=strict --set k8sServiceHost=apiserver.cluster.local --set k8sServicePort=6443
Error: INSTALLATION FAILED: cannot load values.yaml: error converting YAML to JSON: yaml: line 2119: could not find expected ':'
2022-08-02 18:57:56 [EROR] Applied to cluster error: exit status 1
2022-08-02 18:57:56 [DEBG] write cluster file to local storage: /root/.sealos/default/Clusterfile
2022-08-02 18:57:56 [INFO] 
      ___           ___           ___           ___       ___           ___
     /\  \         /\  \         /\  \         /\__\     /\  \         /\  \
    /::\  \       /::\  \       /::\  \       /:/  /    /::\  \       /::\  \
   /:/\ \  \     /:/\:\  \     /:/\:\  \     /:/  /    /:/\:\  \     /:/\ \  \
  _\:\~\ \  \   /::\~\:\  \   /::\~\:\  \   /:/  /    /:/  \:\  \   _\:\~\ \  \
 /\ \:\ \ \__\ /:/\:\ \:\__\ /:/\:\ \:\__\ /:/__/    /:/__/ \:\__\ /\ \:\ \ \__\
 \:\ \:\ \/__/ \:\~\:\ \/__/ \/__\:\/:/  / \:\  \    \:\  \ /:/  / \:\ \:\ \/__/
  \:\ \:\__\    \:\ \:\__\        \::/  /   \:\  \    \:\  /:/  /   \:\ \:\__\
   \:\/:/  /     \:\ \/__/        /:/  /     \:\  \    \:\/:/  /     \:\/:/  /
    \::/  /       \:\__\         /:/  /       \:\__\    \::/  /       \::/  /
     \/__/         \/__/         \/__/         \/__/     \/__/         \/__/

                  Website :https://www.sealos.io/
                  Address :github.com/labring/sealos

reason

2022-08-02 18:57:54 [DEBG] render env dir: /var/lib/containers/storage/overlay/42a607e6035faa68ddb6e9521ebfee64b111fdb4fd4badcb4d4407198e48a189/merged/charts