talhelper parsing issue for extraManifests yaml starting with ---

[ajaykumar4]

NodeConfigs: extraManifests:

• tailscale.yaml

  tailscale.yaml

  ---
  apiVersion: v1alpha1
  kind: ExtensionServiceConfig
  name: tailscale
  environment:
  - TS_AUTHKEY="<tailscale_key>"

  error message : missing kind from yaml talhelper version: 1.7.4

[budimanjojo]

Thanks for the report @AjayKumar4!

I just tried this and it works fine here. I think the talconfig.yaml you provided is wrong. It should be like this:

nodes:
  - extraManifests:
      - tailscale.yaml

The nodeConfigs key is inline for nodes and controlPlane and worker struct. So you use it like this:

nodes:
  - <everythingInsideNodeConfigsStruct>:
worker:
  <everythingInsideNodeConfigsStruct>:
controlPlane:
  <everythingInsideNodeConfigsStruct>:

The only downside of having --- in the file is that the generated manifest will have double document delimiter, but it should work just fine as long as the provided file is a valid yaml formatted document.

[budimanjojo]

Wait, you're using talhelper v1.7.4 which is VERY outdated. Please update it first.

[ajaykumar4]

talhelper version 2.4.10

and this is syntax, i'm using

nodes:
  - extraManifests:
      - tailscale.yaml

error applying new configuration: rpc error: code = Unknown desc = failed to parse config: missing kind for command talhelper gencommand apply --config-file ~/Personal/home-lab/kubernetes/bootstrap/talos/talconfig.yaml --out-dir ~/Personal/home-lab/kubernetes/bootstrap/talos/clusterconfig --extra-flags="--insecure" | bash

generated file

 version: v1alpha1
debug: false
persist: true
machine:
  type: controlplane
  token: <token>
  ca:
    crt: <crt>
    key: <key>
  certSANs:
    - 192.168.1.250
    - 127.0.0.1
  kubelet:
    image: ghcr.io/siderolabs/kubelet:v1.30.1
    extraArgs:
      rotate-server-certificates: "true"
    extraMounts:
      - destination: /var/openebs/local
        type: bind
        source: /var/openebs/local
        options:
          - bind
          - rshared
          - rw
    defaultRuntimeSeccompProfileEnabled: true
    nodeIP:
      validSubnets:
        - 192.168.1.0/24
    disableManifestsDirectory: true
  network:
    hostname: jetson01
    interfaces:
      - deviceSelector:
          hardwareAddr: 48:b0:2d:3c:c0:ae
        addresses:
          - 192.168.1.7/24
        routes:
          - network: 0.0.0.0/0
            gateway: 192.168.1.254
        mtu: 1500
        dhcp: false
        vip:
          ip: 192.168.1.250
    disableSearchDomain: true
  install:
    disk: /dev/mmcblk1
    image: factory.talos.dev/installer/cc93087ce425dbefb9648afca095901bc6d080f20a631848432f99757000bc0d:v1.7.4
    wipe: false
  files:
    - content: |-
        [plugins."io.containerd.grpc.v1.cri"]
          enable_unprivileged_ports = true
          enable_unprivileged_icmp = true
        [plugins."io.containerd.grpc.v1.cri".containerd]
          discard_unpacked_layers = false
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          discard_unpacked_layers = false
      permissions: 0o0
      path: /etc/cri/conf.d/20-customization.part
      op: create
  sysctls:
    fs.inotify.max_queued_events: "65536"
    fs.inotify.max_user_instances: "8192"
    fs.inotify.max_user_watches: "524288"
    net.core.rmem_max: "2500000"
    net.core.wmem_max: "2500000"
  features:
    rbac: true
    stableHostname: true
    kubernetesTalosAPIAccess:
      enabled: true
      allowedRoles:
        - os:admin
      allowedKubernetesNamespaces:
        - system-upgrade
    apidCheckExtKeyUsage: true
    diskQuotaSupport: true
    kubePrism:
      enabled: true
      port: 7445
    hostDNS:
      enabled: true
      forwardKubeDNSToHost: false
      resolveMemberNames: true
cluster:
  id: JjM_GMcxukRZUc1IOrDbJSbaUG3GBBpIJ5kcMQlRn6o=
  secret: <secert>
  controlPlane:
    endpoint: https://192.168.1.250:6443
  clusterName: aj-home-lab
  network:
    cni:
      name: none
    dnsDomain: cluster.local
    podSubnets:
      - 10.69.0.0/16
    serviceSubnets:
      - 10.96.0.0/16
  token: <token>
  secretboxEncryptionSecret: <secretboxEncryptionSecret>
  ca:
    crt: <crt>
    key: <key>
  aggregatorCA:
    crt: <crt>
    key: <key>
  serviceAccount:
    key: <key>
  apiServer:
    image: registry.k8s.io/kube-apiserver:v1.30.1
    certSANs:
      - 192.168.1.250
      - 192.168.1.250
      - 127.0.0.1
    disablePodSecurityPolicy: true
    auditPolicy:
      apiVersion: audit.k8s.io/v1
      kind: Policy
      rules:
        - level: Metadata
  controllerManager:
    image: registry.k8s.io/kube-controller-manager:v1.30.1
    extraArgs:
      bind-address: 0.0.0.0
  proxy:
    disabled: true
    image: registry.k8s.io/kube-proxy:v1.30.1
  scheduler:
    image: registry.k8s.io/kube-scheduler:v1.30.1
    extraArgs:
      bind-address: 0.0.0.0
  discovery:
    enabled: true
    registries:
      kubernetes:
        disabled: false
      service:
        disabled: false
  etcd:
    ca:
      crt: <crt>
      key: <key>
    extraArgs:
      listen-metrics-urls: http://0.0.0.0:2381
    advertisedSubnets:
      - 192.168.1.0/24
  coreDNS:
    disabled: true
  allowSchedulingOnControlPlanes: true
---
---
apiVersion: v1alpha1
kind: ExtensionServiceConfig
name: tailscale
environment:
  - TS_AUTHKEY="<ts_key>"

[budimanjojo]

Ah, so the issue is talosctl doesn't like the double document delimiter. This should be easily fixed with removing the --- when it exists, but the code will be very ugly. I see what I can do.

[ajaykumar4]

yup, this function need some upgrade https://github.com/budimanjojo/talhelper/blob/32f7d1b6e241999e5adfb01457e5d3e1f1b3fb2b/pkg/generate/config.go#L164

[budimanjojo]

Created the PR for the fix, but currently I'm still working on fixing the go module issue so this will be released alongside v3.0.0. For now you can remove the --- in tailscale.yaml file for the workaround. Thank you for reporting this!

[ajaykumar4]

instead of trim you can do check in https://github.com/budimanjojo/talhelper/blob/32f7d1b6e241999e5adfb01457e5d3e1f1b3fb2b/pkg/talos/networkconfig.go#L80

// CombineYamlBytes prepends and returns `---\n` before `input`, unless the input already starts with `---\n`.
func CombineYamlBytes(input [][]byte) []byte {
    delimiter := []byte("---\n")
    var result []byte
    for k := range input {
        if !bytes.HasPrefix(input[k], delimiter) {
            result = append(result, delimiter...)
        }
        result = append(result, input[k]...)
    }
    return result
}

[budimanjojo]

Oh yeah good catch, putting it there makes more sense. Thank you! I'll update the commit