Error "split items cannot be empty" with helm processor and OpenShift GitOps

[joschi36]

When trying to deploy a Helm Application, I get the following error message. I think this error comes from helmProcessor.go but I do not know how I could debug this further.

Error

Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = Manifest generation error (cached): plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: `argocd-lovely-plugin` failed exit status 1: 2024/10/31 08:47:44 split items cannot be empty

Environment Information

We are running the plugin inside OpenShift with the OpenShift Gitops Operator, similar to the ArgoCD Operator but supported by RedHat.

Application (redacted)

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: demo
  namespace: demo
spec:
  destination:
    name: in-cluster
  project: default
  source:
    path: demo
    plugin:
      name: argocd-lovely-plugin
      parameters:
      - name: LOVELY_HELM_VALUES
        string: "{}"
    repoURL: https://github.com/xxx/yyy.git
    targetRevision: main
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - PruneLast=true

Installation (redacted)

apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
  name: xxxx
  namespace: xxxx
  annotations:
    argocd.argoproj.io/sync-options: Delete=false
spec:
  repo:
    serviceaccount: repo-sa
    mountsatoken: true
    sidecarContainers:
      - name: lovely-plugin
        image: xxxx # Custom Image using ubi9 as base
        envFrom:
          - secretRef:
              name: argocd-lovely-plugin-env
        securityContext:
          runAsNonRoot: true
        volumeMounts:
            # Import the repo-server's plugin binary
          - mountPath: /var/run/argocd
            name: var-files
          - mountPath: /home/argocd/cmp-server/plugins
            name: plugins
            # Starting with v2.4, do NOT mount the same tmp volume as the repo-server container. The filesystem separation helps
            # mitigate path traversal attacks.
          - mountPath: /tmp
            name: lovely-tmp
    volumes:
      - emptyDir: {}
        name: lovely-tmp

Custom Image Containerfile

Because of OpenShift special handling of users, we had to build our own Containerfile supporting it. However it includes all the binary from lovely-vault-plugin:1.1.1

FROM ghcr.io/crumbhole/lovely-vault-plugin:1.1.1 AS source

FROM registry.access.redhat.com/ubi9:9.4

# Switch to root for the ability to perform install
USER root

# Install tools needed for your repo-server to retrieve & decrypt secrets, render manifests
# (e.g. curl, awscli, gpg, sops)
RUN yum update -y
RUN yum install -y curl git --allowerasing
RUN yum clean all
RUN rm -rf /var/cache/yum/* /tmp/* /var/tmp/*

ENV LOVELY_HELM_PATH=/usr/local/bin/helm
ENV LOVELY_HELMFILE_PATH=/usr/local/bin/helmfile
ENV LOVELY_KUSTOMIZE_PATH=/usr/local/bin/kustomize
ENV LOVELY_PLUGINS=
ENV LOVELY_PREPROCESSORS=
ENV HOME=/tmp
ENV HELM_CONFIG_HOME=/tmp/.helm
ENV HELM_CACHE_HOME=/tmp/.helm
ENV HELM_DATA_HOME=/tmp/.helm
COPY --from=source /usr/local/bin/yq /usr/local/bin/yq
COPY --from=source /usr/local/bin/helm /usr/local/bin/helm
COPY --from=source /usr/local/bin/helmfile /usr/local/bin/helmfile
COPY --from=source /usr/local/bin/kustomize /usr/local/bin/kustomize
COPY --from=source /usr/local/bin/argocd-vault-plugin /usr/local/bin/argocd-vault-plugin
COPY --from=source /usr/local/bin/argocd-lovely-plugin /usr/local/bin/argocd-lovely-plugin
ADD *.pem /etc/pki/ca-trust/source/anchors/
RUN update-ca-trust

# Switch back to non-root user
USER 999
ENV LOVELY_PLUGINS="argocd-vault-plugin generate -"
RUN mkdir -p /tmp/.helm
COPY --from=source --chown=999 /home/argocd/cmp-server/config/plugin.yaml /home/argocd/cmp-server/config/plugin.yaml
COPY --from=source /entrypoint.sh /entrypoint.sh
# /var/run/argocd/argocd-cmp-server does NOT exist inside the image, must be mounted from argocd
ENTRYPOINT [ "/entrypoint.sh" ]

Installation Details

• OpenShift GitOps Operator
• ArgoCD Version: v2.12.6+4dab5bd
• Lovely Version: 1.1.1 (Vault Flavor)
• OpenShift Version: 4.16.14
• Kubernetes Version: v1.29.8+f10c92d

[Joibel]

I believe this error is coming from:

      parameters:
      - name: LOVELY_HELM_VALUES
        string: "{}"

I'm not sure why you get that error at the moment, but that definitely looks to be wrong. HELM_VALUES should be a list of one or more file names.

[joschi36]

Hi @Joibel

You, of course, are right, I totally missed the variable is to specify files. I was porting the original Helm integration, where you could specify a JSON string.

Thx for helping out. :)