search

cue-lang / cue

The home of the CUE language! Validate and define text-based and dynamic configuration
https://cuelang.org
Apache License 2.0
4.89k stars 279 forks source link

cmd/cue: yaml.Unmarshal reporting "non-concrete value" within cmd #2998

Open SomniVertix opened 3 months ago

SomniVertix commented 3 months ago

What version of CUE are you using (cue version)?

$ cue version
cue version v0.7.1

go version go1.21.6
      -buildmode exe
       -compiler gc
  DefaultGODEBUG panicnil=1
     CGO_ENABLED 1
          GOARCH arm64
            GOOS darwin

Does this issue reproduce with the latest stable release?

Yes - this was tested on v0.8.0 as well. However, in v0.8.0 it COMPLETELY breaks, not just in the ways that I mentioned before. This was also discussed on slack.

What did you do?

When running the following:

command: "gen-argocd-cluster-specs": {
    read: file.Read & {
        filename: "./dist/low-clusters.gen.yml"
        contents: string
    }

    v: yaml.Unmarshal(read.contents)
    v: [...#ArgoCDClusterSpecs]

    "basic-print": cli.Print & {
        text: v[0].metadata.name
    }
}

low-clusters.gen.yml type.cue

I am able to get the output: internal-shared-dev-east as expected. However, when running in the loop:

package kubernetesbase

import (
    yaml "encoding/yaml"
    "tool/file"
    "tool/cli"
)

command: "gen-argocd-cluster-specs": {
    read: file.Read & {
        filename: "./dist/low-clusters.gen.yml"
        contents: string
    }

    v: yaml.Unmarshal(read.contents)
    v: [...#ArgoCDClusterSpecs]

    for i, spec in v {
        "print-\(i)": cli.Print & {
            text: spec.metadata.name
        }
    }
}

I get the following error:

command."gen-argocd-cluster-specs".v: error in call to encoding/yaml.Unmarshal: non-concrete value string:
    ./gen_tool.cue:15:5
    ./gen_tool.cue:12:13

Note: When I try this on v0.8.0, it errors out with the same error even in the non-loop example. I tried on master as well, and that produced the same results at v0.7.1.

Reference slack discussion

myitcv commented 2 months ago

@SomniVertix - thanks for raising this. This is an issue related to CUE calculating the dependencies between parts of the configuration. In the first example it is able to detect a relationship between the basic-print and read tasks. In the second, it can't see the dependency because it is happening "at the same level" as the read field. The solution (until we come up with a better way of discovering such dependencies is to wrap the loop version in a field (name does not matter):

exec cue cmd gen-argocd-cluster-specs
stdout 'basic: internal-shared-dev-east'
stdout 'loop: internal-shared-qa-east'

-- x_tool.cue --
package kubernetesbase

import (
    "encoding/yaml"
    "tool/file"
    "tool/cli"
)

command: "gen-argocd-cluster-specs": {
    read: file.Read & {
        filename: "./dist/low-clusters.gen.yml"
        contents: string
    }

    // wrap the tasks in an outer field
    something: {
        v: yaml.Unmarshal(read.contents)

        "basic-print": cli.Print & {
            text: "basic: \(v[0].metadata.name)"
        }

        for i, spec in yaml.Unmarshal(read.contents) {
            "print-\(i)": cli.Print & {
                text: "loop: \(spec.metadata.name)"
            }
        }
    }

}
-- dist/low-clusters.gen.yml --
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: east
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-shared-dev-east
  stringData:
    config: |2-
      some multiline string here
    name: internal-shared-dev-east
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: west
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-shared-dev-west
  stringData:
    config: |2-
      some multiline string here
    name: internal-shared-dev-west
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: east
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-shared-qa-east
  stringData:
    config: |2-
      some multiline string here
    name: internal-shared-qa-east
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: west
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-shared-qa-west
  stringData:
    config: |2-
      some multiline string here
    name: internal-shared-qa-west
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: east
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-pega-dev-east
  stringData:
    config: |2-
      some multiline string here
    name: internal-pega-dev-east
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: west
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-pega-dev-west
  stringData:
    config: |2-
      some multiline string here
    name: internal-pega-dev-west
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: east
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-pega-qa-east
  stringData:
    config: |2-
      some multiline string here
    name: internal-pega-qa-east
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: west
      infrastructure_environment: low
      infrastructure_location: int
    name: internal-pega-qa-west
  stringData:
    config: |2-
      some multiline string here
    name: internal-pega-qa-west
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: east
      infrastructure_environment: low
      infrastructure_location: oci
    name: oci-low-ash-eck
  stringData:
    config: |2-
      some multiline string here
    name: oci-low-ash-eck
    server: some string here
  type: Opaque
- ApiVersion: v1
  kind: Secret
  metadata:
    labels:
      argocd.argoproj.io/secret-type: cluster
      region: east
      infrastructure_environment: low
      infrastructure_location: oci
    name: oci-low-ash-infra
  stringData:
    config: |2-
      some multiline string here
    name: oci-low-ash-infra
    server: some string here
  type: Opaque
myitcv commented 2 months ago

Re-opening, because as @mvdan points out I haven't linked to a specific issue whether the dependency "problem" outlined above is going to be fixed. We should have such an issue.