String interpolation consistency in AML

mamiu commented 2 years ago

The doc section about Template secrets gives the following example:

// ...
secrets: {
    "my-template": {
        type: "template" // required
        data: {
            template: """  // required
            a long \(localData.configValue.key)
            a ${secret://my-secret-data/key}
            """
        }
    }
    "my-secret-data": {
        type: "opaque"
        data: {
            key: "value"
        }
    }
}

So there are two ways of string interpolation in AML (if this will become the name of the acorn markup language):

\(some.referenced.key) (e.g. \(localData.configValue.key))
${custom-URI-scheme://object-name/attribute} (e.g. ${secret://my-secret-data/key})

But IMHO the markup language would benefit a lot from having just one way of doing string interpolations. Or at least offer to write all string interpolations in only one syntax, but still support both of the current syntaxes.

For example: Since I really like the idea of using custom URI schemes (similar to how it's done with deep linking on mobile devices) it would be great to support a few more schemes like ref or reference, var or variable, etc.

Then the example of the referenced key above would look like this: ${reference://localData.configValue.key}

Or the string interpolation in the two examples in the docs about Stateful applications and Templates could look like this: ${var://i} or ${variable://Name}

ibuildthecloud commented 2 years ago

Thanks for the feedback. There's an important technical difference between the two syntax. I 100% agree we should be consistent, but let me explain why they are currently different. The \(...) syntax is evaluated by the markup language at parsing time. The ${...} syntax is evaluated by the backend after the fact. The reason for this is that the AML parser does not have access to secrets, by design, it just has a "late binding" reference.

We can make the two syntax consistent but I am concerned that someone will get confused about the slight different in behavior. For example, the following string "password: \(std.base64(${secret://my-secret/data}) will never work because the parser evaluating the base64 function will not have the value of the secret to base64 encode.

ibuildthecloud commented 2 years ago

@vincent99 Ideas?

vincent99 commented 2 years ago

Maybe turn the info only available at runtime into an explicitly named made-up top-level variable? \(runtime.secret["my-secret"].data)

mamiu commented 2 years ago

Thanks for the explanation @ibuildthecloud.

Would it theoretically be possible to evaluate both string interpolations by the backend? Because not only would this enable a uniform syntax, but it would also make use cases like the one you mentioned possible.

Just in case it were possible, I think it made more sense not to use the custom URI scheme syntax for functions (e.g. like password: ${func://std.base64(secret://my-secret/data)}) but rather treat all non URI strings within string interpolations as functions calls (e.g. password: ${std.base64(secret://my-secret/data)}).

ibuildthecloud commented 2 years ago

There's a danger of people misusing secrets if we allow the parser to have the value. We also need to be able to render the acornfile without context to do things like validation. But maybe it's possible... I need to figure out what is the actual danger. Right now it's theoretical.

mamiu commented 2 years ago

I see what you're saying. Keen to hear about your assessment or decision once you've figured out the actual danger.

denist-huma commented 2 years ago

I add to this that doc is not clear about interpolation or substitution. https://docs.acorn.io/authoring/args-and-profiles#in-a-string-or-template You need to stress the \(some.referenced.key) is needed. I've read it the "()" are good way, so at first I tried this "image: ghcr.io/huma-engineering/huma-score-engine-server:(args.tag)" that is rendering the same "image: ghcr.io/huma-engineering/huma-score-engine-server:(args.tag)" string.

$ acorn render .
acorns: {}
args:
  dev: false
  tag: 97cb9bd
containers:
  server:
    entrypoint:
    - python
    - main.py
    env:
      HU_SE_LOGIN: milano
      HU_SE_PASSWORD: password
    files: {}
    image: ghcr.io/huma-engineering/huma-score-engine-server:(args.tag)
    permissions:
      clusterRules: []
      rules: []
    ports:
      expose: []
      internal: []
      publish: 8000/http
    sidecars: {}
images: {}
jobs: {}
profiles:
  dev:
    dev: true
secrets: {}
volumes: {}

denist-huma commented 2 years ago

@ibuildthecloud I prefer to use client side string interpolation, so I can use "acorn render". I is just as convinient as "helm template ." or "kubectl --dry-run"

The (...) syntax is evaluated by the markup language at parsing time.