search

carvel-dev / ytt

YAML templating tool that works on YAML structure instead of text
https://carvel.dev/ytt
Apache License 2.0
1.63k stars 137 forks source link

proposal: improve template readability #8

Closed lionelvillard closed 2 years ago

lionelvillard commented 5 years ago

Problem: it's quite cumbersome to prefix each template command by #@. Also the YAML inside the function body cannot (?) indented, which is inconsistent with programming best practices, at odd with YAML itself and starlark conventions.

Proposal: allow function definitions to be defined without the #@ prefix, as part of the template prologue, before the YAML. The starlark syntax should be extended to allow YAML values, eg:

def some_yaml():
  name: max
  cities:
  - SF
  - LA
end

I don't have the whole proposal fleshed out yet. I just putting this here to see if there is some interest in this strawman proposal. If yes I can expand and give more details.

nimakaviani commented 5 years ago

hey @lionelvillard and thanks for the feedback.

I am guessing you meant to have the function definition inside a yaml document (a .yml file) rather than a startlark library. Cause you can define a function as a starlark library and include it in your template similar to the helper functions in this playground example.

As for the yaml document, one of the design goals in ytt has been to keep the templated yaml document syntactically valid. I am afraid having a function definition in the yaml document would go against it, rendering the yaml document invalid.

As for the yaml snippet inside a function, same syntactic validity requirement applies. The reason you cannot indent the yaml document inside the function is that there is no immediate parent node for the block inside and for the code annotation above it to be associated to, which again renders the inner yaml document invalid.

It would be good if you can elaborate on your ideas a bit more. thanks!

cppforlife commented 5 years ago

I just putting this here to see if there is some interest in this strawman proposal.

i'm interested in more more details as well.

Also the YAML inside the function body cannot (?) indented

you can do so by starting a new document in which you can change indentation level.

---
#@ def some_yaml():
  name: max
  cities:
  - SF
  - LA
#@ end

---
yaml: #@ some_yaml()

The starlark syntax should be extended to allow YAML values

i've previously toyed with this idea but found it somewhat hard to keep syntax lightweight. would love to see what you think.

i've even at some point tried out using python style comments to keep yaml content in starlark. i have it on my todo list to clean that up. eg

def foo():
  val = 123
  """yaml
  foo: #@ val
  """
end
lionelvillard commented 5 years ago

thanks for the comments! I'll work on a quick PoC and get back to you when I have something to show.

lionelvillard commented 5 years ago

The PoC is there: https://github.com/lionelvillard/starlark-go/tree/yaml

I only added support for YAML mappings.

What's missing:

cppforlife commented 5 years ago

ive been fighting flu here, but ive been chewing on the idea. ill try to write something up this week or early next week.

Sent from my iPhone

On Apr 1, 2019, at 7:03 AM, Lionel Villard notifications@github.com wrote:

The PoC is there: https://github.com/lionelvillard/starlark-go/tree/yaml

I only added support for YAML mappings.

What's missing:

YAML sequences: this can easily be done by changing the expression statements semantic when the expression starts with -. This is ok since this kind of expressions are useless. escaping YAML to starlark: this one is easy. I would suggest to use { ... } to escape to starlark. support for the . notation to access YAML properties. How do you do this in ytt? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

nimakaviani commented 5 years ago

sorry for the delay as I have been traveling for the Cloud Foundry summit.

I had a look at the implementation and I think it is a smart hack to incorporate yaml into the syntax of Starlark.

IMHO, the primary reason for templating a declarative language with conditionals and loops is the lack of such language constructs in the configuration language itself. In an imperative language like Starlark however, these language constructs for maps, arrays, objects, etc already exist. So in this case it is mostly the syntactic sugar that we are adding.

Doing so raises the following questions / concerns for me:

lionelvillard commented 5 years ago

Hi @nimakaviani , here are answers addressing some of your concerns.

  • we manage to get rid of #@ from the beginning of a function definition but still require to fall back to the ytt annotation inside the yaml document in a Starlark file when referring to data values. so how much gain do we really have?

In this proposal the ytt annotation is replaced by a construct to escape back to starlark. I propose using { ... } since this is common. One (minor) advantage over #@ is { .. } can be interleaved with YAML (eg: My { ... } list of { ... } expressions)

Also I find is strange I can't use just # for writing a comment. I need to use #!. That's not pure YAML and not pure Python.

  • are we making it more confusing for people by not having a .star file to be a fully pythonic language anymore?

No sure I understand this concern. Do you mean a purely pythonic language? Having one language instead of two is usually better. No need to explain to user what is .star and what is .yml with extensions. IMHO, this is about consistency and consumability.

  • are we making it prone to more ambiguity by making structs sensitive to indentation where naturally they don't need to be?

As a pythonic language user I expect whitespaces to be significant so that shouldn't be a problem.

I didn't realize you have extended starlark with a new struct datatype. Interesting. Not sure how it compare with just using dictionary. That's a starlark go extension.

  • by allowing the Starlark code to also provide configuration declarations, are we making it harder for people to figure out where the declarations come from. Ideally I would like for a Starlark file in ytt to only hold logic and not any declaration of the configuration structure.

Can this be achieved by prescribing how to structure files? Modules in __yy_libs, maybe logic in a ./helpers directory?

lionelvillard commented 5 years ago

FYI, I added support for YAML sequence and dictionary dot expression: https://github.com/lionelvillard/starlark-go/tree/yaml

cppforlife commented 5 years ago

@lionelvillard what are you thoughts about different yaml ways of interpreting values of different types (ie integer vs string vs float vs something else). im personally not a big fan.

i can imagine having a more constrained whitespace enabled syntax that would lean on starlarks interpretation of strings, int, etc. in that case we are mostly talking about supporting more compact maps and arrays format (in addition to regular { ...: ... } and [ ... , ... ]). though i wonder if there are some ambiguities.

there is also a question of yaml documents.

lionelvillard commented 5 years ago

@cppforlife the only reason for having (s|i|b|f){ ... } is to disambiguate with { ... } (mapping flow style). We don't need to carry any type information, and I agree with you, we should not. We just needs{ ... } (s = starlark}. For the record, the idea of using a prefix is coming from Python string literal.

there is also a question of yaml documents.

This is not an issue for a lexical/grammar pov.

pivotaljohn commented 2 years ago

This discussion is more than two years stale. Closing the issue doesn't delete it; if the interest resparks, we can re-open. Closing, in the meantime.