Declarative DSL

[nwmcsween]

Consider a DSL that is similar to make with explicitly set defaults to map to fqdn's (via import/whatever). This way targets could be handled by an external program of some sort that 'schedules' hosts (e.g host with least latency to $someplace start http server).

[josephholsten]

I've been thinking about cfg mgmt in golang, and this DSL is where puppet & chef really shine.

One of their most useful accidents (not present in ansible, terraform) is the ability to create resource-providers entirely in the DSL out of other resources. I'm of the opinion that this lack is causing a lot of reuse pain in the Ansible Galaxy.

I'd love to help spike a plugin/DSL syntax using lua like the https://github.com/mozilla-services/heka project is doing.

[purpleidea]

On Mon, Feb 1, 2016 at 1:49 AM, Joseph Anthony Pasquale Holsten < notifications@github.com> wrote:

I've been thinking about cfg mgmt in golang, and this DSL is where puppet & chef really shine.

One of their most useful accidents (not present in ansible, terraform) is the ability to create resource-providers entirely in the DSL out of other resources. I'm of the opinion that this lack is causing a lot of reuse pain in the Ansible Galaxy.

Can you be a bit more specific about what you mean here? Are you talking about create_resources() ?

I'd love to help spike a plugin/DSL syntax using lua like the https://github.com/mozilla-services/heka project is doing.

Great, I'd love to have you involved. I've actually just started looking at lua as a possibility because of it's ease of embed-ability. Nothing certain yet, but a few links that were discussed the other day include:

https://github.com/Shopify/go-lua https://github.com/yuin/gopher-lua

and examples: https://github.com/kabukky/journey https://github.com/xyproto/algernon

More discussion to come!

[nwmcsween]

Puppet and chef both have bad DSL's in my opinion. A less constricting way to do this is write a DSL that allows you to shell out (such as make) and just focuses on the 'resolving' graphs, this has many benefits.

[purpleidea]

On Mon, Feb 1, 2016 at 3:17 AM, Nathan notifications@github.com wrote:

Puppet and chef both have bad DSL's in my opinion. A less constricting way to do this is write a DSL that allows you to shell out (such as make) and just focuses on the 'resolving' graphs, this has many benefits.

Could you elaborate on this comment please? I really agree about the DSL part, which I do think needs improving, but your comments about "resolving graphs" isn't clear.

Thanks!

[dalen]

One of their most useful accidents (not present in ansible, terraform) is the ability to create resource-providers entirely in the DSL out of other resources. I'm of the opinion that this lack is causing a lot of reuse pain in the Ansible Galaxy.

Can you be a bit more specific about what you mean here? Are you talking about create_resources() ?

I think this was in reference to the define keyword in Puppet which lets you create a new resource type without actually writing a type & provider.

[purpleidea]

On Mon, Feb 1, 2016 at 8:47 AM, Erik Dalén notifications@github.com wrote:

I think this was in reference to the define keyword in Puppet which lets you create a new resource type without actually writing a type & provider.

If that is the correct interpretation, then it is absolutely a design requirement for the DSL. The idea of composed "resources" is fundamental, and while the user should be able to do it in modules, the only thing I haven't completely figured out is how much of the useful secondary resources and stdlib I will write in native DSL vs. golang. Eg, if I have a "hosts" (/etc/hosts) resource, should it be native golang or a combination of the inotify file type and a template, etc... That conversation will have to be postponed till we have a DSL.

[purpleidea]

Also, I realized that I have mentioned this in discussion, but maybe it's not clear in the blog post, but the current "yaml" definition files are merely placeholders for testing the engine before there is a proper DSL. It is absolutely temporary, and I can't wait to remove all of that parsing code.

[josephholsten]

the define keyword in Puppet which lets you create a new resource type without actually writing a type & provider.

Yep, this.

I can't wait to remove all of that parsing code.

<3

[josephholsten]

TL;DR: external process resource-providers require a trade-off between implementation complexity and smart dependency things

@nwmcsween I'd love to do everything by having external tools make the changes. But most unix tools aren't designed to act idempotently. More so, a single tool often doen't do everything required. For example, a mere mkdir or cat will not create files with a specified owner, group or mode. So mere make-style file existence isn't enough to actually verify a file resource has been created correctly. At which point, our dependency resolver needs to know how to check if a thing is in the desired state. Which brings us back to needing resource implementation code.

Ansible's external-plugin model gives the separation between the execution-coordinator and the resource-provider implementation. But it runs each resource a single time, which internally does a separate check-phase and apply-phase. Compare this to puppet, &c which do a complete check-phase in all resources before doing an apply-phase. I don't have a concrete example right now, but this causes the dependency system in ansible to be very weak.

Terraform's external plugin model uses full-duplex communication between the coordinator and the resource provider. This allows a complete check-phase across all resources separate from the apply-phase. It also means the provider doesn't need to be run a separate check-process from the apply-process, even lets you have a single process for all resources using the same provider. The pain here is that you don't get to write plugins as standard unix tools naively reading from ARGV & STDIN, writing to STDOUT.

And I don't even want to think about parallel execution implications for resource providers. But that sounds especially unpleasant with ansible-style plugins.

[nwmcsween]

This is exactly the same as make, the onus is on the developer to make sure there are no hidden dependencies

On Tue, Feb 2, 2016, 12:08 PM Joseph Anthony Pasquale Holsten < notifications@github.com> wrote:

TL;DR: external process resource-providers require a trade-off between implementation complexity and smart dependency things

@nwmcsween https://github.com/nwmcsween I'd love to do everything by having external tools make the changes. But most unix tools aren't designed to act idempotently. More so, a single tool often doen't do everything required. For example, a mere mkdir or cat will not create files with a specified owner, group or mode. So mere make-style file existence isn't enough to actually verify a file resource has been created correctly. At which point, our dependency resolver needs to know how to check if a thing is in the desired state. Which brings us back to needing resource implementation code.

Ansible's external-plugin model gives the separation between the execution-coordinator and the resource-provider implementation. But it runs each resource a single time, which internally does a separate check-phase and apply-phase. Compare this to puppet, &c which do a complete check-phase in all resources before doing an apply-phase. I don't have a concrete example right now, but this causes the dependency system in ansible to be very weak.

Terraform's external plugin model uses full-duplex communication between the coordinator and the resource provider. This allows a complete check-phase across all resources separate from the apply-phase. It also means the provider doesn't need to be run a separate check-process from the apply-process, even lets you have a single process for all resources using the same provider. The pain here is that you don't get to write plugins as standard unix tools naively reading from ARGV & STDIN, writing to STDOUT.

And I don't even want to think about parallel execution implications for resource providers. But that sounds especially unpleasant with ansible-style plugins.

— Reply to this email directly or view it on GitHub https://github.com/purpleidea/mgmt/issues/2#issuecomment-178792819.

[cavaliercoder]

I'm a long time user of Puppet DSL and I've come to really hate the constraints it imposes. It is improving, but everything has to be reengineered (we finally have 'for' loops!). I prefer the idea of leveraging an existing language and providing an interface for declaring resources. Basically, like Chef does with Ruby.

Consider the proof that most Puppet modules implement Ruby code to augment the shortcomings of the DSL anyway.

There's so much to start building in terms of language internals, but also writing a parser, testing framework, etc. Would it be worth it?

I'd like to see a design where the user is writing Go/other code to declare resources, dependencies, etc. while taking advantage of Go's conditionals, operators, testing framework, quality tools, etc. But using an API interface which eases/enforces idempotency, extensibility, etc. Admittedly it creates opportunity for users to start breaking things with an unconstrained API... You'd also have to include the Go compiler in your toolset. Go is also not the most declarative DSL like language...

Method chaining/LINQy stuff would be ideal IMHO. Ignoring broken syntax, something like:

AddResource(YumRepo{
  Name: "centos-base",
  URL: "http://...",
})

AddResource(Package{
  Name: "vim-enhanced",
  Version: ">=7.3",
}).DependsOn(Resources.Where({
  Type: "YumRepo",
}))

That's my thoughts though :)

[purpleidea]

@cavaliercoder Good comments, here are some thoughts to help continue the discussion...

1) No matter what decision we end up with, someone will be unhappy. Since this is the case, I've instead tried to focus on what the correct thing is, and go from there.

2) There is a chance, that there will eventually be some sort of golang and/or c compatible API that you'll be able to use to build the graphs through the API however you want. This might facilitate doing the sort of thing you mentioned above. Doing this in golang will probably be easiest, since the API will be written in golang. There will be a discussion about if this will be something that just splits the community and/or if it's something we want to encourage or not.

3) Being declarative is very important, in particular because we want to enable the possibility of being able to write analysis tools that would integrate and figure out or detect certain conditions in the multi machine space. Eg: warning, this might cause an infinite loop across N hosts sort of thing. Do you think this sort of analysis would be possible if we used a functional, non-declarative language?

4) Lastly, if you do want to reuse an existing language (and I agree, I would love to do this instead of having to create one) then we need a suitable candidate! I've casually added Lua to the NACK list and obviously the imperative languages like python/ruby/etc are on there too. I've been trying to learn more about functional reactive programming. I don't suppose anyone knows of a declarative functional programming language or wants to build one. In any case, do you have some suggestions of what language we could use?

Cheers!

[vincentbernat]

LISP could be a candidate. It's easy to implement, it's easy to parse, it's supported out of the box by any editor. It's easy to make it look like a custom DSL while still being extensible. It's easy to keep it functional. The only downside is that some people don't like parentheses position.

For example, if I take on of your example:

(resources
  (exec
    ((name "exec1")
     (cmd "echo hello from exec1")
     (timeout 0)
     (watchcmd "sleep 10s")
     (state 'present))
    ((name "exec2")
     (cmd "echo hello from exec2")
     (watchcmd "sleep 10s")
     (state 'present))))
(edges
  ((name "e1")
   from (exec "exec1")
   to (exec "exec2")))

Some parentheses could be removed for example:

(exec (("exec1"
            timeout 0
            state 'present)))

So, it looks like the YAML. No need to learn anything. If you want to use a loop, it's easy (but you have to learn the backquote):

(resources
 (exec
   `(loop for i upto 10 collect
     (list (concat "exec" (str i))
            cmd (concat "echo hello " (str i))
            timeout 0
            state 'present))))

If you a user want to introduce a new type of resource, they can write a function. If they want to extend the DSL, they can write a macro.

[purpleidea]

@vincentbernat Thanks for your input!

One probable requirement of the DSL will include the ability to make it reactive, and also to have the compiler/lexer written in golang. Do you know if either of these already exist in lisp?

Having said that, I did consider lisp, but I'm not sure if it would be a good choice for mainstream users. Readability is an important property. Having said that, there's nothing to stop someone from writing a lisp frontend.

[vincentbernat]

For a more readable variant, you can look at Clojure syntax which have lists [] and maps {}. Not using parentheses everywhere make it more readable to some users. Note that the main advantage of LISP is that you get both the programming language stuff and the DSL feel. It's difficult to have that in non-homocoinicitic languages.

What you don't want is to transform YAML into a programming language (lke Ansible).

For Go, unfortunately, I know there are many LISP implemented in Go but I have no real experience to point to a good one. What's great with LISP is that you can take a minimal implementation and works from here with macros.

[purpleidea]

On Thu, Jul 7, 2016 at 6:05 AM, Vincent Bernat notifications@github.com wrote:

For a more readable variant, you can look at Clojure syntax which have lists [] and maps {}. Not using parentheses everywhere make it more readable to some users. Note that the main advantage of LISP is that you get both the programming language stuff and the DSL feel. It's difficult to have that in non-homocoinicitic languages.

Same uncertainty on how reactive aspects would fit in.

What you don't want is to transform YAML into a programming language (lke Ansible).

Agreed!

For Go, unfortunately, I know there are many LISP implemented in Go but I have no real experience to point to a good one. What's great with LISP is that you can take a minimal implementation and works from here with macros.

Thanks again for the idea! If you'd like to get more involved with patch writing but need some help getting going and learning about some internals, LMK :)

[vincentbernat]

Unfortunately, I don't have time at the moment. Writing random comments in GitHub issue is easier than proposing patches. :)

BTW, as an example of LISP as DSL, there is Riemann (http://riemann.io/, but it's written in Clojure, so, more natural choice).

[nwmcsween]

A full blown language would kill the idea of build system like DAG, it would also bloat and complicate it immensely. My opinion is if you want extras you simply shell out.

On Thu, Jul 7, 2016, 5:21 AM Vincent Bernat notifications@github.com wrote:

Unfortunately, I don't have time at the moment. Writing random comments in GitHub issue is easier than proposing patches. :)

BTW, as an example of LISP as DSL, there is Riemann (http://riemann.io/, but it's written in Clojure, so, more natural choice).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/purpleidea/mgmt/issues/2#issuecomment-231061986, or mute the thread https://github.com/notifications/unsubscribe/AAmBbwr1R6cZSB8WaJpeqwKZB8w7rFC2ks5qTO9SgaJpZM4HKge_ .

[dnaeon]

Hi,

Just found your project, looks nice! :)

I'm also working on a similar project, where I've just recently implemented a DSL based on Lua. You might want to look at it in case you decide to go with Lua.

The project I'm working on is located at https://github.com/dnaeon/gru

[purpleidea]

@dnaeon Funny enough someone send me a pointer to your project just the other day, although I haven't had a chance to dig into the code yet, sorry.

I did look into using Lua, but for some design reasons unfortunately I don't think it would be a good choice, mainly because of a lack of safety constraints available in the language that we would require. The fact that you mention using it as a DSL, makes me think it's possibly a safe subset of the language that you're using?

In any case, I'd love to chat about it more, and maybe you want to join forces and help out in mgmt? Can I persuade you to have a look at my intro article or video?

https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/ https://www.youtube.com/watch?v=GVhpPF0j-iE&html5=1

I'm hanging out in the #mgmtconfig IRC channel on Freenode which you're more than welcome to join.

Cheers and thanks for the message, James @purpleidea

[dnaeon]

Hey James,

I'll check out the video and article, thanks for the links!

This week I'm traveling most of the time, but once I get back from my vacation I'll ping you in IRC.

Best regards, Marin

[purpleidea]

@dnaeon Sounds good. - Enjoy your vacation!

Cheers, James

[neilhwatson]

Jumping in here and I hope I get the context right. If you're going to invent a DSL for mgmt please consider publishing it as an open standard. There are so many OS config management tools and each has its own language, leading to lock-in. An open standard for future tools to use would be a great thing.

[purpleidea]

@neilhwatson I personally have no objections to this, there's nothing proprietary about what we're building. The first step though is to get the code written so we have a minimally viable project. Can you help with some patches ?

[shawncatz]

Hashicorp Config Language? I'm guessing it's been considered, but I didn't see it in this conversation (some did mention terraform, but not HCL)... Already implemented in go. Becoming well known. Much of the graph/dependency resolution might be borrowed from existing tools like Terraform.

I'm really interested in being involved in next-gen config management. As much as I love Chef, and others, they are all pretty terrible ecosystems.

[josephholsten]

@shawncatz I'd be interested in what you find lacking in the chef or puppet ecosystems. I've noticed a much higher amount of modular code and code reuse in those ecosystems than in ansible, terraform and salt.

I think one of the reasons is first class resource types. It requires a complete redesign and rewrite to convert an implementation from ansible task into ansible module, or terraform module into terraform provider. This makes it much harder to get a broad community of resource implementations, and makes it more expensive to prototype new approaches, and creates a hierarchy of contributors: "real" resource devs vs mere users.

I say this as a contributor to HCL and a committer on terraform. I <3 it, but its architecture has serious drawbacks for long term growth.

[purpleidea]

@shawncatz How can we help you get more involved? (Feel free to discuss this on IRC with us in #mgmtconfig on Freenode)

@josephholsten and @shawncatz I looked briefly at HCL, and unfortunately I think it lacks some of the features we need, since it's much closer to a yaml than a full DSL. Thanks for the idea though!

[josephholsten]

If we were looking for a fancy JSON/YAML/TOML alternative, https://github.com/vstakhov/libucl was the inspiration for HCL, and much nicer IM(NS)HO.

I'm mostly in favor of Lua as a golang encore nation language, but if we want a LISP, guile is quite nice.

[purpleidea]

As was suggested, I figured I'd like to the HCL lib here: https://github.com/hashicorp/hcl I had a quick look and it might turn out to be a reasonable base to modify for use in mgmt. hackers wanted.

[dnaeon]

Hey @purpleidea,

Just a thought about HCL - I've used HCL in the previous releases of Gru, and overall was pretty happy with it.

The syntax is fine and configuration expressed in HCL looks nice.

At some point you would want to introduce variable interpolations, add custom functions in the HCL configuration, etc. and would most likely end up with HIL as I did already.

At the end I gave up on HCL and HIL, mainly because of the reasons listed in this issue.

• https://github.com/dnaeon/gru/issues/30

With that said, I'm not saying that HCL/HIL are bad, they just didn't cut it for me and for what I wanted to achieve.

Anyways, thought I'd share this with you, in case you decice to go that way.

P.S. Seems we can't get a hold on IRC because of the time difference :) I'll be joining in there, maybe I'll get a hold of you next time :)

[purpleidea]

Hey @dnaeon, thanks for your comments, they're helpful. We're still looking into building more safety into the language, but maybe the HIL stuff will help. As an aside, I think you might enjoy reading this article: https://ttboj.wordpress.com/2016/10/07/remote-execution-in-mgmt/ We should chat sometime!

[mildred]

For the syntax, there are many options. I quite like the way ansible went by using YAML, there is not yet another syntax to learn, and it's an object model compatible with JSON that is very well known. HCL from Hashicorp is also good because it's simple and also maps to JSON. This can be really useful when you write tools that interacts with it (hint, I do).

I quite prefer having a syntax that is purely declarative, than something complex taken from a full-blown programming language. I want something very predictable.

LISP are greats, but I agree, not everyone wants it. But there is the possibility to have something similar but with other syntaxes. For example, there is the Io language syntax that is really neat. It looks like other programming languages, but with a LISP-like model below.

The example above could be written:

resources(
  exec(
    name("exec1")
    cmd("echo hello from exec1")
    timeout(0)
    watchcmd("sleep 10s")
    state(present)
  )
  exec(
    name("exec2")
    cmd("echo hello from exec2")
    watchcmd("sleep 10s")
    state(present)
  )
)
edges(
  edge(
    name("e1")
    from(exec, "exec1")
    to(exec, "exec2")
  )
)

You replace some parenthesis with curly braces, allow some operators to mean something, and you have a syntax that none would find strange, with all the power of a simple data model behind.

[purpleidea]

@mildred I've got a rough syntax in mind, but the syntax/lexing isn't the hard part for me, it's the FRP part that I'm trying to build. I'm hoping to have a very rough draft visible in 2017 that people can play with and comment on and that we can change if need be. If you think you can help out with some of the lexer/parser code, please let me know!

[mildred]

With all the languages I worked with until now, I came to love writing parsers from scratch, if that can help

[purpleidea]

On Wed, Feb 15, 2017 at 11:44 AM, mildred notifications@github.com wrote:

With all the languages I worked with until now, I came to love writing parsers from scratch, if that can help

If you have experience writing parsers and would like to help, we'd love your involvement!

[mildred]

@purpleidea, if you have an idea of the syntax, do you think it's matured enough so you can share it?

[purpleidea]

On Tue, Mar 21, 2017 at 7:20 PM, mildred notifications@github.com wrote:

@purpleidea, if you have an idea of the syntax, do you think it's matured enough so you can share it?

I'm working on it on my spare time, I've got if statements implemented, and I'm working on the "fact" and "function" API. After that's implemented, I'll push an early branch. If you've got an idea for a better name than "fact" (facts like in puppet, except they vary over time) please LMK :) If you'd like to help implement this all please lmk too!

Cheers

[mildred]

fact is good, and a fact in real life can change too. As for functions, these could be called modules or components (their job is to bundle together multiple resources)

[mildred]

facts and the file_line resource made me think how to express this in a syntax. I use terraform daily and something nice about it is that you define resources in any order and you can use variables from other resources in a resource. The terraform engine does the ordering itself and only errors out on cycles. You don't have to order the resources yourself.

I think we need some kind of interpolation syntax that can do that and uses SendRecv for it.

The only issue with the terraform syntax is the interpolation syntax. it's an after-thought and doesn't work nicely. You have to embed every variable in a string. Apart from that, I think we can get inspiration from it.

In my imagination, it would work like this:

# Declare we want to know about this file, this could be implicit
fact(file, name: "/etc/news/inn.conf")

# a file_line resource to modify the file
resource(file_line, name: "/etc/news/inn.conf") {
  input: fact.file[name].content
  section {
    before: "before pattern"
    after: "after pattern"
  }
  replace_line {
    pattern: "(option_name) = .*"
    replacement: "$1 = new_value"
  }
}

resource(file, name: "/etc/news.inn.conf") {
  # output is a computed value of the file_line resource
  content: resource.file_line[name].output
}

In this short example, the order of resources can be inferred from their dependency. the file resource needs a value from the file_line resource which needs a value from the file fact. This is a chain of SendRecv which also imposes edges for specific ordering.

[purpleidea]

I use terraform daily and something nice about it is that you define resources in any order

This is the case with the lang I am building. Funny enough this is not the case in puppet! (A mistake IMHO)

you can use variables from other resources in a resource.

Can you point me to some specifics docs and examples about this part please?

The terraform engine does the ordering itself and only errors out on cycles. You don't have to order the resources yourself.

This is the case in mgmt already.

[mildred]

Terraform build on the Hashicorp DSL that is generic and doesn't know what a variable is. It's basically a glorified JSON. Using variables in terraform is done in strings only using the interpolation syntax

Basically, a resource contains attributes you can set yourself to configure the resource. All of these attributes can be referenced from elsewhere. Attributes can also be optional or computed by terraform itself. It doesn't prevent from referencing them.

Terraform also contains data source (equivalent of the facts) that are executed before the resources and contains mostly computed values.

Example of a DNS resource that references an EC2 instance IP address:

resource "aws_route53_record" "main_alias" {
  zone_id = "${aws_route53_zone.internal_domain.zone_id}"
  name    = "www"
  type    = "CNAME"
  ttl     = "300"
  records = ["${aws_instance.coreos.public_ip}"]
}

resource "aws_instance" "coreos" {
  ami                         = "${data.aws_ami.stable_coreos.id}"
  instance_type               = "t2.micro"
  key_name                    = "${aws_key_pair.admin_key.key_name}"
  availability_zone           = "eu-west-1a"
  vpc_security_group_ids      = [ "${aws_security_group.base_sg.id}" ]
  subnet_id                   = "${data.aws_subnet.main.id}"
  associate_public_ip_address = true
  user_data = "..."
}

[purpleidea]

@mildred Cool, so this line:

records = ["${aws_instance.coreos.public_ip}"]

You basically have to know that the aws_instance produces that var, right? (Or did I misunderstand this?) If that's true, that should mean you automatically cause the 2nd thing in your example to happen before the first, right?

If so in many ways this smells like the send/recv stuff. That's great news!

The rough idea in mgmt (fake syntax since lang is WIP) is something like this:

# this resource produces some value named bar
x1 = resource::foo {
  some_param: 42,
  some_other_param: "hello",
}

f1 = resource::file {
  contents => r1.bar, // send->recv from x1
}

[mildred]

You basically have to know that the aws_instance produces that var, right? (Or did I misunderstand this?) If that's true, that should mean you automatically cause the 2nd thing in your example to happen before the first, right?

Yes, in the doc, aws_instance documents the public_ip attribute. Terraform uses the variable bindings to automatically order the resources.

[ChrisMcKenzie]

Hey, I am coming from an experiment (https://github.com/chrismckenzie/preflight) I was working on and @purpleidea invited me to come participate on this instead, with that being said my approach started with terraform inspiration, so naturally I started with using HCL as @mildred suggested. I really enjoy using HCL for thing and have had reasonable success with some of my other projects. (https://github.com/chrismckenzie/dropship)

TL;DR: I would love to put together a Proposal that would add HCL to mgmt. @mildred if you are interested in working on this together you are welcome too!

[purpleidea]

@ChrisMcKenzie Hey! Glad to see you've landed and are digging. Here's a bit of info which I hope you will find helpful:

Implementing a new "frontend" eg a language like HCL is I think quite straightforward on the mgmt side, since you only have to implement a small API. We already have: a) A small yaml -> mgmt graph format frontend b) An improved yaml -> mgmt frontend (thanks to @mildred) c) A Puppet -> mgmt frontend (thanks to @ffrank) d) There is an additional frontend a friend is prototyping, but not clear on the state so far. e) I'm working on an mgmt language. I have a mini prototype which is not quite finished.

It would be great to have your addition if this interests you. Alternatively, you're welcome to work on the language (e) stuff although it's not ready yet.

To implement a frontend, you just need to fulfill the GAPI: https://github.com/purpleidea/mgmt/blob/2f6c77fba212db615f5b2d7d580739f9234198bf/gapi/gapi.go#L49

You can look at the yamlgraph2/gapi.go for an example, or even examples/lib/*.go for some examples using it via raw golang. Once this is done, it usually gets tied in here: https://github.com/purpleidea/mgmt/blob/2f6c77fba212db615f5b2d7d580739f9234198bf/lib/cli.go#L57 (you could add a new entry for hcl for example.

LMK if you have more questions.

Thanks, James

[rismoney]

I like the idea of a full language, in conjunction with a DSL. Powershell DSC I think does this but everything is really posh, just like chef everything I think is Ruby valid.

Puppet DSL can be confining but the declarative nature and DAG is rather nice (maybe it's not a stellar implementation in clojure/Ruby) but fundamentally I like it.

Using an approach like Jenkins does with groovy, whitelisting subsets of a real language coupled with a DSL allows you to avoid the pigeon hole that HCL can't get around as everything is an afterthought now.

I just came across this page https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9 and essentially all the broken examples that won't work on TF I think should be workable without obscure hackish influence at every turn.

Overall I think what your attempting is awesome and look forward to where you take this!

[purpleidea]

@rismoney The terraform link was a good summary! I wrote a similar article many years ago about how to do iteration in puppet: https://ttboj.wordpress.com/2013/11/17/iteration-in-puppet/ -- the language work I'm doing will probably extend those concepts further but be substantially similar. If you have objections, speak up now :)

Would you like to get involved in helping build the language? Please let me know! Join us in #mgmtconfig IRC to help out!

[rismoney]

https://github.com/hashicorp/hcl2/blob/master/hcl/hclsyntax/spec.md https://github.com/hashicorp/hcl2/blob/master/hcl/spec.md

[purpleidea]

I forgot to mention it earlier, but we now have a language in git master...

It's not finished, but it's a solid start, and you can read about what I was doing here:

https://purpleidea.com/blog/2018/02/05/mgmt-configuration-language/

I've been thinking about this for a long time, even before this issue was opened, so I'm glad we've finally got something merged. Patches welcome if you find any issues!

Thanks!