search

hashicorp / nomad

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
https://www.nomadproject.io/
Other
15k stars 1.96k forks source link

Include / Import External Files (split Job in more files) #7954

Open pcbl opened 4 years ago

pcbl commented 4 years ago

Hi Folks,

I must start with a disclaimer, mentioning that I am new to NOMAD and might be missing something.

The question is simple: Does HCL support some sort of include/import, so that I could split a job file among several files, for the sake of maintenance?

The scenario: I do have one specific nomad job wich includes several Tasks. Those have some level of dependency among them, therefore we keep it within a single job. As the file is becoming somehow too big, we would like to split every Task within a separate file, including all of them on the main Job file.

Would that be possible? I have searched a little bit and found this issue, which seems somehow similar, but it is quite old (2017). https://github.com/hashicorp/hcl/issues/187

Maybe there´s a good way to do it... I certainly could put up a script that merges files and so on... But just wanted to ensure that I am not missing any cool HCL / NOMAD feature! :-)

Would anyone has a suggestion?

BR Pedro

tgross commented 4 years ago

Hi @pcbl! This isn't something you can do today with Nomad's job files. We do support it for Nomad configuration files. We have plans to implement HCL2 in the near future and I think this would be an interesting feature to look into.

pcbl commented 4 years ago

Ok @tgross , thanks for the heads up. I will keep an eye here case this feature comes up. By now I will end up build some solution to merge the files myself before I start the job. Thx!

radeksimko commented 4 years ago

As part of looking into this I'd also recommend standardizing nomad as the file extension for job specs, or coming up with an extension which can even differentiate agent config from a job spec - e.g. *.job.nomad vs *.agent.nomad.

Product-specific extension (unlike generic .hcl) enables editors to provide more accurate highlighting, completion etc. and they don't have to guess what flavour of HCL it is based on content and what schema to expect.

sofixa commented 3 years ago

So now with HCL2 it's possible to include files:

https://www.nomadproject.io/docs/job-specification/hcl2/functions/file/file

But without any templating/rendering, unlike terraform which has templatefile, which can render a template from a file. Are there plans to add this to Nomad's HCL2 implementation ?

Oloremo commented 3 years ago

Looking for that as well and actually not sure how to use https://www.nomadproject.io/docs/job-specification/hcl2/functions/file/file to split the HCL2.

Can I do something like:

job "name" {
  group "name" {
    task "name1" {
      file("path/to/file/with/all/stanzas/from/that/task")
    }
    task "name2" {
      file("path/to/file/with/all/stanzas/from/that/task")
    }
  }
}

?