kaspergrubbe commented 4 years ago

Nomad version

Output from 0.10.2, bug also tested and verified on 0.11.0-beta2.

Operating system and Environment details

Issue

When I read the documentation for ParseJob (https://nomadproject.io/api-docs/jobs/#parse-job) it says the following about Canonicalize:

Canonicalize (bool: false) - Flag to enable setting any unset fields to their default values.

Reproduction steps

curl -X "POST" "http://nomadserver.vpn.kaspergrubbe.com:4646/v1/jobs/parse" \
     -H 'Accept: application/json' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "Canonicalize": true,
  "JobHCL": "job \\"example\\" { parameterized = { meta_required = [\\"SLEEP_TIME\\"] } type = \\"service\\" group \\"cache\\" {} }" | jq '.["ParameterizedJob"]'
}'

Output:

{
  "MetaOptional": null,
  "MetaRequired": [
    "SLEEP_TIME"
  ],
  "Payload": ""
}

Expected output:

{
  "MetaOptional": null,
  "MetaRequired": [
    "SLEEP_TIME"
  ],
  "Payload": "optional"
}

When I go to http://nomadserver.vpn.kaspergrubbe.com:4646/ui/jobs/example/definition I also see that Payload is optional there:

When reading the structs file, it also doesn't seem like an empty value is even valid: https://github.com/hashicorp/nomad/blob/d3e72883343ca5c5dbf23c72a78ac9174e55f940/nomad/structs/structs.go#L4491-L4524

kaspergrubbe commented 4 years ago

I also just tested with 0.11.0-beta2 and the issue is also present there.

tgross commented 4 years ago

Thanks for the detailed report on this @kaspergrubbe! We'll take a look.

tgross commented 2 years ago

Doing some issue cleanup and was still able to reproduce this with HCLv1:

$ curl -XPOST -s "http://localhost:4646/v1/jobs/parse" \
     -d $'{
  "Canonicalize": true,
  "HCLv1": true,
  "JobHCL": "job \\"example\\" { parameterized = { meta_required = [\\"SLEEP_TIME\\"] } type = \\"service\\" group \\"cache\\" {} }"}' | jq '.["ParameterizedJob"]'

Returns:

{
  "MetaOptional": null,
  "MetaRequired": [
    "SLEEP_TIME"
  ],
  "Payload": ""
}

However, I've also discovered that the HCL2 parser doesn't really work at all for this API for any non-trivial jobspecs.

$ curl -XPOST -s "http://localhost:4646/v1/jobs/parse" \
     -d $'{
  "Canonicalize": true,
  "JobHCL": "job \\"example\\" { parameterized = { meta_required = [\\"SLEEP_TIME\\"] } type = \\"service\\" group \\"cache\\" {} }"}'
input.hcl:1,68-72: Invalid single-argument block definition; A single-line block definition must end with a closing brace immediately after its single argument definition.%

There's no way to express most of the jobspec as a single-line block in HCL2. When we eventually remove HCL1 entirely we may need to remove this API as well.

hashicorp / nomad