search

hashicorp / levant

An open source templating and deployment tool for HashiCorp Nomad jobs
Mozilla Public License 2.0
829 stars 125 forks source link

Levant does not honour region defined inside job spec #289

Open ExalDraen opened 5 years ago

ExalDraen commented 5 years ago

Description

Levant does not use the region specified inside a job spec to deploy the given job to the correct region (cf. docs for region key) Instead, the job is submitted to the default region, or the region specified by the NOMAD_REGION environment variable if that is set.

This behaviour is confusing and differs from the native nomad CLI. The native CLI will use the region specified in a job spec to choose the region to which a job will be submitted.

See the docs for how region is meant to be interpreted.

Bug Analysis

The native CLI explicitly configures the nomad client to use the region specified in the job file - e.g. see the code for job run: https://github.com/hashicorp/nomad/blob/master/command/job_run.go#L165

In contrast, levant does not do this as far as I can see from the code. This results in the client not being configured with the correct region.

Running tcpdump demonstrates this difference - note the difference in query parameters:

# Captured while running nomad job run redis.nomad
tcpdump: listening on enp0s20f0u5u3, link-type EN10MB (Ethernet), capture size 262144 bytes
....x..:PUT /v1/jobs?region=us-central1 HTTP/1.1
# Captured while running levant deploy -ignore-no-changes redis.nomad
tcpdump: listening on enp0s20f0u5u3, link-type EN10MB (Ethernet), capture size 262144 bytes
....x..+PUT /v1/job/redis/plan HTTP/1.1
....w..EPUT /v1/validate/job HTTP/1.1
...-.V..PUT /v1/jobs HTTP/1.1

Relevant Nomad job specification file

Exact same as https://github.com/hashicorp/nomad-guides/blob/master/application-deployment/redis/redis.nomad but with region and datacenter set to us-central1.

Output of levant version:

$ levant version
Levant v0.2.7
Date: 2019-03-19T08:26:24Z
Commit: 9e952d55f171e63f5c7955e826401eac91ed0b28
Branch: 0.2.7
State: 0.2.7
Summary: 9e952d55f171e63f5c7955e826401eac91ed0b28

Output of nomad version:

CLI executable:

Nomad v0.8.6 (ab54ebcfcde062e9482558b7c052702d4cb8aa1b+CHANGES)

Servers:

Nomad v0.9.0 (18dd59056ee1d7b2df51256fe900a98460d3d6b9)

Debug log outputs from Levant:

Attempt to deploy to us-central1 when default is another region - deployment stalls as no nodes in the other region have the correct region and datacenter set.

levant deploy -log-level=DEBUG -ignore-no-changes redis.nomad
2019-05-30T11:01:35+08:00 |DEBU| template/render: no variable file passed, trying defaults
2019-05-30T11:01:35+08:00 |DEBU| helper/files: no default var-file found
2019-05-30T11:01:35+08:00 |DEBU| template/render: no command line variables passed
2019-05-30T11:01:35+08:00 |DEBU| levant/plan: triggering Nomad plan
2019-05-30T11:01:35+08:00 |INFO| levant/deploy: triggering a deployment job_id=redis
2019-05-30T11:01:35+08:00 |INFO| levant/deploy: beginning deployment watcher for job job_id=redis
2019-05-30T11:01:35+08:00 |DEBU| levant/deploy: deployment b3bf1e04-4bcc-6a9d-f08d-90ff78cf1449 running for 0.14s job_id=redis
(cancelled)
vizotovclarus commented 2 years ago

it's happen to me when i.m run job in Jenkins-in-docker in Jenkins-on-VM same levant and job+parameters works fine

so sad, now ned thing how inject parameters another way in levant process, or use another runner, or some else.

qk4l commented 5 months ago

There is PR which based description fixed this #428