Overriding sidecar_task docker config overrides other settings

Nomad version

Nomad v1.1.4 (acd3d7889328ad1df2895eb714e2cbe3dd9c6d82)

Operating system and Environment details

Vagrant with Virtualbox running Ubuntu focal Linux ubuntu-focal-node1 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Issue

When creating a job that sets a sidecar_task stanza, I override some of the docker config to pass custom Envoy flags to change the log formatting rules. By doing so Nomad seems to use job wide default resource and log manager config rather than the default values used by the sidecar. The sidecar ends up with 300MB instead of 128MB, and log rotation set to 10 max files instead of 2.

So it applies:

https://www.nomadproject.io/docs/job-specification/resources#memory

instead of

https://www.nomadproject.io/docs/job-specification/sidecar_task#default-envoy-configuration

Reproduction steps

Create a job that overrides the docker config in sidecar_task stanza. The default resources of a sidecar use the normal job defaults instead of the sidecar specific defaults, casuing more resources to be used than expected.

Expected Result

I expected Nomad to merge the inbuilt sidecar defaults with my own overrides. Resources should remain as documented on the page https://www.nomadproject.io/docs/job-specification/sidecar_task#default-envoy-configuration rather than https://www.nomadproject.io/docs/job-specification/resources#memory, if the memory has not been overridden by me.

Actual Result

Memory allocated used job defaults of 300MB.

Job file (if appropriate)

job elasticsearch {
  datacenters = ["dc1"]
  type = "service"

  reschedule {
    delay = "5s"
    delay_function = "constant"
    unlimited = true
  }

  group "elasticsearch" {
    count = 1

    restart {
      attempts = 3
      delay = "0s"
      mode = "fail"
    }

    volume "elasticsearch" {
      type = "host"
      read_only = false
      source = "elasticsearch"
    }

    network {
      mode = "bridge"
    }

    service {
      name = "elasticsearch"
      port = "9200"

      connect {
        sidecar_service { }

        sidecar_task {
          config {
            args = [
              "-c",
              "${NOMAD_SECRETS_DIR}/envoy_bootstrap.json",
              "-l",
              "${meta.connect.log_level}",
              "--concurrency",
              "${meta.connect.proxy_concurrency}",
              "--disable-hot-restart",
              "--log-format",
              "{\"level_name\": \"%l\", \"file\": \"%g\", \"message\": \"%j\"}",
            ]

            logging {
              type = "journald"
            }
          }
        }
      }
    }

    task "elasticsearch" {
      user = "790"
      driver = "docker"

      config {
        image = "elasticsearch:7.10.1"
        volumes = ["elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml"]

        logging {
          type = "journald"
        }

        ulimit {
          memlock = "-1"
        }
      }

      volume_mount {
        volume   = "elasticsearch"
        destination = "/usr/share/elasticsearch/data"
        read_only   = false
      }

      env {
        ES_JAVA_OPTS = "-Xms512m -Xmx512m"
      }

      template {
        data = <<EOH
node.name: "es1"
cluster.name: "es"
network.host: 0.0.0.0

bootstrap.memory_lock: true
cluster.initial_master_nodes: "es1"

xpack.security.enabled: false
xpack.monitoring.enabled: false
xpack.graph.enabled: false
xpack.watcher.enabled: false
EOH
        destination = "elasticsearch.yml"
      }

      resources {
        cpu    = 100
        memory = 1024
      }
    }
  }
}

hashicorp / nomad