Is it worth mentioning how to enable logs?

[xynova]

Should the examples show how to turn logs on?

  metadata = {
    gce-container-declaration = module.container_metadata_spec.metadata_value
    google-logging-enabled = "true"
  }

[morgante]

Sure, feel free to add this to an example.

[dts]

This would be great - I am happy to push up a PR if that's helpful, but I don't want to step on anyones toes.

[xynova]

no worries, I forked the project in order to do it and never did, sorry about that

[morgante]

A PR would definitely be appreciated.

[dts]

@morgante - let me know if you had something else in mind r.e. PR #62

[0xjjpa]

Sorry to comment on a closed thread, but albeit the logging seems to be enabled, it seems that unless we enable gcplogs this would effectively only log the machine vitals and not the logs of the actual container.

It seems that there's already some train of thought for enabling this, but I've yet to find an example on how to enable this for a COS image. From my understanding, we need to add something like the following in the COS image under /etc/docker/daemon.json:

{
  "log-driver": "gcplogs",
  "log-opts": {
    "gcp-project": "removed",
    "env": "host"
  }
}

Right now I'm using cos-dev-84-13078-0-0 and I get the following information:

$ cat /etc/docker/daemon.json
{
        "live-restore": true,
        "storage-driver": "overlay2",
        "mtu": 1460
}

In short, even if we add this line (i.e. google-logging-enabled = "true") in our VM metadata definition, the agent has yet to know how to send the container information to GCP. Would that be a correct statement?

[morgante]

Interesting, I haven't dug into this but can reopen the issue if someone wants to investigate further.

[0xjjpa]

Probably relevant to link https://github.com/GoogleCloudPlatform/konlet/issues/56, as it seems that there's an ongoing conversation there about the limitation of enabling gcplogs on Konlet as of right now.

[DeanBrunt]

For anyone who stumbles upon this. I had this issue and it was remedied by adding the google-logging-enabled metadata key.

I got caught out as I was using a non-default service account which didn't have permissions to write logs, so the agent came up and then was unable to write to Stackdriver.

Once I sorted the permissions out all was fine.

[meldron]

Edit: so in the end google-logging-enabled = "true" was enough.

I guess I also had some problems with the service key or something like that.

It took me awhile to get logging working:

locals {
  additional_metadata = {
    google-logging-enabled    = "true"
    google-monitoring-enabled = "true"
    user-data                 = file("./etc/gcp_logging_driver.yml")
  }
}

module "gce-container" {
  source  = "terraform-google-modules/container-vm/google"
  version = "~> 2.0"

  container = {
    image = local.image
    env   = local.env_vars
  }

  restart_policy = "Always"
}

module "mig_template" {
  source               = "terraform-google-modules/vm/google//modules/instance_template"
  version              = "6.1.0"
  network              = google_compute_network.default.self_link
  subnetwork           = google_compute_subnetwork.default.self_link
  machine_type         = var.machine_type
  service_account      = var.service_account
  name_prefix          = var.mig_name
  source_image_family  = "cos-stable"
  source_image_project = "cos-cloud"
  source_image         = reverse(split("/", module.gce-container.source_image))[0]
  metadata             = merge(local.additional_metadata, map("gce-container-declaration", module.gce-container.metadata_value))
  tags                 = local.target_tags
  labels = {
    "container-vm" = module.gce-container.vm_container_label
  }
}

Content of ./etc/gcp_logging_driver.yml

#cloud-config

write_files:
    - path: /etc/docker/daemon.json
      content: '{"log-driver":"gcplogs"}'

runcmd:
    - systemctl restart docker

This user-data scripts overwrites /etc/docker/daemon.json with

{"log-driver":"gcplogs"}

[fdcds]

FYI: There is a new Google Ops Agent that is supposed to replace the legacy logging and monitoring agents, but it does not appear to support Google Container-Optimized OS: https://github.com/GoogleCloudPlatform/ops-agent/issues/325

[hakuno]

For anyone who stumbles upon this. I had this issue and it was remedied by adding the google-logging-enabled metadata key.

I got caught out as I was using a non-default service account which didn't have permissions to write logs, so the agent came up and then was unable to write to Stackdriver.

Once I sorted the permissions out all was fine.

It solves the problem. Thanks.