[Q] How to add missing metadata to Google logging agent?

[raszkiewicz]

I'm shipping logs from on-premise to the GCP using google-fluentd agent but on the GCP side it has missing few metadata like resource.labels.instance_id

Is there any way to have it set via funentd configuration?

[junkett]

Hi have you figured this out? I was able to set this up like this (this is my template for ansible - but I guess you will get the idea):

# Configure all sources to output to Google Cloud Logging
<match **>
  @type google_cloud
  buffer_type file
  buffer_path /var/log/td-agent/buffers
  buffer_chunk_limit 512KB
  flush_interval 5s
  disable_retry_limit false
  retry_limit 3
  retry_wait 10
  max_retry_wait 300
  num_threads 8
  use_grpc true
  partial_success true
  enable_monitoring true
  monitoring_type opencensus

  zone {{ region }}
  vm_id {{ ovh_id }}
  vm_name {{ ansible_hostname }}
  labels namespace:syslog,provider:{{ provider }},rack:{{ rack }}
  use_metadata_service false
</match>

I am however not able to convince the plugin to send data for type: "general_node" only for type:"gce_node" I was able to send this via fluentbit but not fluentd... If anyone has idea - I would be very grateful for any hints, thanks...