Closed huib-coalesce closed 2 months ago
@huib-coalesce Sorry, missed this before.
Aren't the messages being sent to the error output? Filing a Google Cloud case with job IDs might be useful so the team can look further.
No they're not as far as I'm aware.
ConvertToDataDogEvent
(left) shows data going in and out (on the right):
Create KV pairs
shows data going in, but not out:
Write Datadog events
shows no incoming data:
WrapDatadogWriteErrors
shows no incoming/outgoing data:
FlattenErrors
shows no incoming/outgoing data:
Same for WriteFailedRecords
:
And the dev-dd-dead-letter
topic doesn't show anything either:
Could you please check that Log Router Sinks is getting logs properly? I guess you are missing "roles/pubsub.publisher" role for the sink.
In my case, i have created the sink my own not with module but maybe it can refer you how to add it.
resource "google_logging_project_sink" "datadog_sink" {
name = "kubernetes_container_error_logs"
destination = "pubsub.googleapis.com/${google_pubsub_topic.export_logs_to_datadog.id}"
filter = "resource.type=k8s_container AND log_id(stderr) AND severity>=ERROR"
unique_writer_identity = true
}
resource "google_project_iam_member" "pubsub_publisher_permisson_sink" {
project = var.project_id
role = "roles/pubsub.publisher"
member = google_logging_project_sink.datadog_sink.writer_identity
}
Could you please check that Log Router Sinks is getting logs properly?
The Log Router Sink is receiving content:
I guess you are missing "roles/pubsub.publisher" role for the sink
To try your suggestion, I've added:
resource "google_project_iam_member" "pubsub_publisher_permisson_sink" {
project = var.project_id
role = "roles/pubsub.publisher"
member = module.log_export.writer_identity
}
And changed the filter from
(severity >= DEBUG AND resource.type=\"cloud_function\") OR (severity >= WARNING AND resource.type=\"cloud_run_revision\")
to
severity>=ERROR
which ensures there are regular messages to pass through:
The Log Router Sink:
The service account:
The Topic Metrics shows content arriving:
Same for the Topic Subscription:
The point where messages are come in, but are not going out:
I however did notice:
But I don't know the significance of that.
This issue has been marked as stale due to 180 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the issue at any time. Thank you for your contributions.
This issue has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
Related Template(s)
Pub/Sub to Datadog template
What happened?
Exported log messages don't appear in Datadog.
Steps to reproduce
I've used the following Terraform definition to set the infrastructure up.
main.tf
variables.tf
terraform.tfvars
Result
I have a Log Router Sink.
That's connected to a Pub/Sub Topic:
It has a Topic Subscription with a Delivery type of Pull:
And finally, there's the Dataflow Job
However, it appears to me that the messages are never sent to Datadog.
But there are no errors in the logs:
And there's nothing in Datadog:
Beam Version
Newer than 2.46.0
Relevant log output
No response