grafana / alloy-modules

Apache License 2.0
21 stars 11 forks source link

[Module components] Add provides receivers module #15

Open qclaogui opened 7 months ago

qclaogui commented 7 months ago

What this PR does

This PR comes from receivers provider, a module provides receivers components for collecting data(logs metrics traces profiles).

An simple use case config.alloy like so:

import.git "provider" {
  repository     = "https://github.com/qclaogui/codelab-monitoring.git"
  revision       = "main"
  path           = "alloy-modules/provider"
  pull_frequency = "24h"
}

// get the receivers from provider
provider.self_hosted_stack "compose" {
  metrics_endpoint_url  = "http://mimir:8080/api/v1/push"
}

// scrape metrics and write to metric receiver
prometheus.scrape "default" {
  targets = [
    {"__address__" = "127.0.0.1:12345"},
  ]

  forward_to = [
    provider.self_hosted_stack.compose.metrics_receiver,
  ]
}

Here is an example of my use of all-in-one.alloy in a docker compose scenario:

logging {
    level  = coalesce(env("ALLOY_LOG_LEVEL"), "info")
    format = "logfmt"
}

/********************************************
 * Grafana LGTMP Stack Receiver Provider
 ********************************************/
import.git "provider" {
    repository     = "https://github.com/qclaogui/codelab-monitoring.git"
    revision       = "main"
    path           = "alloy-modules/provider"
    pull_frequency = "24h"
}

+provider.self_hosted_stack "compose" {
+   metrics_endpoint_url  = coalesce(env("SELF_HOSTED_METRICS_ENDPOINT_URL"), "http://gateway:8080/api/v1/push")
+   logs_endpoint_url     = coalesce(env("SELF_HOSTED_LOGS_ENDPOINT_URL"), "http://gateway:3100/loki/api/v1/push")
+   traces_endpoint_url   = coalesce(env("SELF_HOSTED_TRACES_ENDPOINT_URL"), "http://gateway:4318")
+   profiles_endpoint_url = coalesce(env("SELF_HOSTED_PROFILES_ENDPOINT_URL"), "http://gateway:4040")
+}

/********************************************
 * Metrics
 ********************************************/
import.file "metrics" {
    filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/metrics"
}

metrics.labels_scrape "compose" {
    label_prefix    = "metrics.grafana.com"
+   forward_to      = [provider.self_hosted_stack.compose.metrics_receiver]
    scrape_interval = "15s"
}

/********************************************
 * Logs
 ********************************************/
import.file "logs" {
    filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/logs"
}

logs.labels_scrape "compose" {
    label_prefix = "logs.grafana.com"
    forward_to   = [logs.keep_labels.compose.receiver]
}

// logs Processing And Transformation
logs.keep_labels "compose" {
+   forward_to = [provider.self_hosted_stack.compose.logs_receiver]
}

/********************************************
 * Traces
 ********************************************/
import.file "traces" {
    filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/traces"
}

// traces Processing And Transformation process_and_transform
traces.process_and_transform "compose" {
+   metrics_forward_to = [provider.self_hosted_stack.compose.metrics_receiver]
+   logs_forward_to    = [provider.self_hosted_stack.compose.logs_receiver]
+   traces_forward_to  = [provider.self_hosted_stack.compose.traces_receiver]
}

tracing {
    // Write all spans. Don't do this in production!
    sampling_fraction = 1

    // Forward Alloy internal spans to traces process.
    write_to = [traces.process_and_transform.compose.alloy_traces_input]
}

/********************************************
 * Profiles
 ********************************************/
import.file "profiles" {
    filename = coalesce(env("ALLOY_MODULES_FOLDER"), "/etc/alloy/modules") + "/compose/profiles"
}

profiles.labels_scrape "compose" {
    label_prefix = "profiles.grafana.com"
+   forward_to   = [provider.self_hosted_stack.compose.profiles_receiver]
}

Which issue(s) this PR fixes or relates to

12

In addition, I also add the grafana_cloud component in the receivers provider moudle

bentonam commented 7 months ago

@qclaogui thank you for the contribution. Would you mind ensuring that the CLA is signed, all checks should pass, it is currently failing the markdownlint checks. Also please ensure the branch is up-to-date. Thanks!

CLAassistant commented 7 months ago

CLA assistant check
All committers have signed the CLA.

qclaogui commented 6 months ago

it is currently failing the markdownlint checks. Also please ensure the branch is up-to-date.

Hi @bentonam, Thank you for taking the time to reviews, I have run make lint-markdown and it has passed the markdownlint check