grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.88k stars 3.45k forks source link

Promtail: Add scrape_config to accept unstructured logs via HTTP #4316

Closed ldb closed 3 years ago

ldb commented 3 years ago

Is your feature request related to a problem? Please describe. At the moment there is a way to send 'unstructured' (in the sense of schemaless) logs to Loki via the STDIN scrape_config in Promtail. Structured logs can be sent to Loki using the loki_push_api scrape_config using HTTP. We propose to add a similar feature that accepts unstructured logs via HTTP similar to the STDIN way. This enables a set of new integrations such as with Vercel Log Drains for sending logs via HTTP.

Describe the solution you'd like Add a scrape_config to accept generic logs via HTTP.

Describe alternatives you've considered Other ways of connecting cloud services to Promtail, which usually involve writing custom glue code to accept logs via HTTP only to pipe them into Promtail.

Additional context

aengusrooneygrafana commented 3 years ago

Hi Leo, thanks for submitting this - as discussed, we'll check in with @cyriltovena to get his 👀 on this!