grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.39k stars 3.39k forks source link

question about logstash-output-loki #9755

Open tbs575 opened 1 year ago

tbs575 commented 1 year ago

Using logstash-output-loki with docker, have some question.

Just using filebeat, logstash-output-loki will be ok? I setup logstash-output-loki container, expose 5044 to accept logs from filebeat, and then send logs to loki. I saw filebeat working by checking filebeat's log. but I can not saw any incoming log in loki by grafana.

logstash-output-loki config

input {
  beats {
    port => 5044
  }
}

filter {
  mutate {
    add_field => {
      "job" => "logstash"
    }
  }
}

output {
  loki {
    url => "http://10.200.101.2:3100/loki/api/v1/push"
  }
  # stdout { codec => rubydebug }
}

check logstash-output-loki container log, found error

[2023-06-21T06:58:08,500][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch"}
[2023-06-21T06:58:08,520][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.

I wounder why access elasticsearch? I have no any elasticsearch server in my enviroment.

is there any missing? thanks your help

miglocatiq commented 1 year ago

I'm having the same problem. The Logstash logs show errors like these:

[2023-08-31T09:41:50,064][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
[2023-08-31T09:41:50,594][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch: Name or service not known"}

But I don't have elastic search anywhere in my setup and I don't think I should need it.

miglocatiq commented 1 year ago

I'm having the same problem. The Logstash logs show errors like these:

[2023-08-31T09:41:50,064][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
[2023-08-31T09:41:50,594][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://elasticsearch:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch: Name or service not known"}

But I don't have elastic search anywhere in my setup and I don't think I should need it.

I fixed the error by adding this to the helm values:

logstashConfig:
  logstash.yml: |
    http.host: "0.0.0.0"
HelonLee135 commented 1 month ago

Is anyone else following this? I'm following what the tier above says and it still doesn't work, when I click test in grafana it says: Data source connected, but no labels received. Verify that Loki and Promtail is configured properly.