Are there examples of properly formatted log files that will work going to Loki from the logstash-output-loki plugin?

[mwimpelberg28]

I currently have two configurations for logstash-output-loki going to Grafana Cloud. One works and the other doesn't. This example works because I'm using the generator block.

input {
    generator {
        message => "Hello world!"
        count => 10
        add_field => {cluster=> "foo" namespace=>"bar"}
    }
}

output {
  loki {
            url => "https://logs-prod-006.grafana.net/loki/api/v1/push"
            username => "REDACTED"
            password => "REDACTED"
    }
  stdout {
    codec => rubydebug
  }
}

This configuration does not work

input {
  file {
    path => "/var/log/logfile.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

output {
  loki {
            url => "https://logs-prod-006.grafana.net/loki/api/v1/push"
            username => "REDACTED"
            password => "REDACTED"
    }
  stdout {
    codec => rubydebug
  }
}

My log file is one line 2024-08-05 11:39:40.070 Hello world! When I run sudo -u logstash /usr/share/logstash/bin/logstash --debug -f /etc/logstash/conf.d/logstash.conf This is the error that I receive

[DEBUG] 2024-08-05 17:17:09.087 [[main]<file] sincedbcollection - writing sincedb (delta since last write = 1722878229)
{
         "event" => {
        "original" => "2024-08-05 11:39:40.070\tHello world!"
    },
    "@timestamp" => 2024-08-05T17:17:09.080078510Z,
           "log" => {
        "file" => {
            "path" => "/var/log/logfile.log"
        }
    },
       "message" => "2024-08-05 11:39:40.070\tHello world!",
      "@version" => "1",
          "host" => {
        "name" => "ip-192-168-49-236"
    }
}
[DEBUG] 2024-08-05 17:17:10.801 [pool-3-thread-1] cgroup - One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[DEBUG] 2024-08-05 17:17:10.925 [pool-3-thread-1] jvm - collector name {:name=>"G1 Young Generation"}
[DEBUG] 2024-08-05 17:17:10.928 [pool-3-thread-1] jvm - collector name {:name=>"G1 Old Generation"}
[DEBUG] 2024-08-05 17:17:11.044 [Ruby-0-Thread-9: /usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-loki-1.1.0/lib/logstash/outputs/loki.rb:88] loki - Max batch_wait time is reached. Sending batch to loki
[DEBUG] 2024-08-05 17:17:11.057 [Ruby-0-Thread-9: /usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-loki-1.1.0/lib/logstash/outputs/loki.rb:88] loki - sending 102 bytes to loki
[DEBUG] 2024-08-05 17:17:11.830 [Ruby-0-Thread-9: /usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-loki-1.1.0/lib/logstash/outputs/loki.rb:88] loki - failed payload {:payload=>"{\"streams\":[{\"stream\":{},\"values\":[[\"1722878229080078592\",\"2024-08-05 11:39:40.070\\tHello world!\"]]}]}"}
[DEBUG] 2024-08-05 17:17:14.005 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.

Does anyone see an issue with the log file? I know that the Loki output portion works because the generator input sends the log lines to Loki.

[tonyswu]

You need at least one label. Try setting a static one.

[JStickler]

We no longer recommend using Logstash to send logs to Loki. See this PR for more details.