Pushing logs to hosted Loki results in "no credentials provided" error.

[deepu-james]

I use Grafana cloud which provides hosted Loki. The current setup logs to a file and Promtail push the logs to Loki. The Promtail configuration for Loki accepts a URL that has the username and password along with the URL. The format of the URL is: https://9xxx:<my_api_key>=@logs-prod-us-central1.grafana.net/api/prom/push. I was trying the appender to replace the promtail configuration. The appender configuration looks like:

<appender name="LOKI" class="com.github.loki4j.logback.LokiApacheHttpAppender">
    <encoder class="com.github.loki4j.logback.JsonEncoder">
        <label>
            <pattern>host=${HOSTNAME},level=%level</pattern>
            <pairSeparator>,</pairSeparator>
            <keyValueSeparator>=</keyValueSeparator>
            <nopex>true</nopex>
        </label>
        <staticLabels>false</staticLabels>
    </encoder>
    <url>https://9xxx:<my_api_key>=@logs-prod-us-central1.grafana.net/api/prom/push</url>
    <connectionTimeoutMs>30000</connectionTimeoutMs>
    <requestTimeoutMs>5000</requestTimeoutMs>
</appender>

I get this error with the above configuration:

12:45:02,845 |-ERROR in com.github.loki4j.logback.LokiApacheHttpAppender[LOKI] - Loki responded with non-success status 401 on Batch #2d7ce8a75e8 (13 records). Error: {"status": "error", "error": "no credentials provided"}

Does the appender support pushing logs to hosted Loki?

[nehaev]

Thanks for reporting! I haven't checked if appenders can push logs to Grafana Cloud. This could require some additional configs for HttpClient. I'll investigate this and update the issue with the results.

[nehaev]

Looks like we need to add a support for HTTP basic auth in order to make it work. Created a dedicated issue for it: #49.

BTW, Grafana Cloud currently accepts logs only in protobuf format, so you would need to switch from JsonEncoder to ProtobufEncoder.

[nehaev]

Grafana Cloud currently accepts logs only in protobuf format

I need to correct myself. Grafana Cloud accepts JSON as well as Protobuf, but you would need to switch from the deprecated endpoint /api/prom/push to the recommended one /loki/api/v1/push. It is not currently mentioned in Grafana Cloud instruction, but it could be found in Loki docs.

[deepu-james]

Thanks @nehaev, do you know when a version with Basic Auth will be released?

[nehaev]

@deepu-james The final release is expected to be out within one or two weeks. Meanwhile you can try pre-release version 1.0.0-rc1. The configuration changed significantly, and I need to update docs before releasing the whole thing. But given the configuration you've provided in this issue, you can rewrite it as the following (omitting some defaults):

    <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
        <http class="com.github.loki4j.logback.ApacheHttpSender">
            <url>https://logs-prod-us-central1.grafana.net/loki/api/v1</url>
            <auth>
                <username>your_username</username>
                <password>your_api_token</password>
            </auth>
        </http>
        <format>
            <label>
                <pattern>host=${HOSTNAME},level=%level</pattern>
            </label>
        </format>
    </appender>

If you encounter any issues, feel free to report!

[nehaev]

1.0.0 is finally released. Closing this issue.