loki4j / loki-logback-appender

Fast and lightweight implementation of Logback appender for Grafana Loki
https://loki4j.github.io/loki-logback-appender/
BSD 2-Clause "Simplified" License
300 stars 26 forks source link

loki-logback-appender on Android with logback-android - send one time and never send again. #191

Closed allburov closed 11 months ago

allburov commented 11 months ago

Hi, thank you for the wonderful project you've made! We're using Timber as the main log system, looking forward for https://github.com/loki4j/loki-logback-appender/issues/113!

Right now I'm trying to add the Appender to Android project with logback-android and wanna implement my own Sender\Client for that (because there's no ApacheHttpClient or JavaHttpClient available for us, mobile developers :( )

I've written a simple fake client that just log send() command and the problem is that I see in logs it's been called once and this is it , even if I set batchTimeoutMs=1000. No errors after that, it just silently do nothing :(

2023-10-27 20:49:58.138 30788-30788 AndroidLokiHttpClient   com.revi.kitandroid                  D  AndroidLokiHttpClient(com.github.loki4j.client.http.HttpConfig@7c7b7f8)
2023-10-27 20:49:59.168 30788-30828 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()

I do understand that it's likely because of Android infrastructure and not fully available logback version for android, but may be you could guide me where I should look to fix the issue?


Here's a simple client that just logs the send() command and do nothing more (it's written on Kotlin, but it's easy to get idea - nothing more than Log)

package com.revi.kitandroid.util.logger

import android.util.Log
import com.github.loki4j.client.http.HttpConfig
import com.github.loki4j.client.http.HttpConfig.ClientSpecificConfig
import com.github.loki4j.client.http.Loki4jHttpClient
import com.github.loki4j.client.http.LokiResponse
import com.github.loki4j.client.pipeline.PipelineConfig
import com.github.loki4j.client.util.Loki4jLogger
import com.github.loki4j.logback.AbstractHttpSender
import java.nio.ByteBuffer
import java.util.function.Function

class AndroidHttpConfig : ClientSpecificConfig

class AndroidPipelineConfig(
    name: String,
    batchMaxItems: Int,
    batchMaxBytes: Int,
    batchTimeoutMs: Long,
    sortByTime: Boolean,
    staticLabels: Boolean,
    sendQueueMaxBytes: Long,
    maxRetries: Int,
    retryTimeoutMs: Long,
    internalQueuesCheckTimeoutMs: Long,
    useDirectBuffers: Boolean,
    drainOnStop: Boolean,
    metricsEnabled: Boolean,
    writerFactory: PipelineConfig.WriterFactory,
    httpConfig: HttpConfig,
    internalLoggingFactory: Function<Any?, Loki4jLogger?>
) : PipelineConfig(
    name,
    batchMaxItems,
    batchMaxBytes,
    batchTimeoutMs,
    sortByTime,
    staticLabels,
    sendQueueMaxBytes,
    maxRetries,
    retryTimeoutMs,
    internalQueuesCheckTimeoutMs,
    useDirectBuffers,
    drainOnStop,
    metricsEnabled,
    writerFactory,
    httpConfig,
    Function { config: HttpConfig? -> AndroidLokiHttpClient(config) },
    internalLoggingFactory,
    ) {
    companion object {
        fun android(): HttpConfig.Builder? {
            return HttpConfig.builder()
                .setClientConfig(AndroidHttpConfig())
        }
    }
}

class AndroidLokiHttpClient(config: HttpConfig?): Loki4jHttpClient {
    init {
        Log.d(TAG, "AndroidLokiHttpClient($config)")

    }
    override fun close() {
        Log.d(TAG, "close()")
    }

    override fun getConfig(): HttpConfig {
        Log.d(TAG, "getConfig()")
        return config
    }

    override fun send(batch: ByteBuffer): LokiResponse {
        Log.d(TAG, "send()")
        return LokiResponse(200, "")
    }
    companion object {
        private val TAG = AndroidLokiHttpClient::class.java.simpleName
    }
}

class AndroidLokiSender: AbstractHttpSender() {
    override fun getHttpClientFactory(): Function<HttpConfig?, Loki4jHttpClient?> {
        return Function { config: HttpConfig? -> AndroidLokiHttpClient(config) }
    }

    override fun getConfig(): HttpConfig.Builder {
        return AndroidPipelineConfig.Companion
            .android()!!
            .fill(this::fillHttpConfig);
    }
}

Here's my logback.xml

<configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="https://tony19.github.io/logback-android/xml"
    xsi:schemaLocation="https://tony19.github.io/logback-android/xml https://cdn.jsdelivr.net/gh/tony19/logback-android/logback.xsd">
    <appender name="LOGCAT" class="ch.qos.logback.classic.android.LogcatAppender">
        <tagEncoder>
            <pattern>%logger{12}</pattern>
        </tagEncoder>
        <encoder>
            <pattern>[%-20thread] %msg</pattern>
        </encoder>
    </appender>

    <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
        <lazy>true</lazy>
        <batchTimeoutMs>1000</batchTimeoutMs>
        <http class="com.revi.kitandroid.util.logger.AndroidLokiSender">
            <url>http://localhost:8000/</url>
        </http>
        <format>
            <label>
                <pattern>app=KIT,level=%level</pattern>
            </label>
            <message>
                <pattern>l=%level h=${HOSTNAME} c=%logger{20} t=%thread | %msg %ex</pattern>
            </message>
            <sortByTime>true</sortByTime>
        </format>
    </appender>

    <appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
        <appender-ref ref="LOKI" />
    </appender>

    <root level="DEBUG">
        <appender-ref ref="LOGCAT" />
        <appender-ref ref="ASYNC" />
    </root>
</configuration>
allburov commented 11 months ago

Dumb me, of course it logs only one time because I added only a single log.info expression :D Never mind, now I see all send() after each logs.

2023-10-27 20:58:26.716 31367-31367 AndroidLokiHttpClient   com.revi.kitandroid                  D  AndroidLokiHttpClient(com.github.loki4j.client.http.HttpConfig@be9258)
2023-10-27 20:58:27.748 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:28.855 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:29.940 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:31.037 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:38.648 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:39.847 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:40.942 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:48.557 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:49.866 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()

We can assume it works perfectly on Android with logback-android (except it has no client, but it's not a problem I hope and I'll do implement it in few days)