loki4j / loki-logback-appender

Fast and lightweight implementation of Logback appender for Grafana Loki
https://loki4j.github.io/loki-logback-appender/
BSD 2-Clause "Simplified" License
314 stars 27 forks source link

loki-logback-appender on Android with logback-android - send one time and never send again. #191

Closed allburov closed 1 year ago

allburov commented 1 year ago

Hi, thank you for the wonderful project you've made! We're using Timber as the main log system, looking forward for https://github.com/loki4j/loki-logback-appender/issues/113!

Right now I'm trying to add the Appender to Android project with logback-android and wanna implement my own Sender\Client for that (because there's no ApacheHttpClient or JavaHttpClient available for us, mobile developers :( )

I've written a simple fake client that just log send() command and the problem is that I see in logs it's been called once and this is it , even if I set batchTimeoutMs=1000. No errors after that, it just silently do nothing :(

2023-10-27 20:49:58.138 30788-30788 AndroidLokiHttpClient   com.revi.kitandroid                  D  AndroidLokiHttpClient(com.github.loki4j.client.http.HttpConfig@7c7b7f8)
2023-10-27 20:49:59.168 30788-30828 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()

I do understand that it's likely because of Android infrastructure and not fully available logback version for android, but may be you could guide me where I should look to fix the issue?


Here's a simple client that just logs the send() command and do nothing more (it's written on Kotlin, but it's easy to get idea - nothing more than Log)

package com.revi.kitandroid.util.logger

import android.util.Log
import com.github.loki4j.client.http.HttpConfig
import com.github.loki4j.client.http.HttpConfig.ClientSpecificConfig
import com.github.loki4j.client.http.Loki4jHttpClient
import com.github.loki4j.client.http.LokiResponse
import com.github.loki4j.client.pipeline.PipelineConfig
import com.github.loki4j.client.util.Loki4jLogger
import com.github.loki4j.logback.AbstractHttpSender
import java.nio.ByteBuffer
import java.util.function.Function

class AndroidHttpConfig : ClientSpecificConfig

class AndroidPipelineConfig(
    name: String,
    batchMaxItems: Int,
    batchMaxBytes: Int,
    batchTimeoutMs: Long,
    sortByTime: Boolean,
    staticLabels: Boolean,
    sendQueueMaxBytes: Long,
    maxRetries: Int,
    retryTimeoutMs: Long,
    internalQueuesCheckTimeoutMs: Long,
    useDirectBuffers: Boolean,
    drainOnStop: Boolean,
    metricsEnabled: Boolean,
    writerFactory: PipelineConfig.WriterFactory,
    httpConfig: HttpConfig,
    internalLoggingFactory: Function<Any?, Loki4jLogger?>
) : PipelineConfig(
    name,
    batchMaxItems,
    batchMaxBytes,
    batchTimeoutMs,
    sortByTime,
    staticLabels,
    sendQueueMaxBytes,
    maxRetries,
    retryTimeoutMs,
    internalQueuesCheckTimeoutMs,
    useDirectBuffers,
    drainOnStop,
    metricsEnabled,
    writerFactory,
    httpConfig,
    Function { config: HttpConfig? -> AndroidLokiHttpClient(config) },
    internalLoggingFactory,
    ) {
    companion object {
        fun android(): HttpConfig.Builder? {
            return HttpConfig.builder()
                .setClientConfig(AndroidHttpConfig())
        }
    }
}

class AndroidLokiHttpClient(config: HttpConfig?): Loki4jHttpClient {
    init {
        Log.d(TAG, "AndroidLokiHttpClient($config)")

    }
    override fun close() {
        Log.d(TAG, "close()")
    }

    override fun getConfig(): HttpConfig {
        Log.d(TAG, "getConfig()")
        return config
    }

    override fun send(batch: ByteBuffer): LokiResponse {
        Log.d(TAG, "send()")
        return LokiResponse(200, "")
    }
    companion object {
        private val TAG = AndroidLokiHttpClient::class.java.simpleName
    }
}

class AndroidLokiSender: AbstractHttpSender() {
    override fun getHttpClientFactory(): Function<HttpConfig?, Loki4jHttpClient?> {
        return Function { config: HttpConfig? -> AndroidLokiHttpClient(config) }
    }

    override fun getConfig(): HttpConfig.Builder {
        return AndroidPipelineConfig.Companion
            .android()!!
            .fill(this::fillHttpConfig);
    }
}

Here's my logback.xml

<configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="https://tony19.github.io/logback-android/xml"
    xsi:schemaLocation="https://tony19.github.io/logback-android/xml https://cdn.jsdelivr.net/gh/tony19/logback-android/logback.xsd">
    <appender name="LOGCAT" class="ch.qos.logback.classic.android.LogcatAppender">
        <tagEncoder>
            <pattern>%logger{12}</pattern>
        </tagEncoder>
        <encoder>
            <pattern>[%-20thread] %msg</pattern>
        </encoder>
    </appender>

    <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
        <lazy>true</lazy>
        <batchTimeoutMs>1000</batchTimeoutMs>
        <http class="com.revi.kitandroid.util.logger.AndroidLokiSender">
            <url>http://localhost:8000/</url>
        </http>
        <format>
            <label>
                <pattern>app=KIT,level=%level</pattern>
            </label>
            <message>
                <pattern>l=%level h=${HOSTNAME} c=%logger{20} t=%thread | %msg %ex</pattern>
            </message>
            <sortByTime>true</sortByTime>
        </format>
    </appender>

    <appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
        <appender-ref ref="LOKI" />
    </appender>

    <root level="DEBUG">
        <appender-ref ref="LOGCAT" />
        <appender-ref ref="ASYNC" />
    </root>
</configuration>
allburov commented 1 year ago

Dumb me, of course it logs only one time because I added only a single log.info expression :D Never mind, now I see all send() after each logs.

2023-10-27 20:58:26.716 31367-31367 AndroidLokiHttpClient   com.revi.kitandroid                  D  AndroidLokiHttpClient(com.github.loki4j.client.http.HttpConfig@be9258)
2023-10-27 20:58:27.748 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:28.855 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:29.940 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:31.037 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:38.648 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:39.847 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:40.942 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:48.557 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()
2023-10-27 20:58:49.866 31367-31440 AndroidLokiHttpClient   com.revi.kitandroid                  D  send()

We can assume it works perfectly on Android with logback-android (except it has no client, but it's not a problem I hope and I'll do implement it in few days)