Adding optional meta data to Timber logs

[naveenOnarayanan]

We are currently using Timber for all our logging purposes and want to send them up to our backend for monitoring from our production app. However in order to do so we need the ability to pass in optional metadata to each log statement. The extra metadata can be of any type, such as a Map.

Currently using Timber, we don’t have a way to pass in extra information besides Object[] args and overriding the formatMessage() to handle them. The issue with this is that we miss out on type safety and can’t enforce the required schema or make it explicit that the arguments passed in is metadata vs formatting args.

Our current implementation is to add a thin layer on top of Timber which enforces various schema requirements for the consumers ensuring that the data that gets funneled down is compatible for our transport system to send it up to our servers. We’d like to propose upstreaming this to enhance Timber’s API. We’re also interested in getting others thoughts on if it can already be solved by Timber’s current API or why it should remain separate.

Using our current use case:

1. Errors and Warnings should have a unique identifier
   • solved by using Timber.tag() + Lint
2. Errors and Warnings must have a stacktrace
   • can use linting to enforce that they provide an Exception
3. Optional meta data of type Map<String, String> can be passed in with each log
   • This is lacking as we don’t have a clear way of enforcing type safety on these optional args.
   • Furthermore, by overriding formattingMessage() we will be unable to distinguish between arguments that should be formatted into the string vs. arguments that are to be sent up to our servers.

Suggested implementation:

Timber can support generics when logging in order to define a specific log format that the downstream trees can listen and consume.

i.e. e(T logMeta); -- where the generic T is a instance of LogMeta (can implement a common interface defined in Timber i.e. LogMeta)

public interface LogMeta { }

Using this interface the consumer can create a log meta-data class

public class MyLogMeta implements LogMeta {
    public MyLogMeta(String msg, Throwable tr, Map<String, String> extra);
}

then on the client the following call can be made

Timber.e(new MyLogMeta("message",new Exception(),new HashMap<String, String>()));

we can create a tree with the following method signature to intercept the log statement

public class MyLogMetaTree implements Timber.Tree {
    ...
    protected void log(MyLogMeta meta) {
        // do something
    }
}

This way the consumer can define the metadata it expects for one or more of its trees downstream.

[kkovach]

I was considering something similar to this. Thoughts or advice @JakeWharton? Don't use Timber for something like this?

[HughIngram]

I'm working on a pull request to achieve this functionality. However I'm struggling to find a way to implement this in a way that:

• Is generic (using java generics to allow any type of object to be passed in the metadata field)
• Maintains type safety
• Does not break the API.

The best solution I can come up with assumes that the user wants to pass metadata in the format of Map<String, Object> - e.g.:

public final class Timber {
    public static void v(@NonNls String message, Map<String, Object> metaData, Object... args) {
      ...
    }
    ....
}

However I do not feel this would be generic enough for the PR to be accepted.

For structured logging, it looks like HashMap<String, Object> is appropriate for these popular logging systems:

Firebase Mint

edit: I've implemented this, here is the relevant diff.

@naveenOnarayanan @kkovach would this work for your use cases?

[HughIngram]

@JakeWharton @jrodbx

What are your thoughts on this proposal? I know this would be genuinely useful in my use case, but is it simply beyond the scope of Timber's small, extensible API?

[hannesa2]

Here is an other proposal https://github.com/JakeWharton/timber/issues/384 I'm open to apply any changes to get a merge soon

[hannesa2]

@JakeWharton I would love to see some progress here, otherwise Timber is dead in the company where I'm working for.

Now you have two proposals #303 and #384 and they result in a incompatible change with existing. What's about to go ahead with one of them, either in a 5.x branch or even better, on master and tag it as 5.0.alpha1 ?

[JakeWharton]

It's not my top priority. First is releasing the Kotlin conversion as fully source and binary compatible and then releasing it as multiplatform again as source and binary compatible.

[kaushikgopal]

Adding my use case here from issue 392. Initially suggested doubling up on vararg but given the conflation with String formatting I'm starting to realize, implementing it in that specific way might not be the best. That being said, i think the use case is still pretty strong:

The idea is similar to what folks have mentioned here. With many logging services, there's need for adding meta-data at a Log level (vs the global user level). Providing a mechanism of doing this via Timber would be awesome. Specifically:

We use services like DataDog & LogDna to send a sampled logs. Typically we associate user_id, app_version etc. at the log level, which is possible today while planting the branch, but there are times where we want to associate "meta data" per log. For example: when doing certain specific network logging, it would be nice to associate an x-request-id as metadata (facet in DataDog land or just properties in LogDna). Without forking/over ridding Timber there's no way to do this, without doing something scary like a regex match reverse-parse (which doesn't scale for obv. reasons).

[kimrtaylor]

+1 For this. We use Rollbar for our logging, which takes metadata in the form of HashMap<String, Object>, but we can't supply it through Timber. It's not an option to append the metadata to the log message either, as it would disrupt the automatic merging and grouping of logs by Rollbar.

[carterhudson]

@JakeWharton Any movement on this? It would be very useful for our logging efforts. Any thoughts on the PR?

[emouawad]

+1

[rrbrambley]

+1

[l3ender]

We are exploring using Timber in our app and found this same need (we want to send log events to an API).

@JakeWharton - is a submission like https://github.com/JakeWharton/timber/pull/303 something you'd accept? Any guidance would be appreciated.