Couldn't send dynamic field

[morad3741]

Im trying to send dynamic objects using slf4j fluent api and log4j2, for example: logger.atError().addKeyValue("host_name", "33333").addKeyValue("test","value").log("11111111111111111"); This is what I see in Kibana:

{
  "_index": "XXXX",
  "_type": "_doc",
  "_id": "LKN_9nMB8k09b5T7Y1j1",
  "_version": 1,
  "_score": null,
  "_source": {
    "timeMillis": 1597568559213,
    "loggerName": "XXXX",
    "level": "ERROR",
    "message": "host_name=33333 test=value 11111111111111111",
    "thread": "http-nio-7080-exec-2"
  },
  "fields": {
    "timeMillis": [
      "2020-08-16T09:02:39.213Z"
    ]
  },
  "sort": [
    1597568559213
  ]
}

Here is my log4j2.xml appender:

<Elasticsearch name="elasticsearch">
            <IndexName indexName="xxx"/>
            <ThresholdFilter level="DEBUG" onMatch="ACCEPT"/>
            <JacksonJsonLayout>
                <PooledItemSourceFactory poolName="itemPool" itemSizeInBytes="1024" initialPoolSize="3000"/>
            </JacksonJsonLayout>
            <AsyncBatchDelivery batchSize="1" deliveryInterval="2" >
                <IndexTemplate name="xxx" path="classpath:indexTemplate.json" />
                <HCHttp serverUris="http://xxx:9200">
                    <PooledItemSourceFactory poolName="batchPool" itemSizeInBytes="1024000" initialPoolSize="3"/>
                </HCHttp>
            </AsyncBatchDelivery>
        </Elasticsearch>

Can you tell me what im doing wrong?

[morad3741]

I tried to add VirtualProperty with no luck:

 <Elasticsearch name="elasticsearch">
            <IndexName indexName="XXX"/>
            <ThresholdFilter level="DEBUG" onMatch="ACCEPT"/>
            <JacksonJsonLayout>
                <NonEmptyFilter/>
                <VirtualProperty name="field1" value="$${ctx:ctxVariable:-}" dynamic="true" />
                <PooledItemSourceFactory poolName="itemPool" itemSizeInBytes="1024" initialPoolSize="3000"/>
            </JacksonJsonLayout>
            <AsyncBatchDelivery batchSize="1" deliveryInterval="2" >
                <IndexTemplate name="XXXX" path="classpath:indexTemplate.json" />
                <HCHttp serverUris="http://XXXX:9200">
                    <PooledItemSourceFactory poolName="batchPool" itemSizeInBytes="1024000" initialPoolSize="3"/>
                </HCHttp>
            </AsyncBatchDelivery>
        </Elasticsearch>

and changed the logger to log: logger.atError().addKeyValue("field1", authenticationRequest).addKeyValue("test","value").log("11111111111111111"); and still field1 is not present as a seperated field

[rfoltyns]

Hi @morad3741

If you're trying to resolve $${ctx:ctxVariable:-}, then ctxVariable is the one that must be resolvable within LogEvent.contextData. field1 is just it's name in the output document.

Try this:

logger.atError().addKeyValue("ctxVariable", "33333")

[morad3741]

Hi, Im trying to add new field to the JSON dynamically, Just like it shown in the next guide: logback logstash guide in their example: log.info("Order saved", kv("orderId", orderId), kv("status", status)); will create "orederId" and "status" node in the JSON, Im looking for a way to do the same in log4j.

I tried: logger.atError().addKeyValue("ctxVariable", "33333") it didnt added anything to the JSON,

Thanks for your help

[rfoltyns]

My mistake. I assumed that SLF4j2 .addKeyValue works like MDC. It turns out that it doesn't.

According to SLF4J Manual, addKeyValue will just help you log like this:

int newT = 15;
int oldT = 16;

// using classical API
logger.debug("oldT={} newT={} Temperature changed.", newT, oldT);

// using fluent API
logger.atDebug().addKeyValue("oldT", oldT).addKeyValue("newT", newT).log("Temperature changed.");

to get the same output as DefaultLoggingEventBuilder#mergeMarkersAndKeyValuePairs in slf4j-api:2.0.0-alpha1 will just prepend the keys as it used to in old versions.

In order to get the behaviour you expect, you need MDC:

MDC.put("ctxVariable", "someValue");
logger.atError().log("11111111111111111");
MDC.remove("ctxVariable");

It will work only with synchronous loggers.

If you need to use asynchronous loggers, you'll can get MDC to work as expected by overriding LogEvent.contextData serialization with Jackson mix-in as described here.

[rfoltyns]

Closing due to lack of activity