Closed LiorBanai closed 1 year ago
The reason is twofold,
The message is not quite ECS compliant since it expands log.origin.*
in to objects where these are actually dotted field names under the log: { }
object.
This should not break our deserialisation routines though so will create a PR to address this.
I think log
fieldset is special enough for us to try and read this expanded form as well.
Opened https://github.com/elastic/ecs-dotnet/pull/318 to address this.
@Mpdreamz Thanks!
Hi The failed json was produced by python ecs-logging==1.1.0 package. Would updating to latest package (2.0.2) will have it producing the correct format ?
cc @basepi @beniwohli I think this might be a 'bug' in ecs-logging-python
See: elastic/beats@e9272ad/libbeat/ecs/log.go#L48
ECS log
fieldset documentation: https://www.elastic.co/guide/en/ecs/current/ecs-log.html#field-log-origin-file-line
Since origin
is not mapped as an object origin.file.name
is a dotted field name under log
.
Compare with e.g log.syslog
which is an object
and thefore facility.code
is a dotted field name under log: { syslog: { ... } }
https://www.elastic.co/guide/en/ecs/current/ecs-log.html#field-log-syslog
@Mpdreamz Sorry for the delay in my response here. If I'm understanding correctly, you're saying that the python agent will expand origin.file.name
into a nested dictionary? I think that diagnosis is correct. How are you dealing with this in dotnet without having a giant list of exceptions?
If I'm understanding correctly, you're saying that the python agent will expand origin.file.name into a nested dictionary?
Yeah and I believe technically this is not a valid ECS _source
structure. E.g beats and ecs-dotnet will produce
{
"log": {
"origin.file.name" : ""
}
}
How are you dealing with this in dotnet without having a giant list of exceptions?
Since everything is backed by types its impossible to produce it any other way: e.g:
We also support setting key values directly, someone can dynamically call:
var ecsDocument = new EcsDocument();
ecsDocument.AssignField("log.origin.file.name", "Test.cs");
For this we generate prop dispatch setters from the ECS specification: https://github.com/elastic/ecs-dotnet/blob/5fe813d2bad337d7efd084a2790968778fe13333/src/Elastic.CommonSchema/PropDispatch.Generated.cs#L1809
Which take care of ensuring the appropriate typed properties are set.
Thanks for the context. I'll think on this.
Hi, When can a new version be released with the support for the python structures?
Hi, I have the following json
I'm trying to parser it using![image](https://github.com/elastic/ecs-dotnet/assets/36262995/de31a0b4-0bce-4505-a00a-1666df555b2c)
EcsDocument.Deserialize(line);
but getting the following error:using versions:
what could be the reason?