Loggly: MappingConflict

[njam]

https://cargomedia.loggly.com/search/?terms=LogglyNotifications.type%3AMappingConflict&from=-1d&until=now&source_group=&savedsearchid=208919&newtab=true&tabname=loggly%20MappingConflicts&savedsearchid=208919&isfav=False&searchorder=desc

@vogdb can you investigate? cc @tomaszdurka

See also https://github.com/cargomedia/cm-janus/issues/231

[njam]

I asked them about this problem before. The told me we can find out the specific field that causes the mapping conflict by querying the "events" API endpoint (see https://github.com/cargomedia/cm-janus/issues/231#issuecomment-206181201). Could you try that?

[vogdb]

Well. I queried them but I don't see the field that changed its format. I have doubts that context in this log entry is so deep and complicated that it produces conflicts.

[njam]

@vogdb any news from Loggly? What should we do here?

[vogdb]

Well, no news from Loggly. They wrote to me at 5 of May:

Please let us check with our product manager.

and silence after that. I will ask them again.

[vogdb]

This time Support is not so fast and furious. Currently there is a football in emails. Waiting.

[vogdb]

Well, their final response is vague. I better try to experiment with different log messages and see how it behaves.

The mapping conflict can occur whenever there is a single field with more than one type. For example, if you first log an event with the field "context" as a JSON event, and later log another event where the field "context" is a string it will throw a mapping conflict error. This can occur for any field in the event. While nested JSON structures are usually not a problem, the more complex the event is the more likely it is you'll run into an error simply because you may not be able to guarantee that each field has the same type. It's kind of a generic answer but does that help?

[tomaszdurka]

Wow so this response took them so much time. I think this is generic answer they've sent us first time.

[vogdb]

Well. I had a long discussion with them: Me (Their support reception broke all my nice markdown formatting):

Hi! Previously you sent a nice F.A.Q. on how to resolve MappingConflicts. Thank you. I have a question about it. I have a log entry that continues to give those conflicts permanently. log.warning('some error', {context: context}); Can it happen cause I have another log entry that logs near this problem entry the same context but with different message and level? It looks similar to this: log.debug('for debugging', {context: context}); ... log.warning('some error', {context: context}); So, can a mapping conflict arise due to the fact that context is logged twice by different entries with a subtle changes between them? Or does the conflict happen cause context is a very complicated hash object structure? Here is an example of it:

{
"level": "warning",
"message": "cm-api request failed",
"janus": {
"streamId": "90cae108-4794-4dce-8377-e003ec998c24",
"channelId": "7359bef173dcecd42b989661438ef7a1",
"channelKey": "d9aUHt121z.lUuHE8RKDvg__",
"pluginId": 822575168,
"sessionId": 81132590,
"connectionId": "a0b8879e-7379-42f3-a6b9-1a370b171561"
},
"FB": {
"client": "147655961",
"browserWindow": "5c92085e-9396-40bd-88dc-907b37c87ecf",
"user": 15314233
},
"httpRequest": {
"method": "POST",
"uri": "https://url.url",
"body": {
"method": "subscribe",
"params": [
"FqVWLUzZE8swExCfTn5y",
"{\"sessionId\":\"45a3de6f27a7f6ff7b50b39ac76ca19f\"}",
"d9aUHt121z.lUuHE8RKDvg__",
"7359bef173dcecd42b989661438ef7a1",
"{\"streamChannelType\":269}",
"90cae108-4794-4dce-8377-e003ec998c24",
1462334080.24
]
}
},
"exception": "Internal server error\n",
"hostname": "janus2.fb.com",
"timestamp": "2016-05-04T03:54:46+00:00"
}

And another version of context, so you can see how much they differ

{
"level": "warning",
"message": "cm-api request failed",
"janus": {
"streamId": "c8b066c4-e6a6-4ed4-a90f-5807bdfa7e8d",
"channelId": "175ba63ec3cfce167d3752d76c47ea47",
"channelKey": "n5N8R83COxBUtzGmKOBIpw__",
"pluginId": 345551817,
"sessionId": 3888657615,
"connectionId": "77abdcb6-7635-49a6-b9dc-3c5b4ab5f91d"
},
"FB": {
"client": "144882364",
"browserWindow": "f188b852-cd72-4d7e-835a-65b34ee8c791",
"user": 15300369
},
"httpRequest": {
"method": "POST",
"uri": "https://url/url",
"body": {
"method": "CM_Janus_RpcEndpoints.removeStream",
"params": [
"FqVWLUzZE8swExCfTn5y",
"n5N8R83COxBUtzGmKOBIpw__",
"c8b066c4-e6a6-4ed4-a90f-5807bdfa7e8d"
]
}
},
"exception": "<!DOCTYPE html>\n<html>\n <head>\n <meta charset=\"utf-8\">\n .....",
"hostname": "janus2.fb.com",
"timestamp": "2016-05-02T13:46:09+00:00"
}

They:

Please let us check with our product manager.

Me:

Hi! Any news from your product manager?

They:

We got this feedback. Could you answer the question from our product team: I don't see context as a field in the event that you have sent. Maybe I'm just not following the example. Which field has the conflict?

Me:

'Context' field was just used as an alias to describe the problem cause I don't know the exact field name where this error happens. The events that I have sent do not contain 'context' field but they contain another fields. And those fields produce this problem. Some of those fields have very complicated deep nested structure. So my question was can this complicated deep nested structure of field be the cause of mapping conflict?

They:

Thanks for the information. I will revert back as soon as I hear from our PM.

Me(more than 7 days later):

Hi! This issue is still opened.

They:

The mapping conflict can occur whenever there is a single field with more than one type. For example, if you first log an event with the field "context" as a JSON event, and later log another event where the field "context" is a string it will throw a mapping conflict error. This can occur for any field in the event. While nested JSON structures are usually not a problem, the more complex the event is the more likely it is you'll run into an error simply because you may not be able to guarantee that each field has the same type. It's kind of a generic answer but does that help?

[vogdb]

@njam it should do the trick

[codecov-io]

Current coverage is 90.25%

Merging #259 into master will not change coverage

@@             master       #259   diff @@
==========================================
  Files            34         34          
  Lines          1313       1313          
  Methods         323        323          
  Messages          0          0          
  Branches        141        141          
==========================================
  Hits           1185       1185          
  Misses          128        128          
  Partials          0          0          

Powered by Codecov. Last updated by 076c140...fab3c72

[njam]

lgtm

[vogdb]

According to the logs the conflict is solved successfully.