Issue with SQLException

[macosta60]

Hi,

With the last version of elasticsearch (2.*) all sql exception are not logged.

I search a little and when i clear all "Exception.Data" the exception is logged.

here a sample of data serialized with newtonsoft : { "HelpLink.ProdName":"Microsoft SQL Server", "HelpLink.ProdVer":"09.00.5000", "HelpLink.EvtSrc":"MSSQLServer", "HelpLink.EvtID":"2812", "HelpLink.BaseHelpUrl":"http://go.microsoft.com/fwlink", "HelpLink.LinkId":"20476" }

The key of dictionnary contains '.'

I make an extention method that clear all data befor log but it's not a good solution.

cordially

Michael.

[jptoto]

@macosta60 I'm guessing this is a serialization issue. Can you try using one of these two debugging techniques https://github.com/jptoto/log4net.ElasticSearch/wiki/04-Debugging-log4net.ElasticSearch and tell me if it shows an error and what the error is?

[macosta60]

here the result log after run TechnicalLogger.Error(ex.Message,ex);

log4net: log4net assembly [log4net, Version=1.2.15.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a]. Loaded from [C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\e.boby.web.extranet\fa6d78ae\ed5fc807\assembly\dl3\8b6f2fb6\563212ef_8969d101\log4net.dll]. (.NET Runtime [4.0.30319.42000] on Microsoft Windows NT 6.1.7601 Service Pack 1) log4net: defaultRepositoryType [log4net.Repository.Hierarchy.Hierarchy] log4net: Creating repository for assembly [DotNetOpenAuth.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=2780ccd10d57b246] log4net: Assembly [DotNetOpenAuth.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=2780ccd10d57b246] Loaded From [C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\e.boby.web.extranet\fa6d78ae\ed5fc807\assembly\dl3\3f70f8c1\4065e7a6_58c5d001\DotNetOpenAuth.Core.dll] log4net: Assembly [DotNetOpenAuth.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=2780ccd10d57b246] does not have a RepositoryAttribute specified. log4net: Assembly [DotNetOpenAuth.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=2780ccd10d57b246] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: Creating repository [log4net-default-repository] using type [log4net.Repository.Hierarchy.Hierarchy] log4net: Creating repository for assembly [E.Boby.Web.Extranet, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] log4net: Assembly [E.Boby.Web.Extranet, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] Loaded From [C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\e.boby.web.extranet\fa6d78ae\ed5fc807\assembly\dl3\215f6b3a\aac67a16_7070d101\E.Boby.Web.Extranet.dll] log4net: Assembly [E.Boby.Web.Extranet, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] does not have a RepositoryAttribute specified. log4net: Assembly [E.Boby.Web.Extranet, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: repository [log4net-default-repository] already exists, using repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: configuring repository [log4net-default-repository] using .config file section log4net: Application config file is [E:\TFS\Entertainment\Boby\Dev\E.Boby\Web\E.Boby.Web.Extranet\web.config] log4net: Configuring Repository [log4net-default-repository] log4net: Configuration update mode [Merge]. log4net: Logger [root] Level string is [ALL]. log4net: Logger [root] level set to [name="ALL",value=-2147483648]. log4net: Loading Appender [ElasticSearchAppender] type: [log4net.ElasticSearch.ElasticSearchAppender, log4net.ElasticSearch] log4net: Setting Property [ConnectionString] to String value [Server=****;Index=logboby;Port=9200;rolling=false] log4net: Setting Property [Lossy] to Boolean value [False] log4net: Setting Property [Threshold] to Level value [ERROR] log4net: Setting Property [Evaluator] to object [log4net.Core.LevelEvaluator] log4net: Setting Property [BufferSize] to Int32 value [100] log4net: Created Appender [ElasticSearchAppender] log4net: Adding appender named [ElasticSearchAppender] to logger [root]. log4net: Hierarchy Threshold [] log4net: Creating repository for assembly [E.Boby.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] log4net: Assembly [E.Boby.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] Loaded From [C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\e.boby.web.extranet\fa6d78ae\ed5fc807\assembly\dl3\5e4dd840\c3b2a3f5_eb6fd101\E.Boby.Common.dll] log4net: Assembly [E.Boby.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] does not have a RepositoryAttribute specified. log4net: Assembly [E.Boby.Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null] using repository [log4net-default-repository] and repository type [log4net.Repository.Hierarchy.Hierarchy] log4net: repository [log4net-default-repository] already exists, using repository type [log4net.Repository.Hierarchy.Hierarchy]

I don't see any error

[dorinoltean]

The problem is that elastic search does not accept fields that contain '.'

The following line is the exception message from elasticsearch logs.

MapperParsingException[Field name [HelpLink.BaseHelpUrl] cannot contain '.']

[jptoto]

@dorinoltean nice catch, this is correct. @macosta60 I may be able to parse the field data going in and replace any "." with a legal character.

[zaytsevvla]

what about this issue? And IDictionary is a <object, object>, i think it is unsafe to use it. Maybe you want Data = ex.Data.Keys.Cast<object>().ToDictionary(x => x.ToString().Replace('.', '_'), x => ex.Data[x].ToString()); in JsonSerializableException?

[zaytsevvla]

And the same problem exists if key with dot exists in messageObject

log.Debug(new Dictionary<string, string>{ {"a.b","x"} });

maybe we need some drawback?

[jptoto]

The problem is that Elasticsearch can't index messages with the dot . and I'm trying to decide how much I should be trying to scrub that data before it reaches ES or if I should let it fail because it's a function of ES.

[zaytsevvla]

I understand the source of problem. My solution is: for logger properties and exception data replace dots with underscore. For message object: if we can't serialyze it or if we receive 500 HTTP code, then set messageObject = new object() and try again.

[zaytsevvla]

So i made PR with first half and work on second.

[zaytsevvla]

I see the problem with other half: serialization is in elasticsearch client and it send it in chunks. So it increase the problem: one bad message can drop full chunk!

[zaytsevvla]

So maybe you merge my PR and update package now? And problem with messageObject we will solve another day?