Error parsing events due to GEO location

[InlineAsm]

Hello,

I have a self-hosted instance of exceptionless running. I noticed several unprocessed json files in the storage/q folder and started investigating the cause. According to the exceptionless log there is an error parsing a field. The exception is:

2016-02-09 09:17:00.0871  136 ERROR EventPostsJob Error while processing event post "q\a204f4f23850409e85c8b515d7eb4c86.json": MapperParsingException[failed to parse]; nested: NumberFormatException[For input string: "3667,4,9"];  System.ApplicationException: MapperParsingException[failed to parse]; nested: NumberFormatException[For input string: "3667,4,9"]; 
   at Foundatio.Elasticsearch.Repositories.ElasticRepositoryBase`1.<AddAsync>d__4.MoveNext() in C:\projects\foundatio-repositories\src\Elasticsearch\Repositories\ElasticRepositoryBase.cs:line 53
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Exceptionless.Core.Pipeline.SaveEventAction.<ProcessBatchAsync>d__4.MoveNext() in C:\projects\exceptionless\Source\Core\Pipeline\040_SaveEventAction.cs:line 22

Then I started to look into the elasticsearch logs and I got this snippet of log:

[2016-02-09 09:17:00,087][DEBUG][action.bulk              ] [ex-es01] [events-v1-201602][1] failed to execute bulk item (index) index {[events-v1-201602][events][56b9a07c7957130410333b16], source[{"id":"56b9a07c7957130410333b16","organization_id":"5508472bfcce2921ecec573d","project_id":"55084d9dfcce2a21ecb3d5d4","stack_id":"56698a3679571410f00c975b","is_first_occurrence":false,"is_fixed":false,"is_hidden":false,"created_utc":"2016-02-09T08:17:00.0714496Z","idx":{"session-r":"56b997e279571311a8efc20c"},"type":"error","date":"2016-02-09T09:15:37.8804443+01:00","message":"Er zijn één of meer fouten opgetreden.","geo":"52,3667,4,9","data":{"@version":"1.3.1.0","@error":{"message":"Er zijn één of meer fouten opgetreden.","type":"System.AggregateException","data":{"@target":{"ExceptionType":"System.Net.WebException"}},"inner":{"message":"Er is een fout opgetreden bij het verzenden van het aanvraag.","type":"System.Net.Http.HttpRequestException","inner":{"message":"Kan geen verbinding met de externe server maken","type":"System.Net.WebException"}}},"@environment":{"processor_count":8,"total_physical_memory":34031341568,"available_physical_memory":25194328064,"command_line":"\"C:\\Users\\Ilyass\\AppData\\Local\\Programs\\ReflexSystems\\Reflex NG (Acceptation)\\ReflexNG.DesktopClient.exe\" ","process_name":"C:\\Users\\Ilyass\\AppData\\Local\\Programs\\ReflexSystems\\Reflex NG (Acceptation)\\ReflexNG.DesktopClient.exe","process_id":"9496","process_memory_size":179843072,"thread_id":"8628","architecture":"x64","o_s_name":"Microsoft Windows 10 Pro","o_s_version":"6.2.9200.0","ip_address":"127.0.0.1","machine_name":"WKS055","install_id":"3edc1ce15d874c1ba5080afcda8f622b","runtime_version":"4.0.30319.42000","data":{"AppDomainName":"ReflexNG.DesktopClient.exe"}},"@user":{"identity":"Ilyass"},"@location":{"country":"NL"},"@ref:session":"56b997e279571311a8efc20c"}}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:565)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
    at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:492)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:409)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:148)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.performOnPrimary(TransportShardReplicationOperationAction.java:574)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1.doRun(TransportShardReplicationOperationAction.java:440)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException: For input string: "3667,4,9"
    at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
    at sun.misc.FloatingDecimal.parseDouble(Unknown Source)
    at java.lang.Double.parseDouble(Unknown Source)
    at org.elasticsearch.common.geo.GeoPoint.resetFromString(GeoPoint.java:69)
    at org.elasticsearch.index.mapper.geo.GeoPointFieldMapper.parsePointFromString(GeoPointFieldMapper.java:551)
    at org.elasticsearch.index.mapper.geo.GeoPointFieldMapper.parse(GeoPointFieldMapper.java:527)
    at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
    at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
    ... 10 more

As far as I can determine the parsing of this particular part of json "geo":"52,3667,4,9" seems to cause the issue. Is this a bug or is this a setting gone wrong somewhere?

Thank you for your time,

Ivar

[niemyjski]

What local are you running in? I think it's an issue with the number format of the client or server? It should be a lat,long

[InlineAsm]

The server is running on US locale. The clients with the problem are running on NL-nl.

I looked at the client source and the problem is (I think) in the eventbuilder.cs:

public EventBuilder SetGeo(double latitude, double longitude) {
            Target.Geo = latitude + "," + longitude;

[ejsmith]

Hmm, yeah, guess we need to make sure that we specify the string format instead of letting it use the default.

[niemyjski]

@InlineAsm I have a fix for both server and client that fixes this issue. I'll try and getting a new client release out soon: https://github.com/exceptionless/Exceptionless.Net/commit/0a4e07ee458c73efbc213a615be756f37b8adfce

[niemyjski]

The server side is minor and you won't have to update. It's mainly the client that is broken under different cultures and needs the update.

[InlineAsm]

Thank you very much for the fast response and fix!

[InlineAsm]

I'm afraid the fix didn't fix it. You need to specify the CultureInfo.InvariantCulture in the ToString method to properly format the decimal with a point regardless of the culture. I missed it earlier when I was reviewing your change.

[ejsmith]

So even though we are specifying a custom format, it will still change the decimal point to a comma in some locales? Didn't think it would automatically change a custom format string.

[InlineAsm]

Yes, the format alone is not enough. I created a pullrequest on the exceptionless.net repo that fixes it.

Ps. You might want to fix the tostring on the server side too.

[niemyjski]

Aw thanks!

[niemyjski]

I'm fixing this server side as we speak.

[niemyjski]

Thanks again! Can you please let me know if you run into any issues. I can push a new client build soon. If you need a new server build you can grab it via our build artifacts or let me know.

[InlineAsm]

Thanks! I need the client update most as I now have to delete around 8000 json files every 2 days because the clients are sending invalid data. I don't think that I need to update the server. It's running on US-us.

[niemyjski]

Pushing the latest release now (1565) to nuget,

[InlineAsm]

Thanks, I updated the client. I would be interested in the server artifacts too since I checked again and although the server runs an english version of Windows the locale is dutch there too... I couldn't find artifacts for the server build.

[niemyjski]

Looks like we aren't pushing artifacts as part of the build. Can you let me know if the current server works under the different locale or if you still have issues. If you have issues is there any chance you could download the master repo and build it or I'm going to need to push a new minor release and I wanted to wait a week or two to get a few more things into it.

[InlineAsm]

I built the server myself and it works now with the Dutch locale. No need to rush a release. Thank you very much.

[ejsmith]

Awesome, thank you for the feedback!