Problems with UTF-8 characters

[nymoen]

Hi. We have tried the Twitter for GeoEvent 10.2.x version. It works, but the tweets we create does not shows the UTF-8 characters. We have tried to figure out this problem, but we do not find the source of the problem. Here you have an example of a tweet created by GeoEvent:

Any idea what we can do to solve this?

Regards Kjetil N

[rsunderman]

Thank you @nymoen - I've exported this issue and we will investigate it.

[nymoen]

@rsunderman : Have you been able to look into this?

[cris6097]

@nymoen I'm unable to reproduce this issue. I was able to connect TCP Input (using Simulator with converted file to UTF-8) to Send a Tweet.

Contents of file used in simulator:

ASCII 169: ©
ASCII 224: à
ASCII 232: è
ASCII 236: ì
ASCII 242: ò
ASCII 249: ù

See below for snapshot of tweets using Send Tweet.

[nymoen]

@cris6097 : In my case I just have a text in the Twitter Output message. The text in the message is: Dette er en test med bokstavene: æ,ø å: Æ,Ø,Å

PS. We do use the 10.2.2 version of GeoEvent.

[nymoen]

@cris6097 : I did one more test using the characters you used in your test (©, à, è, ì, ò, ù): Here is the result in the tweet:

[cris6097]

@nymoen Thank you for the clarification. It seems this issue is in version 10.2.2 and 10.3.0 but it appears to be fixed in 10.3.1.

[nymoen]

@cris6097 So this is an issue in GeoEvent or in the Twitter Connector? We do have a client using this and they are not happy to send tweets with funny characters.

[cris6097]

@nymoen I believe its an issue with GeoEvent. Could you wait for 10.3.1 public release which is ~April 2015? If not, try reaching out to your sales rep for an early adopter program, which by the way I'm unaware if such programs exists.

[nymoen]

@cris6097 If this is an issue with GeoEvent, then the other outputs should give me funny characters also, correct? If I write to a file or send an email, I should also get the funny characters?

PS! I am not sure we can upgrade to 10.3.1 that easy, because this is a production site running 10.2.2.

[rsunderman]

I tested the 10.2.2 release as well as the 10.3.0 release and our 10.3.1 release candidate using a Tcp/Text input to send event data to a CSV (system file), a JSON (system file), a GeoEvent Cache, and an e-mail output. The issue was not observed for any of these outputs on any of the releases tested. I do not think you will find that other outputs behave like the twitter-out.

The e-mail outbound connector is probably the closest to your use case. That outbound connector uses the same out-of-the-box MessageFormatter adapter as the twitter-out outbound connector.

Because the same 10.3.x binary JAR file was loaded into both the 10.3 release and 10.3.1 release candidate of GeoEvent - we are assuming the issue is somewhere in the core GeoEvent product rather than the Twitter implementation. However, as I mentioned above, the same out-of-the-box adapter is used for both twitter-out and esri-out-message-email-send. The e-mail output is not exhibiting the behavior on any product release. I cannot explain why we are seeing the issue we are seeing with the twitter-out.

I understand that you are working with a production system which is leveraging the GeoEvent 10.2.2 release, so upgrading will not be easy. Please contact Esri Technical Support and submit this issue as a bug against the 10.2.2 release if you expect that your only path forward will be to request a patch for the 10.2.2 product. Please understand that issue investigation and generation of a patch will have to be prioritized within the product development backlog - and the 10.3.1 release will likely be available before a 10.2.2 patch can be developed, tested, and released to you.