search

Azure / azure-sdk-for-java

This repository is for active development of the Azure SDK for Java. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/java/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-java.
MIT License
2.35k stars 1.99k forks source link

Service bus exception when the locale is not UTC. #285

Closed wolfkang closed 11 years ago

wolfkang commented 11 years ago

Dev estimate: 3 Test estimate: 1

Hi. I'm from Korea and the time locale of my computer is not set as UTC. I got the exception like this when receiving a message from the service bus queue.

org.codehaus.jackson.map.JsonMappingException: Can not construct instance of java.util.Date from String value 'Tue, 05 Mar 2013 10:30:52 GMT': not a valid representation (error: Can not parse date "Tue, 05 Mar 2013 10:30:52 GMT": not compatible with any of standard forms ("yyyy-MM-dd'T'HH:mm:ss.SSSZ", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", "EEE, dd MMM yyyy HH:mm:ss zzz", "yyyy-MM-dd")) at [Source: [B@3e68cd21; line: 1, column: 97](through reference chain: com.microsoft.windowsazure.services.serviceBus.implementation.BrokerProperties["LockedUntilUtc"]) at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163) at org.codehaus.jackson.map.deser.StdDeserializationContext.weirdStringException(StdDeserializationContext.java:243) at org.codehaus.jackson.map.deser.std.StdDeserializer._parseDate(StdDeserializer.java:577) at org.codehaus.jackson.map.deser.std.DateDeserializer.deserialize(DateDeserializer.java:28) at org.codehaus.jackson.map.deser.std.DateDeserializer.deserialize(DateDeserializer.java:19) at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299) at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414) at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697) at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580) at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732) at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1935) at com.microsoft.windowsazure.services.serviceBus.implementation.BrokerPropertiesMapper.fromString(BrokerPropertiesMapper.java:31) at com.microsoft.windowsazure.services.serviceBus.implementation.ServiceBusRestProxy.receiveMessage(ServiceBusRestProxy.java:187) at com.microsoft.windowsazure.services.serviceBus.implementation.ServiceBusRestProxy.receiveQueueMessage(ServiceBusRestProxy.java:151) at com.microsoft.windowsazure.services.serviceBus.implementation.ServiceBusExceptionProcessor.receiveQueueMessage(ServiceBusExceptionProcessor.java:109)

Thanks.

jcookems commented 11 years ago

Thanks for bringing this to our attention!

As a workaround, to unblock your app, you can set the thread locale to UTC or US locale.

We will investigate the root cause of this issue and try to issue a fix soon.

jcookems commented 11 years ago

I started digging into this issue, and StdDeserializationContext uses StdDateFormat, which uses the standard SimpleDateFormat but does not set its calendar field:

Subclasses should initialize this field to a Calendar appropriate for the Locale associated with this DateFormat.

So, it seems like the fix will need to inject a serializer that uses a locale-invariant calendar.

jcookems commented 11 years ago

I ran the unit tests after setting my Windows Control Panel\Clock, Language, and Region\Language date,time,format settings as Arabic (Saudi Arabic). Here are the failures,along with the test that failed:

It looks like they are all localized to the SB serializer (haha, I crack myself up)

guangyang commented 11 years ago

candidate

guangyang commented 11 years ago

Also see the MSDN forum here. I believe it was started by @wolfkang but there is another person running into this issue.

gcheng commented 11 years ago

the msdn forum post was written by Donghyeok Kang, the same family name different first name. They could be brothers.

gcheng commented 11 years ago

http://fasterxml.github.com/jackson-databind/javadoc/2.0.0/com/fasterxml/jackson/databind/ObjectMapper.html

gcheng commented 11 years ago

In the above document, there is a setLocale method of ObjectMapper, which may be usful. Why it doesn't show up in my Eclipse?

gcheng commented 11 years ago

we do not reference jackson explicitly in hte code, how do we figure out which version of jackson that we are using.

gcheng commented 11 years ago

we use 1.9.2 ...

gcheng commented 11 years ago

it seems in 1.9.4, the setLocale method still hasn't been implemented. http://jackson.codehaus.org/1.9.4/javadoc/

gcheng commented 11 years ago

another approach appears to be more promising http://stackoverflow.com/questions/5591967/jackson-date-deserialization

gcheng commented 11 years ago

here is the official guide. http://wiki.fasterxml.com/JacksonFAQDateHandling

gcheng commented 11 years ago

next question, what date format we should use...

gcheng commented 11 years ago

here is a sample date Thu, 21 Mar 2013 01:01:11 GMT

gcheng commented 11 years ago

EEE, dd MMM yyyy HH:mm:ss zzz

gcheng commented 11 years ago

Eee, dd Mmm yyyy hh:mm:ss ZZZ

jcookems commented 11 years ago

That looks like RFC-1123

gcheng commented 11 years ago

I end up using EEE, dd MMM yyyy HH:mm:ss zzz, which is identical to a format in the original message saying "doesn't work". Yes, I believe it is RFC1123

gcheng commented 11 years ago

next step, change the locale of my machine to korea, then run the unit tests again.

gcheng commented 11 years ago

doesn't fix the problem on my korea machine ... :(

gcheng commented 11 years ago

java.lang.IllegalArgumentException: org.codehaus.jackson.map.JsonMappingException: Can not construct instance of java.util.Date from String value 'Thu, 21 Mar 2013 18:03:43 GMT': not a valid representation (error: Unparseable date: "Thu, 21 Mar 2013 18:03:43 GMT") at [Source: [B@1218cdf2; line: 1, column: 97](through reference chain: com.microsoft.windowsazure.services.serviceBus.implementation.BrokerProperties["LockedUntilUtc"]) at com.microsoft.windowsazure.services.serviceBus.implementation.BrokerPropertiesMapper.fromString(BrokerPropertiesMapper.java:41) at com.microsoft.windowsazure.services.serviceBus.implementation.ServiceBusRestProxy.receiveMessage(ServiceBusRestProxy.java:188) at com.microsoft.windowsazure.services.serviceBus.implementation.ServiceBusRestProxy.receiveQueueMessage(ServiceBusRestProxy.java:152) at com.microsoft.windowsazure.services.serviceBus.implementation.ServiceBusExceptionProcessor.receiveQueueMessage(ServiceBusExceptionProcessor.java:109) at com.microsoft.windowsazure.services.serviceBus.ServiceBusIntegrationTest.peekLockMessageWorks(ServiceBusIntegrationTest.java:183) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) Caused by: org.codehaus.jackson.map.JsonMappingException: Can not construct instance of java.util.Date from String value 'Thu, 21 Mar 2013 18:03:43 GMT': not a valid representation (error: Unparseable date: "Thu, 21 Mar 2013 18:03:43 GMT") at [Source: [B@1218cdf2; line: 1, column: 97](through reference chain: com.microsoft.windowsazure.services.serviceBus.implementation.BrokerProperties["LockedUntilUtc"]) at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163) at org.codehaus.jackson.map.deser.StdDeserializationContext.weirdStringException(StdDeserializationContext.java:243) at org.codehaus.jackson.map.deser.std.StdDeserializer._parseDate(StdDeserializer.java:553) at org.codehaus.jackson.map.deser.std.DateDeserializer.deserialize(DateDeserializer.java:28) at org.codehaus.jackson.map.deser.std.DateDeserializer.deserialize(DateDeserializer.java:19) at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299) at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414) at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697) at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580) at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2723) at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1926) at com.microsoft.windowsazure.services.serviceBus.implementation.BrokerPropertiesMapper.fromString(BrokerPropertiesMapper.java:35) ... 29 more

gcheng commented 11 years ago

http://stackoverflow.com/questions/10872571/cannot-get-a-mesage-on-the-service-bus-queue

gcheng commented 11 years ago

I can repro the problem in unit test now, do not need to keep my machine in korean locale all the time.

gcheng commented 11 years ago

strange, according to the documentation, it should support RFC-1123 whatever http://jackson.codehaus.org/1.1.2/javadoc/org/codehaus/jackson/map/util/StdDateFormat.html

gcheng commented 11 years ago

it turns out to be a problem of the underlying library.

http://jira.codehaus.org/browse/JACKSON-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

gcheng commented 11 years ago

The above record shows that this issue has been fixed in version 2.0 of jackson, time for an upgrade?

gcheng commented 11 years ago

I fixed it on our side anyway.

jcookems commented 11 years ago

Set locale of my machine to Chinese (Simplified, China) and the Service Bus tests. 

supportedFormatsHaveExpectedJavaTypes(com.microsoft.windowsazure.services.serviceBus.implementation.CustomPropertiesMapperTest)
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
    at com.microsoft.windowsazure.services.serviceBus.implementation.CustomPropertiesMapperTest.supportedFormatsHaveExpectedJavaTypes(CustomPropertiesMapperTest.java:119)
...

supportedJavaTypesHaveExpectedRepresentations(com.microsoft.windowsazure.services.serviceBus.implementation.CustomPropertiesMapperTest)
org.junit.ComparisonFailure: expected:<"[Thu, 14 Oct] 1971 12:34:56 GMT"> but was:<"[星期四, 14 十月] 1971 12:34:56 GMT">
    at org.junit.Assert.assertEquals(Assert.java:125)
    at org.junit.Assert.assertEquals(Assert.java:147)
    at com.microsoft.windowsazure.services.serviceBus.implementation.CustomPropertiesMapperTest.supportedJavaTypesHaveExpectedRepresentations(CustomPropertiesMapperTest.java:83)
...

Same errors when using Arabic (Saudi Arabia)

supportedFormatsHaveExpectedJavaTypes(com.microsoft.windowsazure.services.serviceBus.implementation.CustomPropertiesMapperTest)
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
...
supportedJavaTypesHaveExpectedRepresentations(com.microsoft.windowsazure.services.serviceBus.implementation.CustomPropertiesMapperTest)
org.junit.ComparisonFailure: expected:<"[Thu, 14 Oct] 1971 12:34:56 GMT"> but was:<"[خ, 14 أكت] 1971 12:34:56 GMT">
...
gcheng commented 11 years ago

https://github.com/gcheng/azure-sdk-for-java/pull/28

jcookems commented 11 years ago

Looks good!