Unexpected EDM type from the Table Service: Edm.Double

[JeremyCade]

Which service(blob, file, queue, table) does this issue concern?

table storage

Which version of the SDK was used?

WindowsAzure.Storage 9.3.3 and Microsoft.Azure.Cosmos.Table 1.0.1 Microsoft.Azure.Cosmos.Table 1.0.4 Microsoft.Azure.Cosmos.Table 1.0.5

Which platform are you using? (ex: .NET Core 2.1)

.NET Core 2.2

What problem was encountered?

The following exception is encountered when retrieving entities from Table Storage that have properties of type Double that are not set (or are NaN).

Entities were inserted with WindowsAzure.Storage 9.3.3 and retrieved with Microsoft.Azure.Cosmos.Table 1.0.1.

System.AggregateException: One or more errors occurred. (Unexpected EDM type from the Table Service: Edm.Double.) ---> Microsoft.Azure.Cosmos.Table.StorageException: Unexpected EDM type from the Table Service: Edm.Double. ---> System.InvalidOperationException: Unexpected EDM type from the Table Service: Edm.Double.
   at Microsoft.Azure.Cosmos.Table.RestExecutor.TableCommand.TableOperationHttpResponseParsers.ReadSingleItem(JToken token, String& etag)
   at Microsoft.Azure.Cosmos.Table.RestExecutor.TableCommand.TableOperationHttpResponseParsers.ReadQueryResponseUsingJsonParserMetadataAsync(Stream responseStream, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Table.RestExecutor.TableCommand.TableOperationHttpResponseParsers.TableQueryPostProcessGenericAsync[TElement,TQueryType](Stream responseStream, Func`6 resolver, HttpResponseMessage resp, TableRequestOptions options, OperationContext ctx)
   at Microsoft.Azure.Cosmos.Table.RestExecutor.TableCommand.TableQueryRESTCommandGenerator.<>c__DisplayClass0_0.<<GenerateCMDForTableQuery>b__2>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Cosmos.Table.RestExecutor.TableCommand.Executor.ProcessEndOfRequestAsync[T](ExecutionState`1 executionState, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Table.RestExecutor.TableCommand.Executor.ExecuteAsync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)
   --- End of inner exception stack trace ---

How can we reproduce the problem in the simplest way?

see: https://github.com/JeremyCade/bork.azure.tablestorage for an example of this behaviour.

Basic steps to reproduce:

1) Insert an entity into table storage with WindowsAzure.Storage 9.3.3 library. 2) Retrieve entity from table storage with Microsoft.Azure.Cosmos.Table 1.0.1 3) BORK!

Have you found a mitigation/solution?

No.

Aside

Where is the source for Microsoft.Azure.Cosmos.Table?

[tompetk]

Any update on this one?

[tofer]

Having the same issue with Microsoft.Azure.Cosmos.Table 1.0.8. Any update to this? Has anyone found a workaround? My data set is quite large and not practical to edit the data, and I am mostly committed to the new library now. In my case, this exception is being thrown when I have empty strings in Edm.Double fields.

[PaulCheng]

to retrieve double.NaN value, set table.ServiceClient.DefaultRequestOptions.PayloadFormat = TablePayloadFormat.JsonNoMetadata;

[tofer]

Thanks @PaulCheng. At first I thought this worked (as the query was successful), however now all DynamicTableEntity.Properties are String values in the response. Any suggestions?

[tofer]

It seems like there is a bug in the TableOperationHttpResponseParsers.ReadSingleItem static method. While the documentation for Property Types in a JSON Feed lists Edm.Double as a non-required annotation, it does say:

For Edm.Double, the values NaN, Infinity and -Infinity are represented in JSON using type String, and an odata.type annotation is required

However, the ReadSingleItem method does not check for Edm.Double as a possible value for a *@odata.type property despite the documentation stating that it will return that for NaN values, and thus throws an exception in this situation.

It is a possibility this could be patched? Or is there another workaround I could try? I don't actually need the NaN values in my application. Any workaround that would ignore NaN value properties while still reading the rest of the record would be fine.

[tofer]

Another oddity when using TablePayloadFormat.JsonNoMetadata: Date values stored as strings with EdmType.String get converted to UTC.

I queried table storage directly to confirm it was not the service changing the value:

But then in the DynamicTableEntity.Properties["_Date"] is EdmType.String as expected, but its value has been changed to: 2021-01-24T22:45:00.0000000Z

This was tested in Microsoft.Azure.Cosmos.Table 1.0.8

[ghost]

@tofer I'm having same issue and my entities have Infinity value for Property of type Edm.Double. Did you find any work around to ignore such values and proceed with rest of the entity? I updated Microsoft.Azure.Cosmos.Table to version 1.0.8 but its just showing a better exception message compared to v1.0.7. Here is what I see now: Cannot return Double type for a String typed property. StackTrace : at Microsoft.Azure.Cosmos.Table.EntityProperty.EnforceType(EdmType requestedType) at Microsoft.Azure.Cosmos.Table.EntityProperty.get_DoubleValue() at Amber.Data.FlywheelReading..ctor(IDictionary2 props, String partitionKey, String rowKey)`

[tofer]

@skdev123 I never got a viable solution working other than the suggested PayloadFormat = TablePayloadFormat.JsonNoMetadata workaround. Like I stated in one of my replies that makes every EntityProperty have a string value, so I had to do custom mapping and parsing from there to the object I wanted.