search

influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.66k stars 3.54k forks source link

Stored timestamps in `_value` interpreted as strings in Flux query #14575

Open affo opened 5 years ago

affo commented 5 years ago

When applying a conversion from time to int or uint in e2e tests we get the following error:

--- FAIL: TestFluxEndToEnd/to_int.flux (0.23s)
        end_to_end_test.go:170: failed to read meta data: failed to evaluate map function: strconv.ParseInt: parsing "1970-01-01T00:00:00.000000000Z": invalid syntax

--- FAIL: TestFluxEndToEnd/to_uint.flux (0.22s)
        end_to_end_test.go:162: failed to evaluate map function: strconv.ParseUint: parsing "1970-01-01T00:00:00.000000000Z": invalid syntax

The table that causes the error is this one:

#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,string,string
#group,false,false,false,false,true,true
#default,_result,,,,,
,result,table,_time,_value,_field,_measurement
,,4,2018-05-22T19:53:40Z,1970-01-01T00:00:00Z,k4,m
,,4,2018-05-22T19:53:41Z,2018-05-22T19:53:26.033Z,k4,m
,,4,2018-05-22T19:53:42Z,2018-05-22T19:53:26.033066Z,k4,m
,,4,2018-05-22T19:53:43Z,2018-05-22T19:00:00+01:00,k4,m

The error happens here https://github.com/influxdata/flux/blob/52bf9ec1b0e1b0be208f111b10d2d4638ea8d23c/stdlib/universe/typeconv.go#L183 (for ints).

It seems that the value gets erroneously interpreted as a string instead of a timestamp.

The problem doesn't happen in Flux, so I suspect that this has something to do with the fact that, in InfluxDB, we store data using a to transformation before querying. Maybe to has some problems in storing _value as a dateTime:RFC3339.

affo commented 5 years ago

I discovered that InfluxDB cannot store timestamps as values. Those test must remain skipped.

We can take actions to ensure coverage in InfluxDB, but we should make those changes in Flux.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

affo commented 4 years ago

This seems still relevant: https://github.com/influxdata/influxdb/blob/master/query/stdlib/testing/testing.go#L89

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.