NA values in xts of type "character" produce partial write errors

[xanderdunn]

Windows 7 R 3.4.0 prepare_cran at 025f1af2282b80a64a4de0e652618c87271b95b0

I'm writing a single string column to my InfluxDB:

library(tidyquant) # Provides the as_xts function
name = "Name"
this_column <- my_data[name]
this_column$Date = my_data$Date
xts_column <- as_xts(this_column, date_col=Date)
colnames(xts_column) <- c(name)
influx_write(con=con, db="my_db", xts=xts_column, measurement="my_measurement")

str(xts_data) gives:

An 'xts' object on 2017-05-31 11:28:55/2017-05-31 11:31:32 containing:
  Data: chr [1:1233, 1] "NAME1" "NAME2" "NAME3" "NAME4" "NAME5" "NAME6"
...
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr "Name"
  Indexed by objects of class: [POSIXlt,POSIXt] TZ:
  xts Attributes:
 NULL
NULL

When I print(xts_column), it correctly shows the single "Name" column indexed by my date:

                                   Name
2017-05-31 11:31:32 "NAME1"
2017-05-31 11:31:32 "NAME2"
2017-05-31 11:31:32 "NAME3"
2017-05-31 11:31:32 "NAME4"
2017-05-31 11:31:32 "NAME5"
2017-05-31 11:31:32 "NAME6"
...

I get an impartial write on the influx_write step:

Error: {"error":"partial write:\nunable to parse 'my_measurement NA_to_remove
1496255335': invalid field format\nunable to parse 'my_measurement NA_to_remov
e 1496255335': invalid field format\nunable to parse 'my_measurement NA_to_rem
ove 1496255335': invalid field format\nunable to parse 'my_measurement NA_to_r
emove 1496255335': invalid field format\nunable to parse 'my_measurement NA_to
_remove 1496255335': invalid field format\nunable to parse 'my_measurement NA_
to_remove 1496255335': invalid field format\nunable to parse 'my_measurement N
A_to_remove 1496255335': invalid field format\nunable to parse 'my_measurement
 NA_to_remove 1496255335': invalid field format\nunable to parse 'my_measurement
 NA_to_remove 1496255371': invalid field format\nunable to parse 'my_measurement
 NA_to_remove 1496255371': invalid field format\nunable to parse 'my_measurement
NA_to_remove 1496255371': invalid field format\nunable to parse 'my_measurement
NA_to_remove 1496255371': inva
Execution halted

6 of the 1233 data points get written to the InfluxDB, but with the wrong timestamps:

SELECT * FROM my_measurement
                              Name
2017-05-31 18:28:55+00:00     NAME1
2017-05-31 18:29:31+00:00     NAME2
2017-05-31 18:30:02+00:00     NAME3
2017-05-31 18:30:38+00:00     NAME4
2017-05-31 18:31:11+00:00     NAME5
2017-05-31 18:31:32+00:00     NAME6

It looks like instead of using the xts_column's time index, it's simply inserting them using the current timestamp, and additionally some of those are not being parsed correctly?

[xanderdunn]

Actually, I've realized that the partial write is expected outcome.

Although I had 1233 points, they were not unique. There were only 6 unique times among the 1233 points:

2017-05-31 18:28:55+00:00
2017-05-31 18:29:31+00:00
2017-05-31 18:30:02+00:00
2017-05-31 18:30:38+00:00
2017-05-31 18:31:11+00:00
2017-05-31 18:31:32+00:00

And the points had no unique tags, so it's completely correct behavior that only 6 of them would get written, one for each unique time.

So, the only unexpected thing here is the error message:

Error: {"error":"partial write:\nunable to parse 'my_measurement NA_to_remove
1496255335': invalid field format\nunable to parse 'my_measurement NA_to_remov
e 1496255335': invalid field format\nunable to parse 'my_measurement NA_to_rem
ove 1496255335': invalid field format\nunable to parse 'my_measurement NA_to_r
emove 1496255335': invalid field format\nunable to parse 'my_measurement NA_to
_remove 1496255335': invalid field format\nunable to parse 'my_measurement NA_
to_remove 1496255335': invalid field format\nunable to parse 'my_measurement N
A_to_remove 1496255335': invalid field format\nunable to parse 'my_measurement
 NA_to_remove 1496255335': invalid field format\nunable to parse 'my_measurement
 NA_to_remove 1496255371': invalid field format\nunable to parse 'my_measurement
 NA_to_remove 1496255371': invalid field format\nunable to parse 'my_measurement
NA_to_remove 1496255371': invalid field format\nunable to parse 'my_measurement
NA_to_remove 1496255371': inva
Execution halted

[dleutnant]

fixed with 87b21c

hint: please install from now on from "dev" branch