search

timescale / outflux

Export data from InfluxDB to TimescaleDB
Apache License 2.0
86 stars 22 forks source link

migrate crashes due to 0x00 character in string field #78

Open jbreadner opened 4 years ago

jbreadner commented 4 years ago

While running migrate, the process crashed with the following message:

2020/01/14 21:10:06 pipe_device_monitor_V2_ext: Extracted 4785000 rows from Influx
2020/01/14 21:10:06 pipe_device_monitor_V2_ext: Extracted 4800000 rows from Influx
2020/01/14 21:10:07 pipe_device_monitor_V2_ext: Extracted 4815000 rows from Influx
2020/01/14 21:10:08 pipe_device_monitor_V2_ext: Extracted 4830000 rows from Influx
2020/01/14 21:10:09 pipe_device_monitor_V2_ext: Extracted 4845000 rows from Influx
2020/01/14 21:10:10 pipe_device_monitor_V2_ext: Extracted 4860000 rows from Influx
2020/01/14 21:10:10 pipe_device_monitor_V2_ing could not insert batch of rows in output db
ERROR: invalid byte sequence for encoding "UTF8": 0x00 (SQLSTATE 22021)
2020/01/14 21:10:11 pipe_device_monitor_V2_ext: Extracted 4875000 rows from Influx
^C

The migration stalled and did not exit after that error was thrown; I had to ^C out.

Would it be advisable / possible to add a flag indicating that these characters be stripped out during the migrate process?