timescale / outflux

Export data from InfluxDB to TimescaleDB
Apache License 2.0
90 stars 22 forks source link

isHypertableQueryTemplate incorrect for docker image timescale/timescaledb:2.0.0-pg12 #85

Closed rafaelrpinto closed 12 months ago

rafaelrpinto commented 3 years ago

Hi, While migrating the data from influxdb I can transfer the schema correctly using outflux schema-transfer but outflux migrate using the schema strategy CreateIfMissing fails with the following error:

2021/01/26 23:57:39 pipe_post: pipe_post: could not prepare ingestor
could not check if table post is hypertable

Since the table post is a hypertable I digged into the code and found that the failed validation uses the query isHypertableQueryTemplate which is :

SELECT EXISTS (
                                    SELECT 1
                                    FROM timescaledb_information.hypertable
                                     WHERE  table_schema = 'public' AND table_name='post')

However the hypertable table is not on timescaledb_information schema, but it is on _timescaledb_catalog.

So currently the only option for us to use outflux is to use the schema strategy DropAndCreate.

Thanks.

gubbin commented 3 years ago

I have a fork with your query fix at https://github.com/hatchdata/outflux but it only works with newer TimescaleDB versions. It would need some version logic before it would be ready for a PR.

wiebeytec commented 2 years ago

I run into the same issue with outlux version 0.2.0. Timescaledb version 2 uses timescaledb_information.hypertables and hypertable_name).

There is a blocked pull request: https://github.com/timescale/outflux/pull/89

JamesGuthrie commented 12 months ago

Fixed in #89