airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.25k stars 4.15k forks source link

[destination-snowflake] failing to sync google sheets #47175

Open tom-mont opened 1 month ago

tom-mont commented 1 month ago

Connector Name

destination-snowflake

Connector Version

3.15.0

What step the error happened?

During the sync

Relevant information

I recently upgraded the snowflake destination sync from 3.11.5 -> 3.15.0 and experienced issues with my google sheets sync.

It appears as though the records do sync but it reaches an issue with the snowflake connector. The following warning is thrown:

Warning from destination: java.lang.RuntimeException: SQL compilation error: error line 150 at position 2 invalid identifier '"COMPONENT COMBINATION"' in SYSTEM$MULTISTMT at ' throw `Execution of multiple statements failed on statement {0} (at line {1}, position {2}).`.replace('{1}', LINES[i])' position 4 stackstrace: SYSTEM$MULTISTMT line: 10

default_workspace_job_36613_attempt_20_txt.txt

Log file above

Relevant log output

Caused by: net.snowflake.client.jdbc.SnowflakeSQLException: JavaScript execution error: Uncaught Execution of multiple statements failed on statement "INSERT INTO "GOOGLE_SHEETS"."C..." (at line 2, position 0).
SQL compilation error: error line 150 at position 2
invalid identifier '"COMPONENT
   COMBINATION"' in SYSTEM$MULTISTMT at '    throw `Execution of multiple statements failed on statement {0} (at line {1}, position {2}).`.replace('{1}', LINES[i])' position 4

Contribute

marcosmarxm commented 1 week ago

@tom-mont I found this message in logs:

2024-10-21 12:09:20 destination > WARN sync-operations-3 i.a.c.d.j.DefaultJdbcDatabase(unsafeQuery):138 closing connection because of an Exception net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
Table 'RAW_PROD.GOOGLE_SHEETS._AIRBYTE_RAW_SUPPORT_COACH_NSC' does not exist or not authorized.
    at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowExceptionSub(SnowflakeUtil.java:172) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.jdbc.SnowflakeUtil.checkErrorAndThrowException(SnowflakeUtil.java:105) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.StmtUtil.pollForOutput(StmtUtil.java:490) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.StmtUtil.execute(StmtUtil.java:396) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.SFStatement.executeHelper(SFStatement.java:498) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:215) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:149) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.SFStatement.execute(SFStatement.java:785) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.core.SFStatement.execute(SFStatement.java:693) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:296) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:158) ~[snowflake-jdbc-3.19.0.jar:3.19.0]
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) ~[HikariCP-5.1.0.jar:?]
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) ~[HikariCP-5.1.0.jar:?]
    at io.airbyte.cdk.db.jdbc.DefaultJdbcDatabase.unsafeQuery(DefaultJdbcDatabase.kt:121) ~[airbyte-cdk-core-0.46.0.jar:?]
    at io.airbyte.cdk.db.jdbc.JdbcDatabase.unsafeQuery(JdbcDatabase.kt:198) ~[airbyte-cdk-core-0.46.0.jar:?]
    at io.airbyte.cdk.db.jdbc.JdbcDatabase.queryJsons(JdbcDatabase.kt:218) ~[airbyte-cdk-core-0.46.0.jar:?]
    at io.airbyte.integrations.destination.snowflake.typing_deduping.SnowflakeDestinationHandler$Companion.getTable(SnowflakeDestinationHandler.kt:563) ~[io.airbyte.airbyte-integrations.connectors-destination-snowflake.jar:?]