search

bruin-data / ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
https://bruin-data.github.io/ingestr/
MIT License
2.55k stars 59 forks source link

Error with Snowflake source NUMBER(X,0) field type #19

Open adaykoth opened 6 months ago

adaykoth commented 6 months ago

I run into this error whenever I try to ingest a table from snowflake where a field is of the format NUMBER(X,0):

[2024-04-29 23:22:27] [ua.ingestrrrpred] <class 'dlt.destinations.exceptions.DatabaseTransientException'> [2024-04-29 23:22:27] [ua.ingestrrrpred] 400 POST https://bigquery.googleapis.com/bigquery/v2/projects/idle-fast-food-bbf2a/queries?prettyPrint=false: In NUMERIC(P, 0), P must be between 1 and 29 at [8:15]

This does not happen when I convert the type to either integer or allow a precision for the floating point number (e.g. NUMBER(X,2)).

trakmaker commented 4 months ago

@adaykoth, The error suggests that BigQuery only supports a precision range of (1,29). You need the source datatype that supports this behaviour in your landing area

ref : BigQuery Doc