datafold / data-diff

Compare tables within or across databases
https://docs.datafold.com
MIT License
2.95k stars 270 forks source link

`pg_get_cols` query for Redshift is erroring out #875

Closed mjkanji closed 5 months ago

mjkanji commented 7 months ago

Describe the bug I'm trying to diff two views in Redshift. However, when I run data-diff using either --dbt or the fully qualified URLs, I'm getting the following error:

psycopg2.errors.DatatypeMismatch: query-specified return row and actual function return row do not match

Having run each of the queries shown in --interactive mode, I was able to narrow it down to the following query:

select * from pg_get_cols('my_schema.my_view')
cols(col_name name, col_type varchar)

Looking at the Redhisft docs on pg_get_cols, it seems the right way to run this query is as follows:

select
    * -- or col_name, col_type if you're only interested in those columns  
from pg_get_cols('my_schema.my_view')
cols(view_schema name, view_name name, col_name name, col_type varchar, col_num int)

This query does work.

(Also, as a side note, when I use --interactive, it does turn on debugging logs, but the queries are all run without asking me to interactively confirm the query.)

If possible, please paste these as text, and not a screenshot.

Describe the environment

I'm on Linux (WSL2 on Windows) and data-diff v0.11.1.

github-actions[bot] commented 5 months ago

This issue has been marked as stale because it has been open for 60 days with no activity. If you would like the issue to remain open, please comment on the issue and it will be added to the triage queue. Otherwise, it will be closed in 7 days.

glebmezh commented 5 months ago

Hi @mjkanji,

Thank you for trying out data-diff and for taking the time to open this issue. We made a hard decision to sunset the data-diff package and won't provide further development or support. Diffing functionality will continue to be available in Datafold Cloud. Feel free to take it for a trial or contact us at support@datafold.com if you have any questions.

-Gleb