search

getredash / redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
http://redash.io/
BSD 2-Clause "Simplified" License
26.27k stars 4.36k forks source link

Schema Fetching Can Fail Opaquely due to Bad Table Names #6702

Open dvfeinblum-betterment opened 9 months ago

dvfeinblum-betterment commented 9 months ago

Issue Summary

A team at our org has a tool that created a table in redshift with and incredibly unusual name, containing a few characters that broke a query in the postgres query_runner. The table in question was called analytics_tmp.c("redacted_keyword", "another_redacted_word")_versions.

Because of how the schema hydration actually happens, the characters in this table name broke this query. This failure took us quite a while to debug because of how opaque the worker logging is, particularly around schema fetching. I know there isn't much investment going on in this project now but I'm hoping this issue will act as a guidepost for others who end up with weird sudden schema hydration issues.

Steps to Reproduce

We saw this in the postgres query_runner but I'd wager this exists in other DBs. Go into a pg DB you have wired up to redash and run

CREATE TABLE "foo.c(""bar"", ""baz"")_test" (bang text);

Then just hop into redash and try refreshing the schema.

Technical details:

dvfeinblum-betterment commented 9 months ago

Gonna slap some search terms here me and my team used when we were trying to see if anyone else had run into this:

redash failed getting schema redash force schema refresh redash where are schemas stored redash could not send ssl negotiation packet (this was a red herring) redash adhoc_worker queues redash schema refresh not working redash refresh metadata