This commit copies over some discovery changes from the source-postgres CDC connector (from which the Redshift discovery logic is mostly derived) to allow a source DB type to be mapped to multiple JSON types instead of just one.
Then that feature is used so that float4/float8 columns can be discovered as "either string-format-number or number". This is consistent with what source-postgres does and might help us with NaN-handling at some point in the future, but the real reason it's useful right now is that it unblocks a production capture which is currently having trouble with a column type changing from NUMERIC to FLOAT, because schema inference says that values are string-format-number and discovery says that values are numbers and Flow says that there are no values which can satisfy both of those constraints.
Description:
This commit copies over some discovery changes from the
source-postgresCDC connector (from which the Redshift discovery logic is mostly derived) to allow a source DB type to be mapped to multiple JSON types instead of just one.Then that feature is used so that float4/float8 columns can be discovered as "either string-format-number or number". This is consistent with what
source-postgresdoes and might help us with NaN-handling at some point in the future, but the real reason it's useful right now is that it unblocks a production capture which is currently having trouble with a column type changing fromNUMERICtoFLOAT, because schema inference says that values are string-format-number and discovery says that values are numbers and Flow says that there are no values which can satisfy both of those constraints.This change is