slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
399 stars 27 forks source link

Column type coercion to boolean from tinyint stopped working in 1.2.10 #320

Closed somiandras closed 3 months ago

somiandras commented 3 months ago

Issue Description

I have a couple of boolean columns in a MySQL source defined as tinyint that I coerced into bool in my Postgres target using source_options, but this seems to have stopped working after upgrading to 1.2.10. (worked on 1.2.9 and doesn't work on either 1.2.10 or 1.2.11)

In my replication config I'm adding source options to the MySQL source tables like this:

    source_options:
      columns:
        myFirstColumn: bool
        mySecondColumn: bool
        myThirdColumn: bool

These columns are TINYINT in MySQL, containing 0, 1 and possibly NULL as values. Columns were correctly coerced to boolean in my Postgres target before 1.2.10, now they are VARCHAR(65500).

can't share the full config, the defaults are pretty simple, everything is set up on stream level

defaults:
  object: staging.{stream_table}

  target_options:
    column_casing: snake

...
Paste log here.
flarco commented 3 months ago

Thanks for reporting. Fixed here: https://github.com/slingdata-io/sling-cli/pull/318/commits/676354c3e1de0e176db0cd2ca61004cb2dd64199