slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
301 stars 16 forks source link

sql in a stream does not seem to override default sql #211

Closed fabrice-etanchaud closed 3 months ago

fabrice-etanchaud commented 3 months ago

Hi, doing some testing with 1.1.12, in a oracle-oracle configuration. I found that if I set a default sql, and then define a custom sql in a stream, the default sql is still used.

Am I doing something wrong ?

Thank you !

flarco commented 3 months ago

Hi, can you share the replication config you are using?

fabrice-etanchaud commented 3 months ago

In fact, it seems that the sql set in the default is not taken into account.

Here is my replication file :

source: ORACLE_ODS_V6_CONSULTATION
target: ORACLE_DATA_DEV

env:
  source: SOLODS6

defaults:

  source_options:
    empty_as_null: true
    trim_space: true
    limit: 1

  target_options:
    use_bulk: true

  sql: |
    select 'default' as type_sql from dual

  object: PILOTAGE_TECH.{stream_table}_test_sql

streams:

  data_pmaif.adh_contrat:

  data_pmaif.info_produit:.1:

    sql: |
      select 'override' as type_sql from dual

Thank you !

flarco commented 3 months ago

Actually, the logic is missing! Now added: https://github.com/slingdata-io/sling-cli/pull/208/commits/489268eb719a5dd095ff1eccc801066b52bc8680 Thanks, closing. Should work when v1.1.14 is released.

fabrice-etanchaud commented 3 months ago

incredible ! Thank you !