dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
297 stars 119 forks source link

stage generation ignores value of FIELD_OPTIONALLY_ENCLOSED_BY #149

Closed Kratzbaum92 closed 2 years ago

Kratzbaum92 commented 2 years ago

Describe the bug

DBT ignores the value of the optional parameter field_optionally_enclosed_by.

Steps to reproduce

          +materialized: from_external_stage
          +stage_file_format: 'file_format_csv'
          +field_del: ','
          +field_optionally_enclosed_by: '"'

Expected results

create or replace file format  LZ_CATALOGUE_PRODUCTSUP_FEED.file_format_csv
       type = 'CSV' 
        RECORD_DELIMITER = '\n' 
        FIELD_DELIMITER = ','
        FIELD_OPTIONALLY_ENCLOSED_BY='"'
        SKIP_HEADER = 1 
        NULL_IF = ('null','')
        ESCAPE_UNENCLOSED_FIELD = '\r'
        EMPTY_FIELD_AS_NULL=true
        COMPRESSION=GZIP 
        ENCODING =  'ISO 8859-1';

Actual results

create or replace file format  LZ_CATALOGUE_PRODUCTSUP_FEED.file_format_csv
       type = 'CSV' 
        RECORD_DELIMITER = '\n' 
        FIELD_DELIMITER = ','
        FIELD_OPTIONALLY_ENCLOSED_BY=none
        SKIP_HEADER = 1 
        NULL_IF = ('null','')
        ESCAPE_UNENCLOSED_FIELD = '\r'
        EMPTY_FIELD_AS_NULL=true
        COMPRESSION=GZIP 
        ENCODING =  'ISO 8859-1';

Screenshots and log output

System information

packages:
  - package: dbt-labs/dbt_utils
    version: 0.8.0
  - package: dbt-labs/dbt_external_tables
    version: 0.8.0

Which database are you using dbt with?

The output of dbt --version:

    version: 1.0.0

The operating system you're using:

The output of python --version:

Additional context

Kratzbaum92 commented 2 years ago

different problem not inside the makro