search

slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
301 stars 16 forks source link

Don't Create Parquet File with 0 rows #267

Closed flarco closed 2 months ago

flarco commented 2 months ago

Discussed in https://github.com/slingdata-io/sling-cli/discussions/266

Originally posted by **ShahBinoy** April 17, 2024 Hi @flarco , I have data being fetched from MySQL database using a custom query in Replication. Lot of times there is no data coming back for a certain query, in such scenario, Sling creates empty parquet file, this throws off the collective scanning of data. Is there a way to not create parquet file if the row count returned is 0 ?
flarco commented 2 months ago

New env var SLING_ALLOW_EMPTY in 1.2.6