mara / mara-pipelines

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
MIT License
2.07k stars 100 forks source link

Fix CopyIncrementally with no data #54

Closed jankatins closed 3 years ago

jankatins commented 3 years ago

Main thing is the last commit which handled the case that the last modification comparison query does not return any value but still succeeds (=would load no data). In that case we now succeed the copy task but don't actually attempt to load data.

This also adjusts the printed messages from Copy to be more readable (I hope) and adds imports and asserts to make my IDE happy...