search

dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.96k stars 1.5k forks source link

Data versions don't work properly with self-partitioned assets + partition range materializations #16287

Open smackesey opened 1 year ago

smackesey commented 1 year ago

Something gets screwy in the provenance tracking. To see this, change the logic in 'execution_tests/test_data_versions.py::test_stale_status_self_partitioned to materialize partitions with a range instead of a loop.

smackesey commented 1 year ago

cc @sryza @OwenKephart @gibsondan just so you're aware. I discovered this while working on #16285.

Fortunately I think this is quite a niche scenario.