Some sources are expected to deliver a particular file (dataset) upon every delivery, while other files are delivered more irregularly. The frequency of delivery (e.g. every week / day / 15 minutes) does not matter here.
It would be very useful to have such a 'meta' expectation available for a high level data quality check on the landing zone: if a file (dataset) is missing, some pipelines might not be able to run (or should not run at all).
Some sources are expected to deliver a particular file (dataset) upon every delivery, while other files are delivered more irregularly. The frequency of delivery (e.g. every week / day / 15 minutes) does not matter here.
It would be very useful to have such a 'meta' expectation available for a high level data quality check on the landing zone: if a file (dataset) is missing, some pipelines might not be able to run (or should not run at all).
A previous version of GX used to support an
expect_file_to_existexpectation, which subsequently got removed - see also https://github.com/great-expectations/great_expectations/issues/3979