AbsaOSS / enceladus

Dynamic Conformance Engine
Apache License 2.0
30 stars 14 forks source link

Make it easier to rerun Standardization if Conformance failed #2188

Open benedeki opened 1 year ago

benedeki commented 1 year ago

Background

If Conformance phase fails it leaves behind the temporary files created by Standardization. That prevents Standardization to re-run easily.

Feature

Find and implement a way to make Standardization re-run easier.

Proposed Solution

Solution Ideas:

  1. If auto-clean option is on delete the Standardization target directory if it exists instead of failing
  2. Introduce a new switch force that will ignore the existence of target directory and delete it first if it exists.
benedeki commented 1 year ago

Is _Standardization) re-run easier and cheaper than detecting the presence of the target directory?

yruslan commented 1 year ago

Thanks for the ticket. I like both solution ideas. Maybe the solution 2 actually makes more sense. '--force' can be used only on re-runs, and can do deletions for publish folders as well.

Yes, definitely adding auto-deletion on re-run will make it much easier, cheaper and also less error-prone. Since currently a person needs to delete corresponding folders manually, and can delete a wrong folder just by accident, especially if many re-runs are required.