A task fails if the subsequent task is already completed

[KansaiUser]

I am just starting to use and understand Luigi, I would appreciate some help here.

I have a pipeline built with Luigi in which some taks require other tasks and each task creates a file. Something like:

TaskA-------->TaskB---------> TaskC-------->TaskD
(fileA)             (fileB)                 (fileC)            (fileC)

• The first time I run the pipeline, everything runs well and gets created.

• If I run the pipeline again, nothing gets run since TaskD was completed already.

• If I manually delete fileB (made by TaskB), I expected it to be recreated and everything else running but the pipeline fails.

yes, fileB gets recreated but TaskC fails, with the error message saying that fileC already exist.

Is there a way for the subsequent files to be recreated again? Or fileC be overwritten?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.