When pipeline is interrupted/fails during execution, the *.nc/*.tif files are not deleted from DATA. Due to their presence in next run, the pipeline fails.
I manually fix this by deleting old downloaded *.nc files from root of DATA. The pipeline then successfully runs.
Fix- Make sure to handle any exception and in it, delete these files.
When pipeline is interrupted/fails during execution, the
*.nc/*.tiffiles are not deleted fromDATA. Due to their presence in next run, the pipeline fails. I manually fix this by deleting old downloaded*.ncfiles from root ofDATA. The pipeline then successfully runs.Fix- Make sure to handle any exception and in it, delete these files.