When pipeline is interrupted/fails during execution, the *.nc/*.tif files are not deleted from DATA. Due to their presence in next run, the pipeline fails.
I manually fix this by deleting old downloaded *.nc files from root of DATA. The pipeline then successfully runs.
Fix- Make sure to handle any exception and in it, delete these files.
When pipeline is interrupted/fails during execution, the
*.nc
/*.tif
files are not deleted fromDATA
. Due to their presence in next run, the pipeline fails. I manually fix this by deleting old downloaded*.nc
files from root ofDATA
. The pipeline then successfully runs.Fix- Make sure to handle any exception and in it, delete these files.