Imports information from external systems, which is then validated, parsed, and then submitted as structured data (YAML or JSON) to a separate Github repository.
Apache License 2.0
3
stars
0
forks
source link
Job fails if a previous execution failed before file was deleted #305
Whenever a job fails after downloading the source file and before deleting it said file remains in the disk and when the next execution happens it'll check if the file exists and fail.
This behavior might've been sensible in its original form but as it currently stands it merely causes the app to no longer be able to parse files for that institution until its manually deleted.
This issue is easy to replicate:
Launch job
Stop the app after the download step and before the extraction step completes
It's also easy to happen in Heroku as the heavier jobs (i.e. LEIC timetable) often exceed the memory quota and crash in this exact step, blocking further jobs.
Whenever a job fails after downloading the source file and before deleting it said file remains in the disk and when the next execution happens it'll check if the file exists and fail.
This behavior might've been sensible in its original form but as it currently stands it merely causes the app to no longer be able to parse files for that institution until its manually deleted.
This issue is easy to replicate:
It's also easy to happen in Heroku as the heavier jobs (i.e. LEIC timetable) often exceed the memory quota and crash in this exact step, blocking further jobs.