fix: DIA-1168: Delete kafka topics on job failure / cancel

[hakan458]

We observed an issue where the list of kafka topics was growing on develop Adala instance, even though we implemented logic to delete them after completing a job. However, we did not handle the case of a job failing, or being canceled. I have added a handler to be run when the parent_streaming_task fails, so that we clean up topics, and also cleaning up topics when we cancel a job.

Also added logic to attempt to stop input/output jobs when the parent task fails.

[robot-ci-heartex]

Commit https://github.com/HumanSignal/infra/commit/3144ea8053f8c9a81c06084acccd9bd9987a7da8 is created.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 21 lines in your changes missing coverage. Please review.

Project coverage is 43.73%. Comparing base (a20ead5) to head (9c0f058). Report is 20 commits behind head on master.

Files	Patch %	Lines
server/tasks/process_file.py	0.00%	15 Missing :warning:
server/app.py	0.00%	6 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #131 +/- ## ========================================== - Coverage 44.20% 43.73% -0.47% ========================================== Files 40 40 Lines 1794 1813 +19 ========================================== Hits 793 793 - Misses 1001 1020 +19 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[robot-ci-heartex]

Commit https://github.com/HumanSignal/infra/commit/7cbe0c3eddada655513282936dc6fea28afb2136 is created.

[robot-ci-heartex]

Commit https://github.com/HumanSignal/infra/commit/dd66e852b96c00eacd6bf8c245678464001816c3 is created.

[robot-ci-heartex]

Commit https://github.com/HumanSignal/infra/commit/663b690ce0818ba4dfb3de19b863067dcca29290 is created.

[robot-ci-heartex]

Commit https://github.com/HumanSignal/infra/commit/cd55f815a181208f672d8246a88d637ed64bf32d is created.