Get Tasks up and Running

[sjmf]

Some changes need to be made to the docker configuration to get tasks to run correctly. Currently the task calcuations.dailyUpdateModel fails with a number of errors which need to be worked through involving some re-configuration of the Dockerfile.

[2022-09-15 14:03:49,365: ERROR/ForkPoolWorker-2] Task calculations.dailyModelUpdate[e0393d68-3ed2-44a4-b6e2-82ee964ce8a6] raised unexpected: DoesNotExist('ZentraDevice matching query does not exist.')
Traceback (most recent call last):
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/calculations/tasks.py", line 118, in dailyModelUpdate
    location = ZentraDevice.objects.get(device_sn=settings.STATION_SN).location
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/django/db/models/query.py", line 650, in get
    raise self.model.DoesNotExist(
calculations.models.ZentraDevice.DoesNotExist: ZentraDevice matching query does not exist.

To address this, a Zentra device matching the device in the environment variable needs to be created. Could this be automated somehow?

[2022-09-15 14:25:25,656: ERROR/ForkPoolWorker-2] Task calculations.dailyModelUpdate[d347d411-6456-4c79-b7ad-d543d20de202] raised unexpected: FileNotFoundError(2, 'No such file or directory')
Traceback (most recent call last):
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/calculations/tasks.py", line 156, in dailyModelUpdate
    updateInitialData = runningGenerateRiverFlows(
  File "/app/calculations/generate_river_flows.py", line 518, in runningGenerateRiverFlows
    riverFlowsData = GenerateRiverFlows(
  File "/app/calculations/generate_river_flows.py", line 364, in GenerateRiverFlows
    X = np.loadtxt(open(parametersFilePath), delimiter=",", usecols=range(4))
FileNotFoundError: [Errno 2] No such file or directory: '/Data/RainfallRunoffModelParameters.csv'

To address this, the Data directory needs to be either embedded into the image (+28MB) or mounted as a volume on the deployment host.

However, there is a larger issue here around feedback of task status and failures to the user UI which will need to be tracked in a separate issue. There is currently no display of whether a task has succeeded or failed in the UI.

[sjmf]

I have fixed the above errors in commit 8dedb87. However, a new error is thrown:

Traceback (most recent call last):
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/conda/envs/ManyFEWS/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/app/calculations/tasks.py", line 156, in dailyModelUpdate
    updateInitialData = runningGenerateRiverFlows(
  File "/app/calculations/generate_river_flows.py", line 518, in runningGenerateRiverFlows
    riverFlowsData = GenerateRiverFlows(
  File "/app/calculations/generate_river_flows.py", line 376, in GenerateRiverFlows
    modelfunOutputData = ModelFun(qp, Ep, dt, CatArea, X, F0)
  File "/app/calculations/generate_river_flows.py", line 34, in ModelFun
    S0 = F0[n, 0]  # initial storage level for PDM (mm)
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

This is within the model code, specifically line 156, in the function dailyModelUpdate in file /app/calculations/tasks.py, it looks like the parameter being passed to the function runningGenerateRiverFlows is incorrect somehow.

@Abel-Durham Perhaps you have some insight? I am wondering if this is because the GEFS data is missing as the task to download it has not run (or failed?). The variable F0 doesn't seem to contain whatever line 34 in /app/calculations/generate_river_flows.py was expecting.

[sjmf]

@Abel-Durham The solution to the above error is to run the task calculations.initialModelSetUp first!

I suggest some parameter sanity-checking in the runningGenerateRiverFlows function, and possibly also handling the IndexError and generating a verbose error message that F0 is not populated correctly and to run initialModelSetup first.

Closing this issue as the errors are fixed. See issue #83 for ongoing work.