Evaluation fails in _ensure_process_created

[DavidHuebner]

Describe the bug We are currently seeing a problem in the platform because the list of returned processes includes those that should be deleted and from which we cannot get details. This manifests in an error in the _ensure_process_created when creating an evaluation process.

Proposed workaround In the method _ensure_process_created, we don't actually need the process details, we just need the name of all processes. So we can change the call from _list_processes to just getting the list of process names. This will also speed-up the calls because get_process() is rather slow because the number of processed documents needs to be computed.

Error message

  File "/var/lib/****/workspace/are_RD_evaluation-hd-runner_main/evaluation-hd-runner/evaluation/main.py", line 145, in <module>
    eval_process: Process = hd_pipeline_process.evaluate_against(gold_process,
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 100, in new_function
    return original_function(*args, **kwargs)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 859, in evaluate_against
    return self.project.client._evaluate(
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 100, in new_function
    return original_function(*args, **kwargs)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 3174, in _evaluate
    self._ensure_process_created(project, process_name)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 3192, in _ensure_process_created
    processes = self._list_processes(project)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 100, in new_function
    return original_function(*args, **kwargs)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 2750, in _list_processes
    processes.append(document_collection.get_process(item["processName"]))
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 100, in new_function
    return original_function(*args, **kwargs)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 1084, in get_process
    return self.project.client._get_process(self, process_name)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 100, in new_function
    return original_function(*args, **kwargs)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 2813, in _get_process
    response = self.__request_with_json_response(
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 1800, in __request_with_json_response
    self.__handle_error(raw_response)
  File "/var/lib/****/venv/lib/python3.9/site-packages/averbis/core/_rest_client.py", line 3005, in __handle_error
    raise RequestException(error_msg)
requests.exceptions.RequestException: 404 Server Error: 'Not Found' for url: 'http://health-discovery-hd:8800/health-discovery/rest/experimental/textanalysis/projects/evaluation/documentSources/corpus_icd10_copd_en_validation/processes/corpus_icd10_copd_en_validation-discharge_4instances-SUBSTRING_3'.

Additional Context In principle, this problem can manifest in all calls to list_processes. We will implement a complete fix when we have implemented platform changes.

Please complete the following information:

• Version: Affects Health-Discovery version 7.0 to 7.2.