DAG fails when metadata information is not provided

[chpicone]

Customer reported this issue:

The airflow thing works fine but I have a few Dags they don’t work (even if I trigger them again)
- KIIVBlOE196
- KVBlOE1488
- KVBlOE775
- KVBlOEHun29
- KVBlWE82

This is the stacktrace from the Airflow Log section:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1112, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1285, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 1315, in _execute_task
    result = task_copy.execute(context=context)
  File "/opt/airflow/plugins/custom_operator/georectify.py", line 22, in execute
    return self._geonode_payload()
  File "/opt/airflow/plugins/custom_operator/georectify.py", line 53, in _geonode_payload
    "data_quality_statement": metadata.get('format', None).replace("+", " "),
AttributeError: 'NoneType' object has no attribute 'replace'

Taking a look into the code we can see the cause of the issue: https://github.com/geosolutions-it/oaw-georectify-process-airflow/blob/269ba16a082a3746925d9b4f97c9f383999b78a4/plugins/custom_operator/georectify.py#L53

When we dont have a metadata information the get method assign by default the None object and then we cannot use the replace method.

[chpicone]

We have 2 options:

• if the attribute is missing we can provide an empty string and we can try to continue the process
• if the attribute is missing we can raise an exception (like the current one) but writing a more clear message in the log panel

I asked the customer to decide how to proceed.

[chpicone]

Customer wants to make all attributes mandatories. So the DAG has to fail but reporting a more clear message, like:

AttributeError: 'format' metadata attribute is missing!