We already expose the number of imported contexts, as a counter exported to prometheus, but we might also want to export the number of missing contexts. Currently this fact is simply logged and we decided not to interrupt the job if a context is missing but we could do more in order to make this information more visible just in case missing context is really relevant.
I was thinking today about something in the middle between a) fatal failure and b) not bothering with anything else than just logging. Some kind of a WARN but more visible. I realized we have this metrics reporting functionality which kind of suits this purpose: after importing phase a first batch of metrics is being exported including the number of imported contexts. We could simply add the number of missing contexts so whenever importing phase is completed we could get to know, e.g. via prometheus/grafana alerting mechanism, that some concept was missing and inspect the situation without the interruption of ongoing workflow. If we assess the missing context is important and we should rerun the IIS importing phase then we could kill the mining job.
This is a #1434 follow-up.
We already expose the number of imported contexts, as a counter exported to prometheus, but we might also want to export the number of missing contexts. Currently this fact is simply logged and we decided not to interrupt the job if a context is missing but we could do more in order to make this information more visible just in case missing context is really relevant.
Originally mentioned here: https://support.openaire.eu/issues/9059#note-6: