Today exploring some issues that @mhow2 and in minor degree @valeriocos raised to me, I discovered an important bug that it is affecting indirectly all the creators of metric providers.
The core issue is the fact that metrics are sometimes not executed, whether a new task has been created or reset. Meaning that data in MongoDB and ElasticSearch are not being updated.
Case 1 Task Reset:
So, let's consider that we have created a new project and a new task. We have executed this task, and for whatever reason, we need to do it again, e.g. there was a bug in a metric provider. When we click on restart, only the entry related to the task in the collection analysisTasks will be reset. However, the metrics in the collection metricExecutions will not, specifically the lastExecutionDate will not be reset. Thus, when the task is restarted, the platform will call the readers, create deltas (or use cached deltas), but no metric will be executed. The reason is that the platform prevents the execution of metrics that have a lastExecutionDate after the delta.
I guess this issue is a consequence of the fix proposed for issue #178. Where the platform does not execute metrics that have been already executed in the past and in this way do not affect historic metrics.
Case 2 New additional task:
The issue happens when an additional task is created and has two particular characteristics. One, the new task shares metrics with another already executed task. Two, the new task overlaps in the date range.
As the metrics executions are shared by project and not by task, when users execute the task, the platform will call the readers but no metrics will be executed.
Both issues are are serious. In first place because all the metrics are affected historic and not historic, as well indexer. If metrics are not executed, yes we prevent duplicated data in historic metrics, but none of the data will be updated correctly in MongoDB and the ElasticSearch. In second place, User Partners will believe that metrics and tools created by R&D partners are wrong, not fixed, or not working.
Again, after seeing this issues, I think my proposal of creating collections based on project+task was the correct way to solve issue #178. As it allowed the platform dropping a collection if the task was reset or changed.
I will send an email to all the partners and recommend them rather than creating task to create projects. And rather than resetting tasks, to delete the project and recreate it. That's the current hotfix that I can propose.
Hello,
Today exploring some issues that @mhow2 and in minor degree @valeriocos raised to me, I discovered an important bug that it is affecting indirectly all the creators of metric providers.
The issue is similar to https://github.com/crossminer/scava/issues/178 and quite probable to the implemented solution.
The core issue is the fact that metrics are sometimes not executed, whether a new task has been created or reset. Meaning that data in MongoDB and ElasticSearch are not being updated.
Case 1 Task Reset: So, let's consider that we have created a new project and a new task. We have executed this task, and for whatever reason, we need to do it again, e.g. there was a bug in a metric provider. When we click on restart, only the entry related to the task in the collection
analysisTaskswill be reset. However, the metrics in the collectionmetricExecutionswill not, specifically thelastExecutionDatewill not be reset. Thus, when the task is restarted, the platform will call the readers, create deltas (or use cached deltas), but no metric will be executed. The reason is that the platform prevents the execution of metrics that have alastExecutionDateafter the delta. I guess this issue is a consequence of the fix proposed for issue #178. Where the platform does not execute metrics that have been already executed in the past and in this way do not affect historic metrics.Case 2 New additional task: The issue happens when an additional task is created and has two particular characteristics. One, the new task shares metrics with another already executed task. Two, the new task overlaps in the date range. As the metrics executions are shared by project and not by task, when users execute the task, the platform will call the readers but no metrics will be executed.
Both issues are are serious. In first place because all the metrics are affected historic and not historic, as well indexer. If metrics are not executed, yes we prevent duplicated data in historic metrics, but none of the data will be updated correctly in MongoDB and the ElasticSearch. In second place, User Partners will believe that metrics and tools created by R&D partners are wrong, not fixed, or not working.
Again, after seeing this issues, I think my proposal of creating collections based on project+task was the correct way to solve issue #178. As it allowed the platform dropping a collection if the task was reset or changed.
I will send an email to all the partners and recommend them rather than creating task to create projects. And rather than resetting tasks, to delete the project and recreate it. That's the current hotfix that I can propose.