search

radon-h2020 / radon-defect-prediction-api

A defect prediction for Infrastructure code
https://radon-h2020.github.io/radon-defect-prediction-api/
Apache License 2.0
0 stars 1 forks source link

R-T3.4-10: The defect-prediction tool must improve performances over manual inspection #5

Closed dardin88 closed 2 years ago

dardin88 commented 4 years ago
ID R-T3.4-10
Section WP3: Methodology and Quality Assurance Requirements
Type PERFORMANCE_EFFICIENCY
User Story As an Operations Engineer I want the tool to find more defects than manual inspection
Requirement The defect-prediction tool must improve performances over manual inspection
Extended Description The tool needs to finish its analysis in less time with respect to manual inspection, without loss of effectiveness
Priority Must have
Affected Tools DEFECT_PRED_TOOL
Means of Verification User study with performance improved by at least 15%
Dependency
stefanodallapalma commented 4 years ago

@gcasale This requirement will take time to be addressed. We are designing industrial cases studies with ENG and PRQ to address it.

stefanodallapalma commented 3 years ago

Hi everyone @gcasale @dardin88

Considering that the goal of the tool is to prioritize manual inspection to improve the overall code quality, I suggest to upgrade this requirement (therefore update title and description) so to provide an industrial survey with RADON industrial partners to understand whether they perceive the defect prediction tool helpful in their daily tasks. The reason for this is that, unfortunately, the evaluation of this requirement is impractical, as it would require a controlled experiment or an ethnography. Such study should involve multiple developers or operators detecting manually injected faults using or not the defect prediction tool. At this stage and under the COVID pandemic, such a controlled experiment cannot be performed.

gcasale commented 3 years ago

Agreed, if the industrial partners also agree to it

On Mon, 29 Mar 2021 at 15:15, Stefano Dalla Palma @.***> wrote:

Hi everyone @gcasale @dardin88

Considering that the goal of the tool is to prioritize manual inspection to improve the overall code quality, I suggest to upgrade this requirement (therefore update title and description) so to provide an industrial survey with RADON industrial partners to understand whether they perceive the defect prediction tool helpful in their daily tasks. The reason for this is that, unfortunately, the evaluation of this requirement is impractical, as it would require a controlled experiment or an ethnography. Such study should involve multiple developers or operators detecting manually injected faults using or not the defect prediction tool. At this stage and under the COVID pandemic, such a controlled experiment cannot be performed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

stefanodallapalma commented 3 years ago

We have already prepared and shared a survey that the partners (ENG and PRQ) agreed to fill in the next few months.