Adopt criteria codes from SQA baseline

[samuelbernardolip]

This issue has evolved (from qc_unit/qc_coverage discussion) to cover the transition to align the criteria codes to the ones being used in the SQA baseline.

We will need to create PRs for the current developments using the JePL library. Track list (make sure to only check whenever the PR is already merged):

• [ ] sqaaas-api-spec (PR #)
• [ ] sqaaas-web (PR #)
• [ ] Worsica (PR #)
• [ ] O3AS (PR #)
• [ ] SAPS (PR #)
• [ ] OpenEbench (PR #)
• [ ] LAGO (PR #)

[davrodgon]

Probably we should stick to the ones defined in the SQA Baseline document:

• Code Accessibility [QC.Acc] qc_accessibility
• Licensing [QC.Lic] qc_licensing
• Code Workflow [QC.Wor] qc_workflow
• Code Management [QC.Man] qc_management???
• Code Style [QC.Sty] qc_style
• Code metadata [QC.Met] qc_metadata
• Unit Testing [QC.Uni] qc_unit
• Functional Testing [QC.Fun] qc_functional
• Integration Testing [QC.Int] qc_integration
• Documentation [QC.Doc] qc_doc
• Security [QC.Sec] qc_security
• Code Review [QC.Rev] qc_review???
• Automated Deployment [QC.Aud] qc_deploy or qc_autodeploy ???

I don't think we can implement all, but we can use categories from that list.

[orviz]

The required change in the jpl-validator has been added by @davrodgon in https://github.com/EOSC-synergy/jpl-validator/commit/049a4380191b37cb2806b5e3831a3400449847b0

[BorjaEst]

I was wondering if it wouldn't have more sense to measure coverage by Unit testing and Functional Testing together? Some file-functions in Python are complicated to manage as Unit testing therefore 100% is difficult to achieve only with unittest.

In my case, I run pytest to get the coverage form everything:

├── package
│   ├── __init__.py
│   ├── test <<--- WhiteBox/ Unit tests
│   │   ├── __init__.py
│   │   ├── test_modules.py
│   └── modules.py
├── tests <<--- BlackBox/ Functional tests
│   ├── __init__.py
│   ├── conftest.py
│   ├── mockup
│   │   └── __init__.py
│   └── test_functionals.py
└── tox.ini

Then simply configure in tox:

commands = pytest --basetemp="{envtmpdir}"  \
                  --confcutdir=".."         \
                  --numprocesses='auto'     \
                  --cov="package"

Then Pytest would go through both type of tests and provide me the final coverage. A the end, coverage should be something inherited from the testing phases, not something explicit defined.

The dificutly is how to have the coverage from both environments without repeating each tests 2 times.

[orviz]

Thanks @BorjaEst for your feedback. In your case it really makes sense to just use the functional criterion QC.Fun. As you may know, in the current JePL implementation, both criteria provide the same functionality and are defined with the same properties, so for the time being they just differ on the code name.

However you raised a good point, and we can address this at the level of the criteria definition [1] (in which the JePL is based on). The simplest solution could be to add a new bullet to address this fact. I've open this related issue [2], feel free @BorjaEst to comment there.

[1] https://indigo-dc.github.io/sqa-baseline/#functional-testing-qc.fun, [2] https://github.com/indigo-dc/sqa-baseline/issues/56