QA acceptance criteria for fabric8-analytics release/demo

[tisnik]

Code quality

• [ ] unit tests setup to be run for all PR in the CI for all repositories (1)
• [ ] unit tests coverage is measured in the same job ^
• [ ] unit tests coverage >= 75% for all repositories (1)
• [ ] 100% pylinter pass for all repositories
• [ ] 100% docstyle checker pass for all repositories
• [ ] status of all repositories properly displayed on QA Dashboard (https://fabric8-analytics.github.io/dashboard/dashboard.html)
• [ ] code coverage measurement is presented on weekly basis

Common processes

• [ ] update/finish DoD (definition of done) document
• [ ] ensure that DoD is followed by all members of DevOps team

Stack analysis + component analysis

• [ ] Integration tests cover all API endpoints
• [ ] 100% integration test coverage for all supported packages/components (vertx, ...)
• [ ] Data in S3 in perfect state for all supported packages (stage == prod)
• [ ] Data in Graph DB in perfect state for all supported packages (stage == prod)
• [ ] UI e2e tests (from the new repo to stack analysis) [leo]
• [ ] stability of REST API (longer tests with lower frequency of API calls) [100% responses expected on prod]

CVEs

• [ ] Decide whether we include this feature in release/demo
• [ ] Select subset of CVEs for demo
• [ ] Ensure nearly 100% accuracy for selected CVEs
• [ ] Integration tests CVE DB ⇒ API
• [ ] UI tests API⇒ GUI

Packages/Gemini

• [ ] Decide whether we include this feature in release/demo
• [ ] Select subset of packages for demo
• [ ] Ensure nearly 100% accuracy for selected packages
• [ ] Integration tests CVE DB ⇒ API
• [ ] UI tests API⇒ GUI

Backends

• [ ] #1635 Ensure referential Integrity (RI) for the data stored in S3 database
• [ ] #1636 Ensure referential Integrity (RI) for the data stored in graph database
• [ ] #1144 OSIO analytics platform is scaled properly

Tech. deps (lower priority?)

• [ ] implement standard interface to the graph database that will be used by other services
• [ ] implement standard auth. module
• [ ] integration tests for the auth. module for all its API calls
• [ ] documentation for the REST API services, preferably in the repo itself (AsciiDoc + build script?)

(1) List of all repositories

1. fabric8-analytics-common
2. fabric8-analytics-server
3. fabric8-analytics-worker
4. fabric8-analytics-jobs
5. fabric8-analytics-tagger
6. fabric8-analytics-stack-analysis
7. fabric8-analytics-license-analysis
8. fabric8-analytics-data-model
9. fabric8-analytics-recommender
10. fabric8-analytics-auth
11. fabric8-gemini-server
12. f8a-server-backbone

[tisnik]

FYI: @krishnapaparaju @miteshvp @samuzzal-choudhury @msrb CC: @GeorgeActon

[msrb]

code coverage measurement is presented on weekly basis

I like this idea a lot. We could set up a job that would send an email to our mailing list once a week.

[krishnapaparaju]

@tisnik thanks for putting this list together. will plan (from #147), start working towards priority items.