SPIKE: Strategy for full test coverage on Riff EDU

[adonahue]

We have a business goals to have a commercial version of Riff EDU available in March 2020. A commercial product must - among other things - be scale-able and maintainable, and so reaching 70 -80% unit test coverage for the EDU product is a pre-requisite for commercialization.

The goal of test coverage is to be able to introduce changes to Riff EDU and reduce the likelihood (with high confidence) that we have not introduced new bugs.

Practices vary in terms of when a test suite should be run, if it's automated as part of a build, etc. Part of the spike should be to propose a strategy that makes sense for the Riff team.

Test coverage should be for all aspects of the product code:

• Riff video
• Riff dashboard
• Riff Server
• Mattermost client (using theirs as much as possible)
• Mattermost server (using theirs as much as possible)

Written Spike Findings: https://docs.google.com/document/d/1y6hIj0eI-rmKopz5BxnHtuuoyjEkjf3KYPCTDS4_UV0/edit#heading=h.85kslhepv6

Story Acceptance Criteria A short, written recommendation to be shared and discussed with the team that takes into account the following:

• Using existing and newly released Mattermost tests for their code.
• Tools and frameworks to run tests.
• A strategy that fits with our current transition to packaged dashboard and video code, that will be shared between Riff EDU and Riff platform.
• A recommendation for developer practices to run tests, manual, automated, on check-in, on build, etc.
• Allow us to have different branches or products that we accept as non-conformant.
• Is able to revert to a previous test state, in a previous version of the product, in order to be able to test and release a patch for existing releases, when necessary. (Assumes a formal version control system will be in place).
• How to divide and sequence the work for the various portions of the code base.

[jaedoucette]

Something that was not fully clear to us:

100% coverage is probably not the desired goal (that quality is used only for NASA-grade software).

We think the goal is "get all the old mattermost tests working and passing again, and add new tests for all the new code written by Riff".

Also not clear to us what the reversion / version control criteria is about. We think we already have that capability, and the branching capability.

[jaedoucette]

Notes from sprint planning:

-> 70-80% coverage goal -> Need a building plan to, for all the repos.

[adonahue]

@jaedoucette @jordanreedie @mlippert @brecriffs - this document is super-helpful, thank you! I put a bunch of comments and questions in the doc, please take a look. @ebporter - please review and comment as well, asap. I think we should probably plan to meet later next week to through more of the details on how to approach this work.

[jaedoucette]

@adonahue Sorry, forgot to attach this to the card too! The document is here:

https://docs.google.com/document/d/1y6hIj0eI-rmKopz5BxnHtuuoyjEkjf3KYPCTDS4_UV0/edit?usp=sharing

[juliariffgit]

@adonahue, Can this one move to the completed column?

[adonahue]

@juliariffgit - added this to "last sprint" since that's when the work was completed, but move it to "this sprint" if you prefer to have it there.

[juliariffgit]

@adonahue, "previous sprints" is perfect for this. Thanks!