Is your feature request related to a problem? Please describe.
There is no testing strategy in place for grapevne modules (as opposed to the grapevne application which is tested). Modules are user-provided units of work that reside on a range of github repositories. The testing stategy should therefore be flexible and scalable.
Describe the solution you'd like
Modules should be self-testing to provide quality assurance and allow periodic checks. Standard entry point should be prescribed under the grapevne eco-system, with software solutions made available to install in repositories for users to test their own modules.
A workable solution would be the following:
Standard entry point for testing (module-side):
Provide a .test.sh script to execute any tests. This provides flexibility for module-specific factors, or could simply remove old output files before executing the tests.
The recommended approach to self-test is the following:
Provide a _test rule as part of the module's Snakemake definition. The _test rule can simply list the required inputs (output of the main rule(s)), and can include additional logic to validate the contents of these files.
Provide a config/.test.yaml file that is configured for testing purposes (include any results/in files that are required within the repository)
Provide repository mechanisms to launch module tests when files are modified (e.g. github actions)
This should isolate modules that have been added/amended and only run tests on the relevant modules (designed to run on PRs).
initially a script & a local github action definition file [migrating to a fully-fledged and installable github action for ease of installation]
This issue is largely resolved by https://github.com/kraemer-lab/vneyard/pull/26, although the testing strategy also needs to be recorded in the grapevne documentation. The documentation is due to be refreshed as part of #245 .
Is your feature request related to a problem? Please describe. There is no testing strategy in place for grapevne modules (as opposed to the grapevne application which is tested). Modules are user-provided units of work that reside on a range of github repositories. The testing stategy should therefore be flexible and scalable.
Describe the solution you'd like Modules should be self-testing to provide quality assurance and allow periodic checks. Standard entry point should be prescribed under the grapevne eco-system, with software solutions made available to install in repositories for users to test their own modules.
A workable solution would be the following:
.test.shscript to execute any tests. This provides flexibility for module-specific factors, or could simply remove old output files before executing the tests._testrule as part of the module's Snakemake definition. The_testrule can simply list the required inputs (output of the main rule(s)), and can include additional logic to validate the contents of these files.config/.test.yamlfile that is configured for testing purposes (include anyresults/infiles that are required within the repository)This issue is largely resolved by https://github.com/kraemer-lab/vneyard/pull/26, although the testing strategy also needs to be recorded in the grapevne documentation. The documentation is due to be refreshed as part of #245 .