Implement baseline testing - Githubissues

princeton-ddss / blackfish

Machine learning as a service (MLaaS) for busy researchers

https://princeton-ddss.github.io/blackfish/

0 stars 0 forks source link

Implement baseline testing #13

Open cswaney opened 3 months ago

cswaney commented 3 months ago

Testing that involves interacting with Slurm will generally require mocking.

User tests

Manually confirm the following processes work as expected on supported systems (Linux, macOS).

[x] Setup
[x] Slurm job (local host)
[x] Slurm job (remote host)

Unit tests

These tests only require the package code to available. Mocking used to simulate systems/dependencies as needed.

[x] jobs (w/ mocking)
[ ] config (w/ mocking for BlackfishProfileStore)
[ ] base.Service.__init__
[ ] base.Service.open_tunnel
[ ] base.Service.close_tunnel
[ ] models.nlp.TextGeneration.__init__
[ ] models.nlp.TextGeneration.launch_script
[ ] cli.__main__ (w/ mocking)

Integration tests

These tests require the package to be installed and an application to be running locally. Mock data ("fixtures") used as needed.

[ ] base.Service.refresh
[ ] base.Service.stop
[ ] base.Service.start
[ ] cli.__main__