search

InformaticsMatters / fragnet

Fragment network tooling
Apache License 2.0
1 stars 0 forks source link

A more invasive test.sh #10

Closed alanbchristie closed 4 years ago

alanbchristie commented 5 years ago

At the moment test.sh simply tests that the REST query returns something. As we've now discovered that the results for a key molecule have changed (since last using their 2018 data) the test should be harder: -

  1. For each supplier we should run an availability query for a specific molecule
  2. The results should be checked against a list of results (that should include the smiles, supplier and code). If the results do not match expectations then it's an error. It's an error to have new result rows as well as finding that expected result rows are missing.

This should ideally be configured by a map embedded in the ansible file, with a separate test file for each supplier? And the test script need to insist that the user provide the list of suppliers expected in the deployment. If the list fo suppliers does not match that's also an error.

The end result is a test, run by a user who expects supplier "a" and "b", which errors if the data is anything but supplier "a" and "b" and where the molecule results do not match a pre-defined test for that supplier.

If the test fails the user then has to decide whether it's Bec cause the data has changed (legitimately) or if there's something wrong.

alanbchristie commented 4 years ago

This issue is now resolved and handled by the playbooks in the fragnet-ansible repo.