This project contains tests that can be used for any Beacon implemention using api version 1.0.1 and 1.1.0. The Beacon's responses are validated against the GA4GH's OpenAPI specification and against JSON schemas by CSCfi. Apart from this, the values returned by the beacon are also checked.
The project uses openapi-core and jsonschemas.
Requirements:
python3
andpip3
.
To run, install dependencies:
pip3 install -r requirements.txt
Then run tests from a test file:
python3 beacon_api_tester.py --host http://localhost:5050 --test tests/test-v101-variants.yaml
This assumes that you have a beacon running on the host
specified. The beacon should contain the test data
specified in the tests. By running
python3 beacon_api_tester.py --show_data_files tests/test-v101-variants.yaml
you can see the file names containing the data needed. See more about test data below.
Each test specifies which data is assumes, in the field beacondata
.
Currently, all test data is found in testdata.csv (v101) and testdata_mate.csv (v110). The data is given in comma separated format.
v1.0.1:
The info
(/
) endpoint answer. Json validation and the datasets in the answer.
Queries that are not allowed. Check that these return code 400
.
See tests/v101/test_errors.py
The structure and answers of some specific queries. See tests/test-v101-variants.yaml
.
Having multiple datasets.
v1.1.0:
To create more tests, see adding_tests.md. This file also gives more information on how to interpret each given test file. Contributions (in the form of pull requests) are very welcome!
Validation options
Default: validate against the OpenAPI spec, the JSON schemas and check that the results match (eg. that we get the correct counts).
Other options:
--no_openapi
Don't validate against the OpenAPI specification
--no_json
Don't validate against the JSON Schemas
--only_structure
Don't validate the resulting counts, frequencies etc.
Output options
Default: show information, warnings and error messages.
Other options:
--only_warn
Only print warnings and errorsOther
Default: Beacons are 0-based.
Other options:
--one_based
Test a beacon that is 1-basedThe OpenAPI specification can be downloaded from
the GA4GH's GitHub repo.
To use a local version of it, specify it's path in config/config.py
:
# Path to the OpenAPI spec
SPEC = 'beacon.yaml'
The JSON schemas can be downloaded from
the CSCfi's GitHub repo.
To use local versions, put them in a directory and specify it's path in config/config.py
:
# Directory containing JSON schemas
SCHEMAS = 'schemas'