Closed juhoinkinen closed 1 year ago
Patch coverage: 100.00
% and project coverage change: -0.01
:warning:
Comparison is base (
e9fd177
) 99.57% compared to head (08bf4ce
) 99.57%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
is_trained and modification_time should be nullable, but backend_id probably should not be:
Agree, backend_id should not be nullable. We should ensure that the API never returns data like that.
Kudos, SonarCloud Quality Gate passed!
0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells
No Coverage information
0.0% Duplication
is_trained and modification_time should be nullable, but backend_id probably should not be:
Agree, backend_id should not be nullable. We should ensure that the API never returns data like that.
I marked nobackend
project as private, which I think is enough for this PR.
I started to feel that maybe some of the manually written tests should have not been deleted.
Schemathesis does use the examples defined in the API schema (indirectly stated in the CLI part of the documentation for the --hypothesis-phases=explicit
option). This is true also when using it via pytest: I removed dummy-fi project from Annif test projects and 404 responses from the API schema, and this made those tests fail that use dummy-fi as an example project id in the schema. Reason for fails was shown to be
Received a response with a status code, which is not defined in the schema: 404
and in details "Project 'dummy-fi' not found"
.
But still this does not mean that e.g. the query parameters for /suggest-batch
method are correctly passed and processed in rest.py
. However it seems to need some amount of effort to make up an error that either tests in test_openapi.py
or test_rest.py
wont catch.
For testing the REST API there have been manually written tests since abandoning the unmaintained swagger-tester in #551. Already at that time Schemathesis was considered as a replacement of swagger-tester, but there was a dependency conflict that prevented using it.
Now Schemathesis can be installed to restore testing that the API actually conforms to the OpenAPI specification. Also the specification will now be validated.
With the default settings Schemathesis takes 12-15 s to run the tests in
tests/test_swagger.py
, but using@settings(max_examples=10)
only 1-2 s.~Due to a bug the pyproject.yaml includes pinned starlette installation (
starlette = ">=0.13,<0.21"
).~ Edit: Pinning starlette is not necessary when pinning to recent version Schemathesis3.18.*
.When fixing the errors noted by Schemathesis I marked some properties in the specification as nullable.
is_trained
andmodification_time
should be nullable, butbackend_id
probably should not be: https://github.com/NatLibFi/Annif/blob/6fc315763a395dff86420f3d1f07e3b3fe3ca869/annif/openapi/annif.yaml#L173 I marked this only to make the test pass. One project for the testing config is deliberately missing the backend specification: https://github.com/NatLibFi/Annif/blob/6fc315763a395dff86420f3d1f07e3b3fe3ca869/tests/projects.cfg#L70-L74I'm not really sure what to do for that.