search

Open-EO / openeo-backend-validator

Service to validate back-end compliance with the API specification.
Apache License 2.0
1 stars 3 forks source link

Only run openeo_compliance_tests self-tests from travis #26

Closed soxofaan closed 5 years ago

soxofaan commented 5 years ago

It doesn't make sense to do validation tests of actual backends from travis/github

the pytest test suite has some helpers/internals that still can be tested though

soxofaan commented 5 years ago

Case in point: because the original test run fails in practice (each backend has some compliance issues), I didn't bother checking the travis output and didn't notice that there was a typo in the requirements.txt file. Hence #27

m-mohr commented 5 years ago

I always found it very valuable to have a UI where I can check the results of my back-end without setting up a whole Python environment, this project etc.

soxofaan commented 5 years ago

I understand. Although in the longer term we should probably make a better workflow/UI than the travis page of openeo-backend-validator. (E.g. you probably want the backend to be tested daily/weekly instead of only when there was a code push on the validator code)

In the mean time however we could maybe play with something like https://docs.travis-ci.com/user/build-matrix/#matching-jobs-with-allow_failures to avoid having a "broken" validator github project because some backends are not completely compliant

m-mohr commented 5 years ago

Sure! Maybe we can combine it with the Go interface in the future? @bgoesswe

bgoesswe commented 5 years ago

Yes, I agree, I already plan to integrate the python framework into the webtool, so that the validation calls the python test cases and the manually added endpoints of the Go tool.

m-mohr commented 5 years ago

Replaced by #30.