qgis / QGIS-Server-CertifSuite

Automated test suite to run teamEngine OGC tests against QGIS server
2 stars 7 forks source link

Nightly tests down? #24

Open pathmapper opened 8 months ago

pathmapper commented 8 months ago

From https://github.com/qgis/QGIS-Server-CertifSuite#qgis-server-certifsuite

QGIS server is tested every night against OGC test suites

http://test.qgis.org/ogc_cite/wms_130/latest/report.html is from January 2023

grafik

haubourg commented 8 months ago

Hi @pathmapper , thanks for raising this. @pblottiere 🏉

pblottiere commented 8 months ago

Hello @pathmapper,

Given that OGC tests are running properly in QGIS CI for some time now (3 years?), I didn't put a lot of effort into the nightly tests this year...

I can revive these nightly tests if necessary, but considering that you're the first one to notice since January, I'm still wondering ^^.

pathmapper commented 8 months ago

Hi @pblottiere, @haubourg,

thank you for putting all the work into the OGC conformance testing!

considering that you're the first one to notice since January, I'm still wondering ^^

Right, and only because the QGIS CI was failing unrelated recently :-)

I also think there is no need for nightly testing of WMS 1.3.0 anymore because it's checked regularly on QGIS CI.

With regards to nightly testing of

Looks like OGC API Features 1.0 is only passing on QGIS CI because the tests are pinned to ogccite/ets-ogcapi-features10:1.0-teamengine-5.4 which is outdated.

Tests for WFS 1.1.0 currently are not running on QGIS CI (makes no sense until the tests are passing).

Until OGC API Features 1.0 and WFS 1.1.0 are running with latest OGC tests on Github CI, it would be handy to have tests with QGIS master to know what needs to be fixed to reach that ultimate goal.

IMHO these must not be necessarily automated nightly tests - a documented way how to do it manually would also fit this use case.