search

opengeospatial / ets-wcs20

Executable Test Suite for WCS 2.0.1
Other
1 stars 2 forks source link

Run OGC WCS 2.0.1 core standard take hours and not finish #108

Closed bangph closed 2 years ago

bangph commented 2 years ago

I used http://cite.opengeospatial.org/teamengine/test.jsp on Chrome to run just WCS 2.0.1 core standard for this endpoint http://conformance.rasdaman.com/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities

and it took more than 4 hours and still it haven't finished yet.

Now it is running at this test

Testing wcs2:get-kvp-core-req33 type Mandatory in Test Mode with defaultResult Pass

image

@dstenger do you have any idea for speeding it up? I recall, a few years ago, it could finish in like < 1 hour.

bangph commented 2 years ago

After 2 hours when I opened the ticket, and it is still stuck at:

` Testing wcs2:get-kvp-core-req33 type Mandatory in Test Mode with defaultResult Pass (s0140/d13242e65_1/d13242e193_1/d14560e260_1/d18331e350_1)...

              Assertion: req33: Send otherwise valid GetKVP GetCoverage requests with matching and with violating slicing positions to server under test. Check if appropriate results are returned if and only if the requirement is fulfilled, and an exception otherwise. Pass test if all checks succeed.

`

bangph commented 2 years ago

It really hangs up and cannot finish after 6 hours.

image

ghobona commented 2 years ago

@bangph Thank you for reporting the issue. We are looking into it.

dstenger commented 2 years ago

@bangph There have been several updates of the test suite in the last years as documented in http://cite.ogc.org/teamengine/about/wcs/2.0.1/site/relnotes.html. One of those changes may have an effect on the speed of the test suite.

Can you please document what options you used when executing the tests? Currently, the referenced URL leads to an empty test report.

bangph commented 2 years ago

@dstenger I only used WCS core (the biggest and slowest test for WCS 2.0.1). Years ago this took only ~30 minutes.

image

bangph commented 2 years ago

@dstenger any updates on that after 1 month? Thanks

dstenger commented 2 years ago

Unfortunately, I cannot reproduce your problem as the provided test service leads to an empty test result. Is this expected?

I don't think that there have been any updates to the test suite (see http://cite.ogc.org/teamengine/about/wcs/2.0.1/site/relnotes.html) making the performance worse or adding additional tests in the last years. Of course, there might have been changes on the server providing the test suite. Can you please retry running the tests as we recently increased the resource of the server?

dstenger commented 2 years ago

Also, I migrated this issue to https://github.com/opengeospatial/ets-wcs20/issues as it is related to the test suite/ infrastructure running the test suite.

dstenger commented 2 years ago

This issue is probably the reason for my previously mentioned problem reproducing the issue? #107 Is it possible to provide a service which can be used to reproduce the problem?

bangph commented 2 years ago

@dstenger I'm in my home country now (southest Asia), and http://cite.opengeospatial.org/teamengine/ is just not loadable after 10 minutes with error:

502 Bad Gateway nginx/1.18.0 (Ubuntu)

As I cannot access teamengine webpage to recheck the problem of this ticket, so here is the endpoint for you, please check that:

http://conformance.rasdaman.com/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities

NOTE: this endpoint disabled the extra INSPIRE metadata which caused problem in teamengine in https://github.com/opengeospatial/ets-wcs20/issues/107

bangph commented 2 years ago

@dstenger FYI: Interestingly, http://cite.opengeospatial.org/te2/ is very fast to access. It should be the case as well for http://cite.opengeospatial.org/teamengine/ (not loadable)

bangph commented 2 years ago

@dstenger I'm testing endpoint http://conformance.rasdaman.com/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities on http://cite.opengeospatial.org/te2, will let you know if it can finish

bangph commented 2 years ago

@dstenger It works fast! Under few minutes to run test on beta teamengine http://cite.opengeospatial.org/te2/viewOldSessionLog.jsp?session=s0108.

image

Please check http://conformance.rasdaman.com/rasdaman/ows?service=WCS&version=2.0.1&request=GetCapabilities on http://cite.opengeospatial.org/teamengine/ as I cannot access to that.

dstenger commented 2 years ago

On http://cite.opengeospatial.org/teamengine/ it also took just a few minutes to run the tests with your service. Can you please re-check if you can run the test suite on Production? If you still have problems, we have to check our infrastructure.

bangph commented 2 years ago

@dstenger thanks for rechecking. I can access to http://cite.opengeospatial.org/teamengine fast now and the tests run quickly which make me happy :).

It is much more better than I expected.