search

opengeospatial / ets-sta10

Repository for the Executable Test Suite for OGC Sensor Things API
Other
6 stars 8 forks source link

Server-driven paging neglected #9

Closed mjacoby closed 8 years ago

mjacoby commented 8 years ago

The whole test suite does not take server-driven paging into account, i.e. when ever a request to the server is made with an possible answer with more than one item the test suite must expect to do multiple requests (following the nextLinks) until it can be sure that all data is received.

The best way to see this is by setting the server-driven page size to 1 and see that multiple test cases will fail (e.g. Capability3Tests.checkSkipForEntityType(...) ).

liangsteve commented 8 years ago

Server-driven pagination is configured by the service provider and cannot be configured. How can the CITE test set the server-driven page size to 1?

mjacoby commented 8 years ago

You are right, but i did suppose that you have any kind of server implementation you are developing and testing the Test Suite against and thus could change the server-driven page size to 1 inside some test environment of yours and run the Test Suite against this server so that you can see this issue at a glance.

My main statement was, that the Test Suite is not taking into account server-driven paging but rather assumes that the server-driven page size is big enough so that all requests get the full result directly with the first service call.

For example in the mentioned method Capability3Tests.checkSkipForEntityType(...) a HTTP GET request to http://example.com/v1.0/Observations?$skip=1 is made and is expected to return 11 observations. The test does not check if the response contains a nextLink (i.e. server-driven pagination happened) and it must fetch further results with the consequence that if the server-side page size is smaller than 11 the test will fail.

This problem is not limited to the mentioned method but rather is a general problem of the Test Suite that will occur in multiple methods and tests. Therefore, my hint to you to find all methods/tests that are affected by this issue faster than reading through all the code of the Test Suite could be to run the Test Suite against a server with a page size of 1 if you have to possibility to do so.

taniakhalafbeigi commented 8 years ago

The precondition for this test suite is that the service under test should have a small number of each entity type in SensorThings before the test starts. In other words, the service under test needs to have at least one entity for each entity type and the number of entities for each entity type must be less that the pagination limit. It means that we can access the entities of each entity type without the need to follow the @iot.nextLink. I just added this to the documentation of the Test Suite for clarification. For other levels of testing the data is added to the service by the Test Suite itself. The number of entities that are added to the service under test is fairly small and we assume that the paging number is more than 20 which is reasonable for accessing frequent sensor observations without the need to follow the @iot.nextLink every time.