search

rapp-project / rapp-platform

RAPP Platform is a collection of ROS nodes and back-end processes that aim to deliver ready-to-use generic services to robots
Other
28 stars 20 forks source link

Integration tests - Ensure that web services and ROS services are deployed before execution #327

Open klpanagi opened 8 years ago

klpanagi commented 8 years ago

This issue is mostly related to Continues Integration. Executing integration-tests must ensure that the RAPP Platform was previously fully deployed in order to prevent from getting either HTTP request or ROS Service request errors.

For RAPP Platform scalability purposes, this should not be resolved using sleeping methods because sleeping time values depends both on the physical machine hosting the RAPP Platform and the load of the RAPP Platform (ROS Nodes, Web Services).

etsardou commented 8 years ago

Tried to assess this issue like this.

I used a simple countdown timer in each test class (essentially for each web service), which polls the web service till it is up. I think using a timer with sleep is correct, since we must have an upper time limit while waiting for the services or else the call will hang forever. A drawback of this approach is that the countdown will exist for each inner test (e.g. in this file we will have 9 countdowns). We could check all the services in the rapp_testing_core.py file but this creates dependencies. Using the current approach each test file checks only its own web service.

In this example I am performing an erroneous call (just to get the response quickly) and I check if I get a connection error. The problem I found is this:

In the case when the web services are not up, the code executes fine and the timer times out correctly, raising a unittest assertion. If I launch the web services during the count down, the call immediately does not return connection error since it sees that the IP/Port combination is up, nevertheless, this does not mean that the specific web service is yet up, giving this response and causing all tests to fail:

.404 Client Error: Not Found for url: http://155.207.19.229:9001/hop/cognitive_test_chooser ('Connection aborted.', BadStatusLine("''",))

klpanagi commented 8 years ago

It is expected to have this behavior. Like you mentioned, connection error is different than HTTP 404. We have to poll the service's url path until it is fully initialized and accessible through the www. Using timers for polling, until we get an active connection, will ensure that we always get an active connection no matter of the time needed for the Platform to launch.