[TC_28] NebulOuS manages the execution of IoT data processing pipelines

[jchmielewska]

Objective:

• Test if user can express an IoT data processing pipeline as a NebulOuS application and NebulOuS efficiently manages the execution of the pipeline.

Preconditions:

• NebulOuS core is running.
• Three NebulOuS nodes are active and registered in NebulOuS.
• Application of “IoT APP” is registered in NebulOuS. The APP SLO is configured such as max_age for “step1_output” can’t be bigger than 30 seconds. The objective function is set to minimize the number of instances of step 2.

Steps:

1. Action:
   • User logs in into NebulOuS web UI and launches an instance of the application “Scaling APP”.

Expected Result:

• NebulOuS deploys an instance of the APP step “step1”. No instance of step2 is created.

2. Action:
   • User publishes a message to the pub/sub queue “stream_input” of the application, with the body {“group”:”g1”,”t”:10}

Expected Result:

• NebulOuS detects that the max_age of “step1_output” queue is bigger than 30 and creates an instance of “step2”. Pending messages are processed and max_age for the queue goes to 0. NebulOuS stops the execution of “Step 2” instance.

3. Action:
   • User publishes a burst of 500 messages to the pub/sub queue “stream_input” of the application, with a fixed value for “t” (10) and different values for “group” (“g1”,”g2”,”g3”)

Expected Result:

• NebulOuS detects that the max_age of “step1_output” queue is bigger than 30 and creates an instance of “step2”.
• After a short period of time, NebulOuS reevaluates the SLO and confirms that the max_age is still bigger than 30 seconds and creates more instances of the “step2”.
• This goes on until all the pending messages are processed and max_age for the queue goes to 0.
• NebulOuS stops the execution of “Step 2” instances.

[jchmielewska]

• for automation