Test stops immediately

[vinsworldcom]

In the 04.templates demo, when I run the test, it "stops" after the utils.mgen wrapper shows its output presumably to let the eventservice inject all the EEL events from the scenario.eel file.

I created a copy of that directory, made some adjustments - including changing the radio model and adding some other wrappers and now when I run the test, it blazes through the steps.xml file right to the end and stops the test. It doesn't seem to "pause" to let the EEL file be injected by eventservice.

My steps.xml file:

<steps>
  <step name="emane.start">
    <run wrapper="emane.emane"/>
  </step>
  <step name="routing.start">
    <run wrapper="utils.olsrd"/>
  </step>
  <step name="run">
    <run wrapper="emane.emaneeventd"/>
    <run wrapper="emane.emaneeventservice"/>
    <run wrapper="utils.gpsd"/>
    <run wrapper="mywrapper.gps2radio"/>
    <run wrapper="mywrapper.pointAnt"/>
  </step>
  <step name="stats.snapshot">
    <run wrapper="emane.emaneshsnapshot"/>
  </step>
</steps>

I've "fixed" it by adding:

  <step name="pause">
    <run wrapper="utils.sleepwait">
      <arg name="sleepseconds" value="300"/>
    </run>
  </step>

after the "mywrapper.pointAnt" closing </step>. But this is less than ideal.

How does ETCE "know" to pause to let an entire scenario.eel file be run before moving on to the next step? What is different in the 04.templates that I copied that isn't working in my example?

Cheers.

[eschreiber-alink]

Each step listed in steps.xml completes when each of the associated wrappers finishes across all of the nodes. One of the things to keep in mind when thinking about this is whether the application underlying each wrapper is executed as a daemon.

For example, emane is launched by this step in 04.templates steps.xml:

  <step name="emane.start">
    <run wrapper="emane.emane"/>
  </step>

This step returns immediately because emane is run as a daemon (the --daemonize command line argument is passed to it) and returns immediately when executed.

The run step contains several wrapper calls:

  <step name="run">
    <run wrapper="emane.emaneeventd"/>
    <run wrapper="emane.emaneeventservice"/>
    <run wrapper="utils.gpsd"/>
    <run wrapper="utils.mgen"/>
  </step>

emaneeventservice is also executed as a daemon so is not blocking the return of this step. The eventservice wrapper exports the option to run as a daemon (default, run as daemon):

[me@host]$ etce-wrapper list emane.emaneeventservice -v
-----------------------
emane.emaneeventservice
-----------------------
description:

    Run emaneeventservice with the provided configuration file.

input file name:
    eventservice.xml
output file name:
    emaneeventservice.log
arguments:
    daemonize
        Run as daemon [True, False].
        default: True
    loglevel
        log level - [0,3]
        default: 3

MGEN prevents the run step from returning immediately because the utils.mgen wrapper runs MGEN in the foreground. MGEN is one of the few applications that ETCE executes in the foreground:

[me@host]$ etce-wrapper list utils.mgen -v
----------
utils.mgen
----------
description:

    Execute mgen with the specified script. This wrapper blocks
    until mgen completes. 

input file name:
    mg0.0 JOIN 225.1.20.1 INTERFACE bmf0
en.script
output file name:
    mgen.log

Take a look at the mgen.script file used in the demo:

[me@host]$ cat 04.templates/node.tpl/mgen.script 
TTL 64

0.0 LISTEN UDP 5001

0.0 JOIN 225.1.20.1 INTERFACE bmf0

1.0 ON ${mgen_flow} UDP SRC 5001 DST 225.1.20.1/5001 PERIODIC [1 512] INTERFACE bmf0

Each node listens on port 5001, joins a multicast groups and starts an MGEN traffic flow at T=1.0. Because the script omits the associated terminating actions for each of these commands, MGEN never exits and the run step never completes. This is typical for an ETCE demonstration where we want to leave the emulation up until the user is finished with it.

More frequently, we run ETCE in a batch mode that executes dozens of tests sequentially. In those tests, the MGEN scripts contains sentences that allow it to finish. The script above can be modified so that MGEN completes and returns at T=40.0:

TTL 64

0.0 LISTEN UDP 5001

0.0 JOIN 225.1.20.1 INTERFACE bmf0

1.0 ON ${mgen_flow} UDP SRC 5001 DST 225.1.20.1/5001 PERIODIC [1 512] INTERFACE bmf0

31.0 OFF ${mgen_flow}

40.0 LEAVE 225.1.20.1 INTERFACE bmf0

40.0 IGNORE UDP 5001

In a sequence of tests, each test blocks until its mgen script completes and then proceeds to the next test.

[vinsworldcom]

Thanks for the detailed explanation. That makes sense and is what I was gathering in troubleshooting it. Basically, a wrapper daemonize() will return immediately and the steps will continue to process. A wrapper run() won't return until the command that is 'run' is finished.

And your "This is typical for an ETCE demonstration where we want to leave the emulation up until the user is finished with it" is exactly what we're looking for in some cases and spawned this issue / question.

We've been using the utils.sleepwait wrapper to keep the demo up since there isn't another wrapper (e.g., utils.mgen) that is run() to block step processing. Understanding this is how it works, we could probably create another utils.pause wrapper or such to just block until the user wants to continue.

Thanks!

[eschreiber-alink]

If you are using emaneventservice for your demonstration, you might experiment with just setting it's daemonize option to false. One of the tutorial demos shows how to set wrapper arguments in steps.xml. I believe emaneeventservice does not terminate when run in the foreground, even once it consumes all of the EEL file sentences. You may need to reorder emaneeventservice last wrapper called in the run step if your helper node is executing other applications in that step.

Also, just to clarify - most wrappers do call the run and daemonize WrapperContext methods to execute the underlying program. The daemonize method is a helper function to try to daemonize applications that don't natively provide a daemonize option. If you look at the emane.py or emaneeventservice.py wrapper, you'll see they call run because emane and emaneeventservice provide the --daemonize command line argument. Most of the wrappers are written to prefer the native application daemonization.

[vinsworldcom]

emaneeventservice in the foreground is interesting - never considered that. I'll give it a try.

Thanks for the pointers about run and daemonize. I wrote a few wrappers that need to run Python commands in the background so I've used daemonize to do that: mywrapper.gps2radio and mywrapper.pointAnt as examples in the original question, which is why that step was non-blocking - they daemonized to the background and the steps continued on.