Run and session naming enhancements

[okt-kostik]

The problem we're trying to solve is that we have cases where we have say basic/foo that has 400 iterations and it's very hard to understand the structure of the iterations.

We can simplify things for the reader by using runs/sessions. However there are two problems:

People don't like naming runs

Right now run can be unnamed - this is not good, it would be good to have some kind of settings that enforces named runs in the test suite. Say a setting in tester.conf?

Sessions can't be named

It would be good if we support naming sesssions

How do we split run?

Then the question is how do we minimize copy-paste when splitting the run.

Say if we look at the netdriver:

  99         <run>
 100             <script name="mtu_tcp">
 101                 <req id="X3-SYS04"/>
 102             </script>
 103             <arg name="env" list="env_mtu">
 104                 <value ref="env.peer2peer"/>
 105                 <value ref="env.peer2peer"/>
 106                 <value ref="env.peer2peer"/>
 107                 <value ref="env.peer2peer"/>
 108                 <value ref="env.peer2peer"/>
 109                 <value ref="env.peer2peer"/>
 110                 <value ref="env.peer2peer_ipv6"/>
 111                 <value ref="env.peer2peer_ipv6"/>
 112                 <value ref="env.peer2peer_ipv6"/>
 113                 <value ref="env.peer2peer_ipv6"/>
 114                 <value ref="env.peer2peer_ipv6"/>
 115             </arg>
 116             <arg name="mtu" list="env_mtu">
 117                 <value>576</value>
 118                 <value>1280</value>
 119                 <value>1500</value>
 120                 <value>1600</value>
 121                 <!-- See NET-80 -->
 122                 <value>1948</value>
 123                 <value>9000</value>
 124                 <value>1280</value>
 125                 <value>1500</value>
 126                 <value>1600</value>
 127                 <!-- See NET-80 -->
 128                 <value>1948</value>
 129                 <value>9000</value>
 130             </arg>
 131             <arg name="tx" type="boolean" list="offloads">
 132                 <value>FALSE</value>
 133                 <value>TRUE</value>
 134                 <value>TRUE</value>
 135                 <value>TRUE</value>
 136                 <value>TRUE</value>
 137             </arg>
 138             <arg name="gso_on" type="boolean" list="offloads">
 139                 <value>FALSE</value>
 140                 <value>FALSE</value>
 141                 <value>TRUE</value>
 142                 <value>FALSE</value>
 143                 <value>TRUE</value>
 144             </arg>
 145             <arg name="tso_on" type="boolean" list="offloads">
 146                 <value>FALSE</value>
 147                 <value>FALSE</value>
 148                 <value>FALSE</value>
 149                 <value>TRUE</value>
 150                 <value>TRUE</value>
 151             </arg>
 152             <arg name="pkt_size">
 153                 <value>less</value>
 154                 <value>equal</value>
 155                 <value>more</value>
 156             </arg>
 157             <arg name="sends_num">
 158                 <value>100</value>
 159             </arg>
 160         </run>

We might group based on:

• ipv4/ipv6
• offload settings

I'm not sure how exactly.

For instance:

<run name=foo>:
   common params
   <run name=bar>
     specific params
   </run>

which works for ipv4 step. For the next one it's very painful, cause we will have to split "list" into 5 runs + hardcode the name. So if there are any cool ideas...

[ol-andrewr]

Right now run can be unnamed - this is not good, it would be good to have some kind of settings that enforces named runs in the test suite. Say a setting in tester.conf?

I dislike the idea since it will enforce duplication or some dummy naming. We have really many-many cases when one script is run only once.

[ol-andrewr]

It would be good if we support naming sesssions

What for? It is either root session of the package (so named) or enclosed in run (so named as well).

[ol-andrewr]

Then the question is how do we minimize copy-paste when splitting the run.

Just use session for common params. Basically it is a purpose of the session to solve your use-case. That's it. Don't try to break structure.