Define guidelines for recipes/test_modules/tools to be accepted upstream

[pazdera]

We would like to build an upstream library of tests. It would be nice to have a set of guidelines to follow when people would like to contribute their recipes to us.

The key requirements are (in my opinion):

• As much platform independence as possible
• As much stuff implemented in TestModules as possible
• Clear directory structure
• As much code reuse as possible
• A way of executing the whole library of tests at once easily, skipping those we cannot execute with current setup
• Define well-known parameters of machines and interfaces

[pazdera]

This is likely to take a while. I'm removing it from the current milestone. It will probably be an ongoing process rather than a one-time thing.

[jtluka]

For directory structure I have a first requirement that needs to be respected because of the current design.

The controller supports running a directory of recipes. If this directory contains all of the xml reused code (mixed recipes, tasks, interfaces) running the controller on such directory would fail since it expects only files with as the root tag. Currently we have such directory in repo: recipes/team/*

So, to have a valid directory for runs like these the directory has to have following (or similar) structure:

./recipe_for_featureX/            # this directory's immediate children are valid lnstrecipes or directories, e.g.
./recipe_for_featureX/recipe1.xml
./recipe_for_featureX/recipe2.xml
./recipe_for_featureX/recipe3.xml
./recipe_for_featureX/networks/          # this directory contains network definitions for the recipe, e.g.
./recipe_for_featureX/networks/network-A.xml
./recipe_for_featureX/networks/network-B.xml
./recipe_for_featureX/networks/network-C.xml
./recipe_for_featureX/tasks/               # this directory contains tasks to be performed on a network
./recipe_for_featureX/tasks/task-J.xml
./recipe_for_featureX/tasks/task-K.xml
./recipe_for_featureX/tasks/task-L.xml

Subdirectories are not processed recursively so the directory above should be safe to process:

# lnst-ctl run ./recipe_for_featureX

[olichtne]

That looks like a good idea. I would see this as creating something like recipe packages, it would also be an interesting idea to expand these also contain custom test tools and test modules used by this package. That way if someone created a recipe package and uses some custom test resources it wouldn't be necessary to download them separately (and manually put them in a recognized path) or for the person to wait for their inclusion into our git tree.

This could be done either by lnst recognizing some well known directory names for test tools/modules or we could add a feature to support loading additional config files when a recipe directory is given to lnst.

Let me know what you think about this idea.