Additional options to buildtest build for how buildtest processes YAML files

shahzebsiddiqui commented 4 years ago

This issue is to expand capability to #168

The objective of this issue is to provide the following

Allow user to list content of YAML file. It could be full content or brief output of important metadata (testname, description, maintainer, testtype, etc...)
Provide users with options for name inclusion & exclusion when passing test configuration. Useful during directory traversal
Pass multiple test configuration separated by comma

Listing YAML content

Add option --list and short option -l (eLL)

Proposed Output

$ buildtest build -c hello.f.yml --list
NAME        SUBTEST             DESCRIPTION         
hello.f.yml   hello-world         Hello World Fortran

The subtest is the top-level key in YAML file recall that multiple tests can be defined in one YAML file.

Resolve Path

Add option -r or long option --resolve to print full path to test configuration.

Proposed Output:

$ buildtest build -c hello.f.yml -r
NAME          ABSOLUTE PATH
------------------------------------
hello.f.yml    ~/.buildtest/site/github.com/HPC-buildtest/tutorials/compilers/hello.f.yml

Should work for directory or relative path as well.

$ buildtest build -c github.com/HPC-buildtest/tutorials/compilers/ -r
NAME          ABSOLUTE PATH
------------------------------------
hello.f.yml    ~/.buildtest/site/github.com/HPC-buildtest/tutorials/compilers/hello.f.yml
hello.c.yml    ~/.buildtest/site/github.com/HPC-buildtest/tutorials/compilers/hello.c.yml
hello.cxx.yml    ~/.buildtest/site/github.com/HPC-buildtest/tutorials/compilers/hello.cxx.yml

It doesn't make sense to use -r if absolute path is specified on file or directory but it should print it out anyways. When print the NAME field, just use os.path.basename() to get the file name.

Name Inclusion & Exclusion

The name inclusion and exclusion are meant to filter test-configs to give user freedom on pick what test-configs will be processed and built. This feature is useful when multiple tests processed.

Add option -i or long option --include and -x or long option --exclude all of them are of type string.

Inclusion example: buildtest build -c github.com/HPC-buildtest/tutorials/compilers/ -i hello.f.yml Exclusion Example: buildtest build -c github.com/HPC-buildtest/tutorials/compilers/ -x hello.f.yml

When both inclusion & exclusion are specified, buildtest should ignore exclusion flag all together for instance if user specifies:

buildtest build -c github.com/HPC-buildtest/tutorials/compilers/ -i hello.f.yml -x hello.f.yml

The -x option is ignored and it is equivalent to the Inclusion example defined above

Inclusion and Exclusion should only be check when directory is specified (relpath or abspath)

Specifying multiple test via comma

User can specify multiple test separated by comma. It can be used to specify multiple files or directory or a mix of both.

Multiple Files:

buildtest build -c hello.f.yml,hello.c.yml

Multiple Directory (Relpath):

buildtest build -c github.com/HPC-buildtest/tutorials/compilers/,github.com/HPC-buildtest/tutorials/cuda/

File and Directory (Relpath):

buildtest build -c hello.f.yml,github.com/HPC-buildtest/tutorials/cuda/

Likewise both relpath and abspath for file or directory can be used.

Other Minor Notes

Since buildtest has several checks on how test names can be specified it is important to note a few details

1) When specifying a file that is not absolute path, buildtest should get the first occurrence during directory traversal and stop. For example if user specifies buildtest build -c hello.f.yml and hello.f.yml is found in two sub-directory lets say stampede/compilers/hello.f.yml and lonestar/compilers/hello.f.yml then buildtest should get whichever file was traversed first.

2) The comma syntax should work with all options including inclusion -i, exclusion -x, resolve -r, and listing -l

3) If buildtest is unable to find any YAML files during the search a message should be printed out and buildtest should terminate immediately.

4) If user specifies any invalid path (file or directory) then don't exit but ignore the processing and exit only if no files found (rule 3). The reason is if user specifies one file and directory, buildtest should continue even if one of them is invalid. In example below, buildtest should ignore path /invalid/path and process hello.f.yml.

Example:

buildtest build -c hello.f.yml,/invalid/path/

vsoch commented 4 years ago

I’m glad that you documented these notes! However this should not be worked on until the core functionality is done. This is adding features to something undeveloped, and I’m not convinced they would be requested by users.

shahzebsiddiqui commented 4 years ago

I agree 100%.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

buildtesters / buildtest