mpicbg-scicomp / gearshifft

Benchmark Suite for Heterogenuous FFT Implementations
Apache License 2.0
34 stars 9 forks source link

how to produce a help message beyond boost unit test framework #24

Closed psteinb closed 7 years ago

psteinb commented 8 years ago

how to produce the non-boost-utf help message?

$ ./gearshifft_fftw -- --help
Usage:
gearshifft_fftw [{--auto_start_dbg=|-d }[yes|y|no|n]] [--break_exec_path=<value>] [{--build_info=|-i }[yes|y|no|n]] [{--catch_system_errors=|-s }[yes|y|no|n]] [--detect_fp_exceptions=[yes|y|no|n]] [--detect_memory_leaks=<value>] [{--log_format=|-f }<value>] [{--log_level=|-l }<value>] [{--log_sink=|-k }<value>] [{--output_format=|-o }<value>] [{--random=|-a }[<value>]] [{--report_format=|-m }<value>] [{--report_level=|-r }<value>] [{--report_sink=|-e }<value>] [{--result_code=|-c }[yes|y|no|n]] [{--run_test=|-t }<value>] [--save_pattern=[yes|y|no|n]] [{--show_progress=|-p }[yes|y|no|n]] [--use_alt_stack=[yes|y|no|n]] [{--help=|-? }[yes|y|no|n]]
where:
auto_start_dbg - Automatically starts debugger if system level error (signal) occurs
break_exec_path - For the exception safety testing allows to break at specific execution path
build_info - Shows library build information
catch_system_errors - Allows to switch between catching and ignoring system errors (signals)
detect_fp_exceptions - Allows to switch between catching and ignoring floating point exceptions
detect_memory_leaks - Allows to switch between catching and ignoring memory leaks
log_format - Specifies log format
log_level - Specifies log level
log_sink - Specifies log sink:stdout(default),stderr or file name
output_format - Specifies output format (both log and report)
random - Allows to switch between sequential and random order of test units execution.
Optionally allows to specify concrete seed for random number generator
report_format - Specifies report format
report_level - Specifies report level
report_sink - Specifies report sink:stderr(default),stdout or file name
result_code - Allows to disable test modules's result code generation
run_test - Allows to filter which test units to run
save_pattern - Allows to switch between saving and matching against test pattern file
show_progress - Turns on progress display
use_alt_stack - Turns on/off usage of an alternative stack for signal handling
help - this help message

I'll dig deeper now.

psteinb commented 8 years ago

ok, we have to drop the utf auto-magic somehow to make this work, I found that if I provide:

$ ./gearshifft_x --help

I am not even landing in benchmark.cpp:48! this also holds for

$ ./gearshifft_x -- --help
psteinb commented 8 years ago

what we'd rather should do is (disclaimer: pseudo-code!)

int main(argc,argv){

auto argv_with_gearshift_flags_removed = parse_gearshift_command_line_args(...);

//handle gearshift args if needed

parse_boost_utf_flags(argv_with_gearshift_flags_removed);

//handle boost_utf_flags if needed 

configure_benchmark();

run_benchmark();

return 0;
}
psteinb commented 8 years ago

Example 9 in the boost.utf documentation illustrates how to go through main and setup the test suite/fixture from there

@tdd11235813 : do you think that would work?

tdd11235813 commented 8 years ago

Example 9 would work. I wondered, that you run into wrong help message. Just for the record, you should see:

$ ./gearshifft_clfft -- -h
gearshifft extra command line parameters
Options:
  -h [ --help ]                     Print help messages
<snip>
Test setup error: test tree is empty  <- will be removed after refactoring gearshifft-boost interface

The test tree is generated manually, using Boost test suite and test case, e.g. see here and here. The auto-magic you mentioned is the init_unit_test_suite function, which is hooked up by Boost behind the stage (including program options parsing). So this probably caused the false "landing". Your proposed pseudo-code is what I want, too.

I tried only few solutions, which hide the Boost framework from the benchmark programmer/user, but other things were more important that time, thus the unpleasant stuff came back into benchmark.cpp waiting for refactoring. --run_test=... is not an ideal program option to power benchmarks on, --list_content would be still ok. You already mentioned to get rid off separate zones of command line flags and I second that, to only have gearshifft program options.

tdd11235813 commented 7 years ago

I would close this now as gearshifft reached the wanted level.