Various list of additions for future README updates:
[ ] Include user should expect critter to pass a single extra argument in argv containing a string that critter needs (and that critter will receive implicitly as part of its interception of the MPI_Init symbol)NO LONGER NEEDED
[x] Include remove of all binaries in bin/ before "tar -cvf"
[x] Include match algorithm name in instructions.py with naming conventions specified in experiments/libraries
[x] Include plot names can be specified in instructions.py under the arguments to critter for each test
[ ] Include number of iterations must be specified as part of the input parameters of the test.cpp (and correspondingly, to the instructions.py file). This is necessary for critter's (future) ability to require no modifications to user test.cpp files. NO LONGER NEEDED
[x] Include due to scaplot assumptions, users can only specify node counts that range in a multiplicative manner.. This might want to be made into a separate issue.
[ ] Include `there is the assumption that if the user specifies something like File1 = [["critter",()],["perf",("Performance","Residual","Deviation from Orthogonality")]] inside instructions.py, that all algorithm variants across all libraries present in the test pass in the necessary performance, residual, and deviation from orthogonality arguments in a single call (not necessary if not wanted). NO LONGER NEEDED
[x] Include User also has to implement the interface for libraries (under experiments/libraries and machines (under experiments/machines)
[ ] Include User cannot have much else after the loop over all iterations, else the computation timer get corrupted. NO LONGER RELEVANT
[x] If user does not want to use the critter scripts and only wants raw critter output, they must unset the CRITTER_STATUS environment variable. Otherwise, manually set the CRITTER_STATUS environment variable.
[x] Don't have any '+' characters in the file path to critter, nor in any of the arguments. critter uses this character to separate out parse file strings and assign arguments.
[x] After building the libraries necessary, move all binaries into the bin/ folder, and match up the binary name with the name used in the first argument to the corresponding algorithm class instance specified in the instructions file.
[x] Make sure your (user's) binaries are not named test_allreduce or similar
[ ] What about allowing the user to run the benchmarks on only a subset of the node,ppn,tpr counts that the actual instructions script is used for? It may be even more easy for the user to run a designated benchmark run to get the data and save it, and then run their actual tests. The reason I'm concerned and that something should be done is that these tests don't scale. Even when testing only 5 (the ones used in cacqr2), 32 nodes take 40 minutes, 16 nodes takes 30, and 8 nodes takes around 25.
Various list of additions for future README updates:
user should expect critter to pass a single extra argument in argv containing a string that critter needs (and that critter will receive implicitly as part of its interception of the MPI_Init symbol)
NO LONGER NEEDEDremove of all binaries in bin/ before "tar -cvf"
match algorithm name in instructions.py with naming conventions specified in experiments/libraries
plot names can be specified in instructions.py under the arguments to critter for each test
number of iterations must be specified as part of the input parameters of the test.cpp (and correspondingly, to the instructions.py file).
This is necessary for critter's (future) ability to require no modifications to user test.cpp files. NO LONGER NEEDEDdue to scaplot assumptions, users can only specify node counts that range in a multiplicative manner.
. This might want to be made into a separate issue.User also has to implement the interface for libraries
(underexperiments/libraries
and machines (underexperiments/machines
)User cannot have much else after the loop over all iterations, else the computation timer get corrupted
. NO LONGER RELEVANTCRITTER_STATUS
environment variable. Otherwise, manually set theCRITTER_STATUS
environment variable.'+'
characters in the file path tocritter
, nor in any of the arguments.critter
uses this character to separate out parse file strings and assign arguments.bin/
folder, and match up the binary name with the name used in the first argument to the correspondingalgorithm
class instance specified in the instructions file.test_allreduce
or similarnode,ppn,tpr
counts that the actual instructions script is used for? It may be even more easy for the user to run a designatedbenchmark
run to get the data and save it, and then run their actual tests. The reason I'm concerned and that something should be done is that these tests don't scale. Even when testing only 5 (the ones used incacqr2
), 32 nodes take 40 minutes, 16 nodes takes 30, and 8 nodes takes around 25.Keep appending to this issue.