rrnewton
commented
9 years ago Ok, I think we need to overhaul the whole bench set filtering business. I think we should follow the philosophy of building a datatype that captures the (composable) notion of benchspace filters we want to have.
Right now it's hacky and arbitrary it filters for a match on EITHER the "target" or the "progname". In your case it looks like matches against the "target" field for both of those.
But this doesn't fully explain the example you showed. Because it looks like "SyncCostBench" should NOT match either of those fields for the NumWarpsBench. However, the second message you posted doesn't seem consistent with the mkBenchmark code .... so if those came from the same run there is some really weird bug.
svenssonjoel
I have, amongst others, these two benchmarks. I try to run these with "WHICHBENCH=SyncCostBench" and "WHICHBENCH=SyncCostNumWarpsBench". But when the SyncCostBench runs, it also tries to run SyncCostNumWarps, but in the wrong directory.
[ mkBenchmark "SyncCostBench/SyncCostBench.cabal" [variant,show num_syncs] defaultCfgSpc | num_syncs <- [0..32] , variant <- ["SyncKern1"] ] ++ [ mkBenchmark "SyncCostNumWarpsBench/SyncCostNumWarpsBench.cabal" [variant,show num_warps] defaultCfgSpc | num_warps <- [1..32] , variant <- ["SYNC","NOSYNC"] ]
This is what happens:
Compiling Config 2 of 97: SyncCostNumWarpsBench.cabal (args "SYNC 1") confID "SyncCostBench_SyncCostNumWarpsBench_cabal"
[hsbencher] Found 1 methods that can handle SyncCostBench/SyncCostNumWarpsBench.cabal: ["cabal"] run_benchmarks.exe: getDir: benchmark target path does not exist at all:
I have been digging around in the HSBencher code, but could not pinpoint what might cause this issue.
If it is by design, I think it is highly unintuitive. Or, I need to learn a more robust way of telling benchmarks apart by some other way of naming them uniquely. (please advice)