search

ConSol-Lab / gourd

a command-line tool for configuring, running, and analysing algorithm comparison experiments on supercomputers
4 stars 0 forks source link

Adding custom metrics to the experiment report #12

Open k-sidorov opened 3 months ago

k-sidorov commented 3 months ago

Could you point out a way to introduce custom metrics from the post-processing job into the experiment report? My common use case for post-processing with legacy Gourd involved parsing stdout/stderr for some program-specific output (e.g., objective value from a solver or some internal statistics, such as the number of search tree nodes) and printing it as a CSV string. I have tried doing the same with both post-processing and afterscripts, but without much success; if there is an alternative mechanism for that in the new Gourd, I would appreciate the hint!

mgazeel commented 3 months ago

Welcome to the gourd GitHub!

Postprocessing is currently being reworked as soon as we merge #10 and document the new system we will describe how to collect custom metrics.

In the current version of gourd we believe that this is impossible (other than manually parsing the outputs).