WIP: Introducing the `grcli` development tool

[mabruzzo]

This PR is ~90% complete.

Overview/Motivation

This PR introduces a new development tool called grcli. It is a command-line tool, written in C++, that can be used to configure and execute fairly arbitrary grackle-sample problems. Currently, the primary purpose is to use it for benchmarking.

At some point in the near future (in a different PR), it is my intention to add support for debugging.[^1]

This tool makes use of Google's benchmark library (I'm not 100% sure this is the right tool for the job). The CMake build system is configured to automatically download and build that library as part of compiling grcli.

Description

The outputs of grcli --help are displayed down below (in the collapsible section

grcli description

``` NAME: grcli - A command line interface for benchmarking and debugging Grackle SYNOPSIS: grcli [--version] [--precision] [-h | --help] bench -S... PARAMETER_OPTIONS -O... grcli [--version] [--precision] [-h | --help] show-parameters PARAMETER_OPTIONS grcli [--version] [--precision] [-h | --help] show-initial-units DESCRIPTION: A simple command-line tool intended for debugging and basic benchmarking of grackle. We summarize the subcommands down below. Be note that the current organization is experimental and is subject to change -> bench sets up an example problem and benchmarks it -> show-parameters is intended to display the configured parameters. -> show-initial-units is intended to display the initial units used during configuration of the grackle-solver. In the future, we may add support for customizing these units and using a different set of units during Grackle operations OPTIONS: Some Flags are grouped into categories -S... Prefix for all "scenario arguments". The "scenario" is the algorithm for initializing all grackle fields that are used in a "test-problem." At the moment, this might include * -Sgrid.ax=, Where the quantities in brackets get replaced by various things. For example: * is replaced with 0, 1, or 2 * is replaced with temperature, density, metallicity * is replaced by arguments that specify a sequence of 1 or more values. In practice, this is either a lone floating-point value OR geomspace,,, -O... Prefix for all "operation arguments". Essentially, this denotes the operation that is executed in a given "test-problem." Currently, you can exclusively specify * -Ocalc-cooling-time Computes the cooling time * -Ocalc-dust-temperature Computes the dust-temperature * -Ocalc-pressure Computes the pressure * -Ocalc-temperature Computes the temperature * -Osolve-chemistry-dt= Solves chemistry for the specified timestep (given in code-units). -h; --help Prints this message --version Prints the version of Grackle --precision Prints the floating-point precision selected while compiling Grackle PARAMETER OPTIONS --par-start Every "word" after this option (until the --par-stop option) is parsed as a key-value pair that is used as a grackle parameter. --par-stop The sentinel value that must follow --par-start ```

Here's an example invocation:

Currently, there isn't any way to adjust the code_units. But that's something that can be introduced

./build/src/grcli/grcli bench \
    --par-start metal_cooling=1 use_grackle=1 primordial_chemistry=0 \
    'grackle_data_file="input/CloudyData_UVB=HM2012.h5"' \
    --par-stop \
    -Sgrid.ax0=temperature,geomspace,1e4,1e6,128 \
    -Sgrid.ax1=metallicity,1.0 \
    -Sgrid.ax2=mass_density,geomspace,0.01,10.0,128 \
    -Osolve-chemistry-dt=1e-6

Here's the output on my Mac Book Pro with an M3 Pro processor (I think the output probably looks nicer and is more informative on Linux):

Unable to determine clock rate from sysctl: hw.cpufrequency: No such file or directory
This does not affect benchmark measurements, only the metadata output.
***WARNING*** Failed to set thread affinity. Estimated CPU frequency may be incorrect.
2024-08-07T23:51:08-04:00
Running ./build/src/grcli/grcli
Run on (12 X 24.0001 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB
  L1 Instruction 128 KiB
  L2 Unified 4096 KiB (x12)
Load Average: 1.76, 1.82, 1.98
-----------------------------------------------------
Benchmark           Time             CPU   Iterations
-----------------------------------------------------
scenario      4736382 ns      4723905 ns          148

This translates to a case where the timestep is ~1e-6 Myr.

The following collapsed section shows a snippets of the outputs for calculate_temperature and calculate_cooling_time.

other timings

Calculate cooling time: ``` Load Average: 1.53, 1.65, 1.81 ----------------------------------------------------- Benchmark Time CPU Iterations ----------------------------------------------------- scenario 4680909 ns 4674407 ns 150 ``` Calculate temperature: ``` Load Average: 1.87, 1.82, 1.92 ----------------------------------------------------- Benchmark Time CPU Iterations ----------------------------------------------------- scenario 3330047 ns 3329395 ns 210 ``` This suggests, that up to ~70% of the cooling-time calculation is spent computing temperature (for primordial_chemistry == 0)

ToDo List

There are essentially 3 main things left to do:

• [ ] explore compilation without the benchmark library
• [ ] improve integration with the benchmark library (allow us to properly forward arguments to the benchmark library)
• [ ] consider adding support for "chemistry-data presets" (so that we don't need to specify some really common arguments, like metal_cooling=1 use_grackle=1 every time we run a test-problem

[^1]: In more detail, PR #197 seeks to add debugging tools to dump the state of grackle to disk. That PR introduces some python functions to read in the dumped data. The idea would be to add functionality to grcli to do the same.