search

sharc-lab / LightningSim

A fast, accurate trace-based simulator for High-Level Synthesis.
https://arxiv.org/abs/2304.11219
GNU Affero General Public License v3.0
38 stars 2 forks source link

Simulation Data Serialization / Report File Feature for CLI Tool #4

Open stefanpie opened 1 week ago

stefanpie commented 1 week ago

Currently, there is no way to run LightningSim from the CLI and have it output the simulation analysis results to a file on disk. This data is accessible via the Python API, but it requires adding LightningSim as a dependency and writing code that calls its functions to run the simulation and access data. This feature is intended for those who would like to use LightningSim as a standalone program that dumps the data to disk for further analysis by any means they wish.

I propose adding flags for --dump-json or --dump-ascii to output as much data as possible from the simulation analysis. Ideally, --dump-json should be as comprehensive as possible, including detailed data with high granularity, while --dump-ascii would provide a more text-based report format with summarized data, tables, and a breakdown of key metrics.

The LightningSim author has provided a gist with an example script that can be used in the meantime to dump simple data, specifically the start and stop cycles of modules in an HLS design using the print_details function: https://gist.github.com/ArkaneMoose/36f160b755e986387d5ae00e1bb17535.

ArkaneMoose commented 1 week ago

Current entry point is here: https://github.com/sharc-lab/LightningSim/blob/d3fb26b0388f9cebc5685eb942df2bfe0d8445f4/backend/lightningsim/main.py#L397-L436

This needs to be updated to run that CLI function linked above.

Also we probably want to run the CLI by default and make this a breaking change (e.g., add a --web flag to run the web interface).

ArkaneMoose commented 1 week ago

However, the CLI script doesn't have a way to run incremental simulation with changed FIFO depths (this is currently only accessible through the web interface or directly through the Python API). Right now, this is primarily because I don't have a way to dump the CompiledSimulation Python object to disk and load it back.

However, this shouldn't be too difficult: we should implement serialization of the CompiledSimulation Rust struct using serde and expose methods to Python to invoke serialization/deserialization to disk. (N.B.: the serialization format doesn't matter too much but we should avoid JSON; the graph structure is very lightweight and dumping to a human-readable text format will inflate the file size a lot for no real benefit.)

Once this is implemented, ideally we will have a simulation flow something like this:

$ lightningsim path/to/solution1 -o simulation.lsim --params parameters.json
Analyzing project...
Compiling project...
Running testbench...
Parsing schedule...
Resolving dynamic schedule from trace...
Wrote simulation database to simulation.lsim and default simulation parameters to parameters.json
Modify simulation parameters as desired and run: lightningsim simulation.lsim --params parameters.json

$ lightningsim simulation.lsim --params parameters.json -o output.json
Analyzing stalls...
Wrote simulation output to output.json

This can be a separate issue (feature request).