Check that roadrunner respects the llvm optimization pass options.

[CiaranWelsh]

The RoadRunner::Config class has a set of options that are intended to control llvm code generation optimizations - that is, which passes are applied sequentially to produce faster llvm IR code. These passes are expensive operations: they take about ~36% of the time in a simpe "build and simulate" program:

    bool opt = false;
    Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_CFG_SIMPLIFICATION, opt);
    Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_DEAD_CODE_ELIMINATION, opt);
    Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_GVN, opt);
    Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_INSTRUCTION_COMBINING, opt);
    Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_INSTRUCTION_SIMPLIFIER, opt);
    Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_CFG_SIMPLIFICATION, opt);
    RoadRunner roadRunner(OpenLinearFlux().str());
    roadRunner.simulate(0, 100, 101);

Turning these optimizations off should speed the process of model generation, which is what we want when we are dealing with lots of models. However, running the profiler tool again with these optimizations turned off produces very similar (the same?) results leading me to think that these options are not really doing anything. Therefore, we should check to see whether roadrunner actually respects these options.

[matthiaskoenig]

Just to comment on the benchmarking. You have to use benchmark models which require a lot of work to simulate, i.e. at least in the range of model loading. I saw your benchmarks of speedup 3x for parallelization for simple stochastic simulations of S1 -> S2; J0 = k0*S1. Such models will not do for benchmarking, because the model simulation is not really expensive. You need large models with complex math to see optimization results. I assume you did the same here. If you don't have anything complex in the model and very easy simulations you will not notice much/any improvements.

[matthiaskoenig]

Forgot: You will really need a real benchmarking suite to also avoid regressions in speed. Trying to optimize with toy models without seeing the impact on a larger set of real-world models is very dangerous.

[CiaranWelsh]

Yep - these are valid points. These little models are only really designed for unit testing. I tried again with the Brown 2004 model. That is, build and simulate the brown 2004 model in a loop 1000 times, but adding a unique species each time to prevent roadrunner from caching the model. In this case, we do observe around a 15-30 second speed up when turning off the additional llvm IR optimization passes, which makes sense to me.

However, I do still wonder if there is still something not quite right because even when these optimization flags are set to off, the profiler indicates that a significant amount of the time is spent in the llvm::legacy::PassManager - suggesting these flags aren't doing what I think they should be doing.

Agreed with your benchmarking point. This is something I will get around to in the near future as I'll be comparing different llvm backend configurations.

[hsauro]

Some of these optimizations may not apply to expressions but to whole programs that use loops etc. In order to work it has to find something to actually simplify.

Herbert

On Mon, Oct 25, 2021 at 2:47 AM Ciaran Welsh @.***> wrote:

The RoadRunner::Config class has a set of options that are intended to control llvm code generation optimizations - that is, which passes are applied sequentially to produce faster llvm IR code. These passes are expensive operations: they take about ~36% of the time in a simpe "build and simulate" program:

bool opt = false;
Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_CFG_SIMPLIFICATION, opt);
Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_DEAD_CODE_ELIMINATION, opt);
Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_GVN, opt);
Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_INSTRUCTION_COMBINING, opt);
Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_INSTRUCTION_SIMPLIFIER, opt);
Config::setValue(Config::LOADSBMLOPTIONS_OPTIMIZE_CFG_SIMPLIFICATION, opt);
RoadRunner roadRunner(OpenLinearFlux().str());
roadRunner.simulate(0, 100, 101);

Turning these optimizations off should speed the process of model generation, which is what we want when we are dealing with lots of models. However, running the profiler tool again with these optimizations turned off produces very similar (the same?) results leading me to think that these options are not really doing anything. Therefore, we should check to see whether roadrunner actually respects these options.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/sys-bio/roadrunner/issues/898, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIBSDQUAQL7HYWTMK3QU6DUIURSFANCNFSM5GU2PYOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Herbert Sauro, Professor University of Washington, Bioengineering 206-685-2119, www.sys-bio.org @.*** Books: http://books.analogmachine.org/