search

conjure-cp / conjure-oxide

Mozilla Public License 2.0
9 stars 16 forks source link

Performance testing for #428 #445

Open EEJDempster opened 1 week ago

EEJDempster commented 1 week ago

This is the beginnings of #428, building on Nik's get_solutions_from_conjure to also get the stats.json files from conjure.

Done:

In progress:

To Do:

Will talk to Oz in tomorrows (todays) meeting to discuss how this should be run and what exactly the result should be.

github-actions[bot] commented 3 days ago

Code and Documentation Coverage Report

Documentation Coverage

Click to view documentation coverage for this PR ``` +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | crates/conjure_macros/src/lib.rs | 2 | 66.7% | 1 | 33.3% | +-------------------------------------+------------+------------+------------+------------+ | Total | 2 | 66.7% | 1 | 33.3% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | crates/tree_morph/src/commands.rs | 1 | 100.0% | 0 | 0.0% | | crates/tree_morph/src/helpers.rs | 0 | 0.0% | 0 | 0.0% | | crates/tree_morph/src/lib.rs | 1 | 100.0% | 1 | 100.0% | | crates/tree_morph/src/reduce.rs | 2 | 100.0% | 0 | 0.0% | | crates/tree_morph/src/reduction.rs | 0 | 0.0% | 0 | 0.0% | | crates/tree_morph/src/rule.rs | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 4 | 40.0% | 1 | 33.3% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | ...m_compatability_macro/src/lib.rs | 2 | 100.0% | 1 | 50.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 2 | 100.0% | 1 | 50.0% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | solvers/kissat/src/lib.rs | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | solvers/minion/src/ast.rs | 11 | 11.0% | 0 | 0.0% | | solvers/minion/src/error.rs | 8 | 100.0% | 0 | 0.0% | | solvers/minion/src/lib.rs | 1 | 100.0% | 1 | 100.0% | | solvers/minion/src/run.rs | 2 | 100.0% | 1 | 100.0% | | solvers/minion/src/wrappers.rs | 1 | 100.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 23 | 20.5% | 2 | 11.8% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | crates/conjure_core/src/ast/atom.rs | 1 | 33.3% | 0 | 0.0% | | .../conjure_core/src/ast/domains.rs | 0 | 0.0% | 0 | 0.0% | | ...jure_core/src/ast/expressions.rs | 27 | 93.1% | 0 | 0.0% | | ...conjure_core/src/ast/literals.rs | 1 | 33.3% | 0 | 0.0% | | crates/conjure_core/src/ast/mod.rs | 0 | 0.0% | 0 | 0.0% | | ...ure_core/src/ast/symbol_table.rs | 0 | 0.0% | 0 | 0.0% | | ...es/conjure_core/src/ast/types.rs | 0 | 0.0% | 0 | 0.0% | | ...onjure_core/src/ast/variables.rs | 1 | 50.0% | 0 | 0.0% | | crates/conjure_core/src/bug.rs | 1 | 50.0% | 1 | 50.0% | | crates/conjure_core/src/context.rs | 0 | 0.0% | 0 | 0.0% | | crates/conjure_core/src/error.rs | 1 | 14.3% | 0 | 0.0% | | crates/conjure_core/src/lib.rs | 0 | 0.0% | 0 | 0.0% | | crates/conjure_core/src/metadata.rs | 0 | 0.0% | 0 | 0.0% | | crates/conjure_core/src/model.rs | 3 | 17.6% | 0 | 0.0% | | ...core/src/parse/example_models.rs | 2 | 100.0% | 0 | 0.0% | | ...es/conjure_core/src/parse/mod.rs | 0 | 0.0% | 0 | 0.0% | | ...re_core/src/parse/parse_model.rs | 0 | 0.0% | 0 | 0.0% | | ...jure_core/src/rule_engine/mod.rs | 5 | 71.4% | 5 | 71.4% | | ...src/rule_engine/resolve_rules.rs | 3 | 100.0% | 0 | 0.0% | | ..._core/src/rule_engine/rewrite.rs | 2 | 66.7% | 0 | 0.0% | | ...ure_core/src/rule_engine/rule.rs | 3 | 25.0% | 1 | 100.0% | | ...core/src/rule_engine/rule_set.rs | 4 | 100.0% | 0 | 0.0% | | ...njure_core/src/rules/constant.rs | 1 | 100.0% | 0 | 0.0% | | ...es/conjure_core/src/rules/mod.rs | 1 | 100.0% | 0 | 0.0% | | ...re/src/solver/adaptors/kissat.rs | 1 | 100.0% | 0 | 0.0% | | ...re/src/solver/adaptors/minion.rs | 1 | 100.0% | 0 | 0.0% | | ..._core/src/solver/adaptors/mod.rs | 1 | 100.0% | 0 | 0.0% | | ...s/conjure_core/src/solver/mod.rs | 14 | 33.3% | 1 | 4.2% | | ...ore/src/solver/model_modifier.rs | 7 | 70.0% | 0 | 0.0% | | ...onjure_core/src/solver/states.rs | 7 | 63.6% | 0 | 0.0% | | ...es/conjure_core/src/stats/mod.rs | 0 | 0.0% | 0 | 0.0% | | ...core/src/stats/rewriter_stats.rs | 1 | 20.0% | 0 | 0.0% | | ...e_core/src/stats/solver_stats.rs | 3 | 37.5% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 91 | 42.1% | 8 | 9.8% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | conjure_oxide/src/defaults.rs | 1 | 50.0% | 0 | 0.0% | | conjure_oxide/src/find_conjure.rs | 1 | 50.0% | 0 | 0.0% | | conjure_oxide/src/lib.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/conjure.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/json.rs | 2 | 66.7% | 0 | 0.0% | | conjure_oxide/src/utils/misc.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/mod.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/testing.rs | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 4 | 12.1% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ ```
Click to view documentation coverage for main ``` +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | ...m_compatability_macro/src/lib.rs | 2 | 100.0% | 1 | 50.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 2 | 100.0% | 1 | 50.0% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | crates/conjure_macros/src/lib.rs | 2 | 66.7% | 1 | 33.3% | +-------------------------------------+------------+------------+------------+------------+ | Total | 2 | 66.7% | 1 | 33.3% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | crates/tree_morph/src/commands.rs | 1 | 100.0% | 0 | 0.0% | | crates/tree_morph/src/helpers.rs | 0 | 0.0% | 0 | 0.0% | | crates/tree_morph/src/lib.rs | 1 | 100.0% | 1 | 100.0% | | crates/tree_morph/src/reduce.rs | 2 | 100.0% | 0 | 0.0% | | crates/tree_morph/src/reduction.rs | 0 | 0.0% | 0 | 0.0% | | crates/tree_morph/src/rule.rs | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 4 | 40.0% | 1 | 33.3% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | solvers/kissat/src/lib.rs | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | solvers/minion/src/ast.rs | 11 | 11.0% | 0 | 0.0% | | solvers/minion/src/error.rs | 8 | 100.0% | 0 | 0.0% | | solvers/minion/src/lib.rs | 1 | 100.0% | 1 | 100.0% | | solvers/minion/src/run.rs | 2 | 100.0% | 1 | 100.0% | | solvers/minion/src/wrappers.rs | 1 | 100.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 23 | 20.5% | 2 | 11.8% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | crates/conjure_core/src/ast/atom.rs | 1 | 33.3% | 0 | 0.0% | | .../conjure_core/src/ast/domains.rs | 0 | 0.0% | 0 | 0.0% | | ...jure_core/src/ast/expressions.rs | 27 | 93.1% | 0 | 0.0% | | ...conjure_core/src/ast/literals.rs | 1 | 33.3% | 0 | 0.0% | | crates/conjure_core/src/ast/mod.rs | 0 | 0.0% | 0 | 0.0% | | ...ure_core/src/ast/symbol_table.rs | 0 | 0.0% | 0 | 0.0% | | ...es/conjure_core/src/ast/types.rs | 0 | 0.0% | 0 | 0.0% | | ...onjure_core/src/ast/variables.rs | 1 | 50.0% | 0 | 0.0% | | crates/conjure_core/src/bug.rs | 1 | 50.0% | 1 | 50.0% | | crates/conjure_core/src/context.rs | 0 | 0.0% | 0 | 0.0% | | crates/conjure_core/src/error.rs | 1 | 14.3% | 0 | 0.0% | | crates/conjure_core/src/lib.rs | 0 | 0.0% | 0 | 0.0% | | crates/conjure_core/src/metadata.rs | 0 | 0.0% | 0 | 0.0% | | crates/conjure_core/src/model.rs | 3 | 17.6% | 0 | 0.0% | | ...core/src/parse/example_models.rs | 2 | 100.0% | 0 | 0.0% | | ...es/conjure_core/src/parse/mod.rs | 0 | 0.0% | 0 | 0.0% | | ...re_core/src/parse/parse_model.rs | 0 | 0.0% | 0 | 0.0% | | ...jure_core/src/rule_engine/mod.rs | 5 | 71.4% | 5 | 71.4% | | ...src/rule_engine/resolve_rules.rs | 3 | 100.0% | 0 | 0.0% | | ..._core/src/rule_engine/rewrite.rs | 2 | 66.7% | 0 | 0.0% | | ...ure_core/src/rule_engine/rule.rs | 3 | 25.0% | 1 | 100.0% | | ...core/src/rule_engine/rule_set.rs | 4 | 100.0% | 0 | 0.0% | | ...njure_core/src/rules/constant.rs | 1 | 100.0% | 0 | 0.0% | | ...es/conjure_core/src/rules/mod.rs | 1 | 100.0% | 0 | 0.0% | | ...re/src/solver/adaptors/kissat.rs | 1 | 100.0% | 0 | 0.0% | | ...re/src/solver/adaptors/minion.rs | 1 | 100.0% | 0 | 0.0% | | ..._core/src/solver/adaptors/mod.rs | 1 | 100.0% | 0 | 0.0% | | ...s/conjure_core/src/solver/mod.rs | 14 | 33.3% | 1 | 4.2% | | ...ore/src/solver/model_modifier.rs | 7 | 70.0% | 0 | 0.0% | | ...onjure_core/src/solver/states.rs | 7 | 63.6% | 0 | 0.0% | | ...es/conjure_core/src/stats/mod.rs | 0 | 0.0% | 0 | 0.0% | | ...core/src/stats/rewriter_stats.rs | 1 | 20.0% | 0 | 0.0% | | ...e_core/src/stats/solver_stats.rs | 3 | 37.5% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 91 | 42.1% | 8 | 9.8% | +-------------------------------------+------------+------------+------------+------------+ +-------------------------------------+------------+------------+------------+------------+ | File | Documented | Percentage | Examples | Percentage | +-------------------------------------+------------+------------+------------+------------+ | conjure_oxide/src/defaults.rs | 1 | 50.0% | 0 | 0.0% | | conjure_oxide/src/find_conjure.rs | 1 | 50.0% | 0 | 0.0% | | conjure_oxide/src/lib.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/conjure.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/json.rs | 2 | 66.7% | 0 | 0.0% | | conjure_oxide/src/utils/misc.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/mod.rs | 0 | 0.0% | 0 | 0.0% | | conjure_oxide/src/utils/testing.rs | 0 | 0.0% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ | Total | 4 | 12.5% | 0 | 0.0% | +-------------------------------------+------------+------------+------------+------------+ ```

Code Coverage Summary

This PR: Detailed Report

  lines......: 75.2% (3760 of 4998 lines)
  functions..: 58.6% (378 of 645 functions)
  branches...: no data found

Main: Detailed Report

  lines......: 75.4% (3760 of 4987 lines)
  functions..: 60.1% (378 of 629 functions)
  branches...: no data found

Coverage Main & PR Coverage Change

Lines coverage changed by -0.20% and covered lines changed by 0
Functions coverage changed by -1.50% and covered lines changed by 0
Branches... coverage: No comparison data available