search

Wikunia / ConstraintSolver.jl

ConstraintSolver in Julia: Blog posts ->
https://opensourc.es/blog/constraint-solver-1
MIT License
135 stars 13 forks source link

Activity based search #181

Closed Wikunia closed 3 years ago

Wikunia commented 3 years ago

First try of #96

codecov[bot] commented 3 years ago

Codecov Report

Merging #181 (24769e6) into master (c6ff146) will increase coverage by 0.37%. The diff coverage is 97.94%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #181      +/-   ##
==========================================
+ Coverage   96.07%   96.44%   +0.37%     
==========================================
  Files          36       36              
  Lines        3181     3352     +171     
==========================================
+ Hits         3056     3233     +177     
+ Misses        125      119       -6
Impacted Files Coverage Δ
src/constraints/less_than.jl 100.00% <ø> (+3.44%) :arrow_up:
src/objective.jl 95.23% <ø> (-0.97%) :arrow_down:
src/tablelogger.jl 96.84% <ø> (-0.10%) :arrow_down:
src/type_inits.jl 94.44% <ø> (ø)
src/ConstraintSolver.jl 98.48% <94.00%> (-0.24%) :arrow_down:
src/MOI_wrapper/MOI_wrapper.jl 91.07% <96.29%> (+1.88%) :arrow_up:
src/branching.jl 99.47% <99.27%> (-0.53%) :arrow_down:
src/constraints/equal_set.jl 95.55% <100.00%> (+0.04%) :arrow_up:
src/logs.jl 95.32% <100.00%> (+0.04%) :arrow_up:
src/options.jl 100.00% <100.00%> (ø)
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c6ff146...24769e6. Read the comment docs.

Wikunia commented 3 years ago

Benchmark Report for ConstraintSolver

Job Properties

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["eternity", "5x5_all"] 0.24 (5%) :white_check_mark: 0.36 (1%) :white_check_mark:
["eternity", "5x5_opt"] 0.32 (5%) :white_check_mark: 0.32 (1%) :white_check_mark:
["eternity", "5x5_opt_ind"] 0.38 (5%) :white_check_mark: 0.38 (1%) :white_check_mark:
["eternity", "5x5_opt_rei"] 1.17 (5%) :x: 1.19 (1%) :x:
["eternity", "6x5"] 1.30 (5%) :x: 1.38 (1%) :x:
["graph_coloring", "US_50colors"] 0.44 (5%) :white_check_mark: 0.70 (1%) :white_check_mark:
["graph_coloring", "US_50colors+equal"] 0.57 (5%) :white_check_mark: 0.80 (1%) :white_check_mark:
["graph_coloring", "US_8+equal"] 0.57 (5%) :white_check_mark: 0.77 (1%) :white_check_mark:
["killer_sudoku", "niall_5500"] 3.85 (5%) :x: 3.69 (1%) :x:
["killer_sudoku", "niall_5500_special"] 18.89 (5%) :x: 14.90 (1%) :x:
["killer_sudoku", "niall_5501"] 30.99 (5%) :x: 20.61 (1%) :x:
["killer_sudoku", "niall_5501_special"] 5.49 (5%) :x: 3.79 (1%) :x:
["lp", "issue_83"] 0.93 (5%) :white_check_mark: 0.97 (1%) :white_check_mark:
["sudoku", "top95_1"] 0.87 (5%) :white_check_mark: 0.97 (1%) :white_check_mark:
["sudoku", "top95_11"] 6.52 (5%) :x: 6.99 (1%) :x:
["sudoku", "top95_16"] 4.62 (5%) :x: 5.03 (1%) :x:
["sudoku", "top95_21"] 0.90 (5%) :white_check_mark: 1.01 (1%) :x:
["sudoku", "top95_26"] 0.87 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_31"] 0.44 (5%) :white_check_mark: 0.51 (1%) :white_check_mark:
["sudoku", "top95_36"] 0.87 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_41"] 0.72 (5%) :white_check_mark: 0.81 (1%) :white_check_mark:
["sudoku", "top95_46"] 2.77 (5%) :x: 2.99 (1%) :x:
["sudoku", "top95_51"] 2.14 (5%) :x: 2.34 (1%) :x:
["sudoku", "top95_56"] 0.90 (5%) :white_check_mark: 1.02 (1%) :x:
["sudoku", "top95_6"] 0.87 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_61"] 1.06 (5%) :x: 1.17 (1%) :x:
["sudoku", "top95_66"] 0.60 (5%) :white_check_mark: 0.69 (1%) :white_check_mark:
["sudoku", "top95_71"] 0.88 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_76"] 0.88 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_81"] 5.68 (5%) :x: 5.96 (1%) :x:
["sudoku", "top95_86"] 1.78 (5%) :x: 1.99 (1%) :x:
["sudoku", "top95_91"] 0.58 (5%) :white_check_mark: 0.66 (1%) :white_check_mark:

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

Julia versioninfo

Target

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.7.11-arch1-1 #1 SMP PREEMPT Wed, 29 Jul 2020 21:38:21 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1648 MHz     350296 s      13138 s      14631 s    5249360 s       1105 s

  Memory: 46.99065017700195 GB (45029.9296875 MB free)
  Uptime: 4692.0 sec
  Load Avg:  1.2412109375  1.0732421875  0.947265625
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)

Baseline

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.7.11-arch1-1 #1 SMP PREEMPT Wed, 29 Jul 2020 21:38:21 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1596 MHz     414126 s      13138 s      16665 s    5961749 s       1263 s

  Memory: 46.99065017700195 GB (45186.10546875 MB free)
  Uptime: 5341.0 sec
  Load Avg:  1.06640625  1.09033203125  1.0205078125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)
Wikunia commented 3 years ago

Some killer sudokus are not actually solved in the time limit. Next up is trying to initialize the activity using probing.

Wikunia commented 3 years ago

Benchmark Report for ConstraintSolver

Job Properties

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["eternity", "5x5_all"] 0.55 (5%) :white_check_mark: 0.88 (1%) :white_check_mark:
["eternity", "5x5_opt"] 0.74 (5%) :white_check_mark: 0.75 (1%) :white_check_mark:
["eternity", "5x5_opt_ind"] 2.17 (5%) :x: 2.13 (1%) :x:
["eternity", "5x5_opt_rei"] 4.47 (5%) :x: 4.44 (1%) :x:
["eternity", "6x5"] 1.11 (5%) :x: 1.31 (1%) :x:
["graph_coloring", "US_50colors"] 1.49 (5%) :x: 2.37 (1%) :x:
["graph_coloring", "US_50colors+equal"] 1.91 (5%) :x: 2.72 (1%) :x:
["graph_coloring", "US_8+equal"] 2.19 (5%) :x: 1.84 (1%) :x:
["killer_sudoku", "niall_5500"] 15.59 (5%) :x: 13.48 (1%) :x:
["killer_sudoku", "niall_5500_special"] 8.54 (5%) :x: 6.94 (1%) :x:
["killer_sudoku", "niall_5501"] 6.71 (5%) :x: 6.33 (1%) :x:
["killer_sudoku", "niall_5501_special"] 5.60 (5%) :x: 4.22 (1%) :x:
["lp", "issue_83"] 2.67 (5%) :x: 2.38 (1%) :x:
["sudoku", "top95_1"] 0.90 (5%) :white_check_mark: 0.97 (1%) :white_check_mark:
["sudoku", "top95_11"] 44.27 (5%) :x: 45.29 (1%) :x:
["sudoku", "top95_16"] 45.10 (5%) :x: 46.36 (1%) :x:
["sudoku", "top95_21"] 0.93 (5%) :white_check_mark: 1.01 (1%) :x:
["sudoku", "top95_26"] 0.90 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_31"] 40.07 (5%) :x: 41.98 (1%) :x:
["sudoku", "top95_36"] 0.90 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_41"] 74.72 (5%) :x: 82.79 (1%) :x:
["sudoku", "top95_46"] 96.17 (5%) :x: 95.53 (1%) :x:
["sudoku", "top95_51"] 58.54 (5%) :x: 59.24 (1%) :x:
["sudoku", "top95_56"] 36.28 (5%) :x: 36.94 (1%) :x:
["sudoku", "top95_6"] 0.90 (5%) :white_check_mark: 0.98 (1%) :white_check_mark:
["sudoku", "top95_61"] 84.71 (5%) :x: 86.41 (1%) :x:
["sudoku", "top95_66"] 23.01 (5%) :x: 24.49 (1%) :x:
["sudoku", "top95_71"] 74.67 (5%) :x: 84.23 (1%) :x:
["sudoku", "top95_76"] 95.74 (5%) :x: 96.35 (1%) :x:
["sudoku", "top95_81"] 84.14 (5%) :x: 81.86 (1%) :x:
["sudoku", "top95_86"] 164.54 (5%) :x: 91.31 (1%) :x:
["sudoku", "top95_91"] 89.65 (5%) :x: 92.52 (1%) :x:

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

Julia versioninfo

Target

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.7.11-arch1-1 #1 SMP PREEMPT Wed, 29 Jul 2020 21:38:21 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1672 MHz      83682 s       1331 s       8881 s    3190062 s        480 s

  Memory: 46.99065017700195 GB (45080.66015625 MB free)
  Uptime: 2739.0 sec
  Load Avg:  1.37158203125  1.03173828125  0.64501953125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)

Baseline

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.7.11-arch1-1 #1 SMP PREEMPT Wed, 29 Jul 2020 21:38:21 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1826 MHz     148403 s       1331 s      10905 s    3909554 s        615 s

  Memory: 46.99065017700195 GB (45178.19921875 MB free)
  Uptime: 3394.0 sec
  Load Avg:  1.34033203125  1.2236328125  0.966796875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)
Wikunia commented 3 years ago

The problem seems to be that the same variable is chosen over and over again which doesn't seem optimal. Maybe it should also decay? I think this needs more experiments and changes.

Wikunia commented 3 years ago

Benchmark Report for ConstraintSolver

Job Properties

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["eternity", "5x5_all"] 0.71 (5%) :white_check_mark: 1.09 (1%) :x:
["eternity", "5x5_opt"] 1.16 (5%) :x: 1.21 (1%) :x:
["eternity", "5x5_opt_ind"] 2.82 (5%) :x: 2.85 (1%) :x:
["eternity", "5x5_opt_rei"] 5.40 (5%) :x: 5.36 (1%) :x:
["eternity", "6x5"] 0.77 (5%) :white_check_mark: 0.94 (1%) :white_check_mark:
["graph_coloring", "US_50colors"] 8.46 (5%) :x: 6.87 (1%) :x:
["graph_coloring", "US_50colors+equal"] 9.20 (5%) :x: 7.69 (1%) :x:
["graph_coloring", "US_8+equal"] 9.48 (5%) :x: 5.21 (1%) :x:
["killer_sudoku", "niall_5500"] 9.38 (5%) :x: 9.07 (1%) :x:
["killer_sudoku", "niall_5500_special"] 31.89 (5%) :x: 17.03 (1%) :x:
["killer_sudoku", "niall_5501"] 203.83 (5%) :x: 104.46 (1%) :x:
["killer_sudoku", "niall_5501_special"] 6.93 (5%) :x: 4.14 (1%) :x:
["lp", "issue_83"] 4.49 (5%) :x: 4.23 (1%) :x:
["sudoku", "top95_11"] 99.91 (5%) :x: 92.40 (1%) :x:
["sudoku", "top95_16"] 104.02 (5%) :x: 94.48 (1%) :x:
["sudoku", "top95_31"] 91.33 (5%) :x: 85.37 (1%) :x:
["sudoku", "top95_41"] 171.74 (5%) :x: 168.66 (1%) :x:
["sudoku", "top95_46"] 217.12 (5%) :x: 191.97 (1%) :x:
["sudoku", "top95_51"] 133.53 (5%) :x: 119.86 (1%) :x:
["sudoku", "top95_56"] 91.11 (5%) :x: 82.61 (1%) :x:
["sudoku", "top95_61"] 188.77 (5%) :x: 169.54 (1%) :x:
["sudoku", "top95_66"] 61.39 (5%) :x: 56.13 (1%) :x:
["sudoku", "top95_71"] 171.88 (5%) :x: 171.38 (1%) :x:
["sudoku", "top95_76"] 223.43 (5%) :x: 200.66 (1%) :x:
["sudoku", "top95_81"] 191.17 (5%) :x: 166.27 (1%) :x:
["sudoku", "top95_86"] 273.67 (5%) :x: 171.94 (1%) :x:
["sudoku", "top95_91"] 198.17 (5%) :x: 181.85 (1%) :x:

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

Julia versioninfo

Target

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.3-arch1-1 #1 SMP PREEMPT Sun, 01 Nov 2020 12:58:59 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1768 MHz     107072 s       1423 s       7414 s    1459280 s        361 s

  Memory: 46.98865509033203 GB (46071.79296875 MB free)
  Uptime: 1314.0 sec
  Load Avg:  1.10791015625  1.0302734375  0.7900390625
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)

Baseline

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.3-arch1-1 #1 SMP PREEMPT Sun, 01 Nov 2020 12:58:59 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1576 MHz     170381 s       1423 s       9590 s    2170684 s        521 s

  Memory: 46.98865509033203 GB (46087.90625 MB free)
  Uptime: 1962.0 sec
  Load Avg:  1.06787109375  1.04833984375  0.9423828125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)
Wikunia commented 3 years ago

Benchmark Report for ConstraintSolver

Job Properties

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["eternity", "5x5_all"] 0.20 (5%) :white_check_mark: 0.28 (1%) :white_check_mark:
["eternity", "5x5_opt"] 0.54 (5%) :white_check_mark: 0.53 (1%) :white_check_mark:
["eternity", "5x5_opt_ind"] 0.52 (5%) :white_check_mark: 0.51 (1%) :white_check_mark:
["eternity", "5x5_opt_rei"] 1.55 (5%) :x: 1.55 (1%) :x:
["eternity", "6x5"] 0.74 (5%) :white_check_mark: 0.76 (1%) :white_check_mark:
["graph_coloring", "US_50colors"] 1.47 (5%) :x: 1.38 (1%) :x:
["graph_coloring", "US_50colors+equal"] 1.76 (5%) :x: 1.64 (1%) :x:
["graph_coloring", "US_8+equal"] 1.79 (5%) :x: 1.57 (1%) :x:
["killer_sudoku", "niall_5500"] 11.98 (5%) :x: 8.73 (1%) :x:
["killer_sudoku", "niall_5500_special"] 30.40 (5%) :x: 18.24 (1%) :x:
["killer_sudoku", "niall_5501"] 204.14 (5%) :x: 105.02 (1%) :x:
["killer_sudoku", "niall_5501_special"] 6.86 (5%) :x: 4.21 (1%) :x:
["lp", "issue_83"] 1.95 (5%) :x: 1.82 (1%) :x:
["sudoku", "top95_11"] 5.39 (5%) :x: 4.82 (1%) :x:
["sudoku", "top95_16"] 3.27 (5%) :x: 3.13 (1%) :x:
["sudoku", "top95_31"] 0.77 (5%) :white_check_mark: 0.77 (1%) :white_check_mark:
["sudoku", "top95_46"] 3.55 (5%) :x: 3.24 (1%) :x:
["sudoku", "top95_51"] 1.39 (5%) :x: 1.35 (1%) :x:
["sudoku", "top95_56"] 1.64 (5%) :x: 1.53 (1%) :x:
["sudoku", "top95_61"] 1.65 (5%) :x: 1.59 (1%) :x:
["sudoku", "top95_66"] 1.00 (5%) 0.96 (1%) :white_check_mark:
["sudoku", "top95_71"] 1.35 (5%) :x: 1.31 (1%) :x:
["sudoku", "top95_76"] 1.39 (5%) :x: 1.37 (1%) :x:
["sudoku", "top95_81"] 7.68 (5%) :x: 6.67 (1%) :x:
["sudoku", "top95_86"] 99.98 (5%) :x: 15.24 (1%) :x:

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

Julia versioninfo

Target

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.3-arch1-1 #1 SMP PREEMPT Sun, 01 Nov 2020 12:58:59 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  2930 MHz     275376 s       1423 s      46790 s   29454142 s       4278 s

  Memory: 46.98865509033203 GB (25553.5625 MB free)
  Uptime: 24845.0 sec
  Load Avg:  1.0  0.86474609375  0.58642578125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)

Baseline

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.3-arch1-1 #1 SMP PREEMPT Sun, 01 Nov 2020 12:58:59 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1461 MHz     338729 s       1423 s      49076 s   30162851 s       4415 s

  Memory: 46.98865509033203 GB (25750.53125 MB free)
  Uptime: 25490.0 sec
  Load Avg:  1.0771484375  1.06494140625  0.875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)
Wikunia commented 3 years ago

I should try to break this into several parts which are easier to review as some of this isn't actually for activity based search :disappointed:

Wikunia commented 3 years ago

Benchmark Report for ConstraintSolver

Job Properties

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["eternity", "5x5_all"] 0.24 (5%) :white_check_mark: 0.30 (1%) :white_check_mark:
["eternity", "5x5_opt"] 0.43 (5%) :white_check_mark: 0.42 (1%) :white_check_mark:
["eternity", "5x5_opt_ind"] 1.56 (5%) :x: 1.56 (1%) :x:
["eternity", "5x5_opt_rei"] 0.91 (5%) :white_check_mark: 0.89 (1%) :white_check_mark:
["eternity", "6x5"] 0.34 (5%) :white_check_mark: 0.32 (1%) :white_check_mark:
["graph_coloring", "US_50colors"] 1.08 (5%) :x: 1.03 (1%) :x:
["graph_coloring", "US_50colors+equal"] 1.07 (5%) :x: 1.02 (1%) :x:
["graph_coloring", "US_8+equal"] 1.07 (5%) :x: 1.03 (1%) :x:
["graph_coloring", "le450_5d"] 6.17 (5%) :x: 2.78 (1%) :x:
["graph_coloring", "queen7_7"] 3.39 (5%) :x: 1.89 (1%) :x:
["killer_sudoku", "niall_5500"] 6.69 (5%) :x: 3.08 (1%) :x:
["killer_sudoku", "niall_5500_special"] 4.48 (5%) :x: 1.84 (1%) :x:
["killer_sudoku", "niall_5501"] 81.46 (5%) :x: 5.65 (1%) :x:
["killer_sudoku", "niall_5501_special"] 2.51 (5%) :x: 3.11 (1%) :x:
["lp", "issue_83"] 0.37 (5%) :white_check_mark: 0.31 (1%) :white_check_mark:
["sudoku", "top95_11"] 2.23 (5%) :x: 0.68 (1%) :white_check_mark:
["sudoku", "top95_16"] 1.89 (5%) :x: 0.68 (1%) :white_check_mark:
["sudoku", "top95_31"] 1.63 (5%) :x: 0.62 (1%) :white_check_mark:
["sudoku", "top95_41"] 1.54 (5%) :x: 1.05 (1%) :x:
["sudoku", "top95_46"] 2.77 (5%) :x: 1.04 (1%) :x:
["sudoku", "top95_51"] 2.62 (5%) :x: 0.82 (1%) :white_check_mark:
["sudoku", "top95_56"] 1.83 (5%) :x: 0.59 (1%) :white_check_mark:
["sudoku", "top95_61"] 2.80 (5%) :x: 1.05 (1%) :x:
["sudoku", "top95_66"] 1.33 (5%) :x: 0.37 (1%) :white_check_mark:
["sudoku", "top95_71"] 1.41 (5%) :x: 1.15 (1%) :x:
["sudoku", "top95_76"] 3.15 (5%) :x: 1.12 (1%) :x:
["sudoku", "top95_81"] 2.48 (5%) :x: 1.12 (1%) :x:
["sudoku", "top95_86"] 3.22 (5%) :x: 0.99 (1%) :white_check_mark:
["sudoku", "top95_91"] 0.98 (5%) 0.82 (1%) :white_check_mark:

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

Julia versioninfo

Target

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.12-arch1-1 #1 SMP PREEMPT Wed, 02 Dec 2020 16:14:56 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1712 MHz     176489 s      57078 s      33579 s   35306600 s       5235 s

  Memory: 46.98862838745117 GB (42970.40234375 MB free)
  Uptime: 29661.0 sec
  Load Avg:  1.103515625  1.0732421875  0.75830078125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)

Baseline

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.12-arch1-1 #1 SMP PREEMPT Wed, 02 Dec 2020 16:14:56 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  2160 MHz     244361 s      57078 s      35971 s   36063622 s       5419 s

  Memory: 46.98862838745117 GB (43457.26953125 MB free)
  Uptime: 30351.0 sec
  Load Avg:  1.10107421875  1.13037109375  1.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)
Wikunia commented 3 years ago

Benchmark Report for ConstraintSolver

Job Properties

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["eternity", "5x5_all"] 1.15 (5%) :x: 1.15 (1%) :x:
["eternity", "5x5_opt"] 0.59 (5%) :white_check_mark: 0.61 (1%) :white_check_mark:
["eternity", "5x5_opt_ind"] 0.45 (5%) :white_check_mark: 0.46 (1%) :white_check_mark:
["eternity", "5x5_opt_rei"] 1.12 (5%) :x: 1.14 (1%) :x:
["eternity", "6x5"] 0.25 (5%) :white_check_mark: 0.25 (1%) :white_check_mark:
["graph_coloring", "US_50colors"] 1.03 (5%) 0.90 (1%) :white_check_mark:
["graph_coloring", "US_50colors+equal"] 1.03 (5%) 0.93 (1%) :white_check_mark:
["graph_coloring", "US_8+equal"] 1.03 (5%) 0.92 (1%) :white_check_mark:
["graph_coloring", "le450_5d"] 0.99 (5%) 0.99 (1%) :white_check_mark:
["graph_coloring", "queen7_7"] 1.01 (5%) 0.97 (1%) :white_check_mark:
["killer_sudoku", "niall_5500"] 1.37 (5%) :x: 1.41 (1%) :x:
["killer_sudoku", "niall_5500_special"] 0.86 (5%) :white_check_mark: 0.88 (1%) :white_check_mark:
["killer_sudoku", "niall_5501"] 3.61 (5%) :x: 3.41 (1%) :x:
["killer_sudoku", "niall_5501_special"] 0.28 (5%) :white_check_mark: 0.35 (1%) :white_check_mark:
["lp", "issue_83"] 0.95 (5%) :white_check_mark: 0.92 (1%) :white_check_mark:
["sudoku", "top95_11"] 1.06 (5%) :x: 1.03 (1%) :x:
["sudoku", "top95_16"] 1.06 (5%) :x: 1.03 (1%) :x:
["sudoku", "top95_31"] 1.05 (5%) :x: 1.00 (1%)
["sudoku", "top95_46"] 1.05 (5%) :x: 1.00 (1%)
["sudoku", "top95_66"] 1.08 (5%) :x: 1.01 (1%) :x:
["sudoku", "top95_91"] 1.01 (5%) 0.97 (1%) :white_check_mark:

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

Julia versioninfo

Target

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.13-arch1-1 #1 SMP PREEMPT Tue, 08 Dec 2020 12:09:55 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1464 MHz     524440 s       2979 s      38224 s   28147444 s       4239 s

  Memory: 46.9886360168457 GB (45019.359375 MB free)
  Uptime: 23940.0 sec
  Load Avg:  1.01513671875  0.97119140625  0.634765625
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)

Baseline

Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Arch Linux"
  uname: Linux 5.9.13-arch1-1 #1 SMP PREEMPT Tue, 08 Dec 2020 12:09:55 +0000 x86_64 unknown
  CPU: Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz: 
                 speed         user         nice          sys         idle          irq
       #1-12  1872 MHz     592749 s       2979 s      40704 s   28913041 s       4427 s

  Memory: 46.9886360168457 GB (45015.71875 MB free)
  Uptime: 24637.0 sec
  Load Avg:  1.197265625  1.0673828125  0.8837890625
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, sandybridge)