We provide benchmarks to encourage collaboration and competition in making actuarial software run faster than ever before.
We currently only have 1 benchmark, we are working to expand the benchmarks. Open a discussion in this repository if you have any thoughts to share.
Time measurement is currently the best of 3 runs.
benchmark | classification | container | A100-SXM4-40GB | H100-SXM5-80GB |
---|---|---|---|---|
BasicTerm_ME 100 Million | recursive PyTorch | link | 15.8284s | 7.205s |
BasicTerm_ME 100 Million | compiled iterative JAX | link | 3.448s | 1.551s |
model_point_table.xlsx
. We use these modelpoints, but repeat them 10,000 times for 100,000,000 modelpoints.These benchmarks run in GitHub-hosted runners in GitHub actions. Used for benchmarks that are not computationally intensive.
Benchmarks in this repository:
basic_term_benchmark
: Replicate the cashflows of the LifeLib BasicTerm model
exposures
: Create date partitions for experience studies
mortality
: Read SOA mortality tables and use them in a simple calculation
The below results are generated by the benchmarking scripts in the folders for each language. These scripts are run automatically by GitHub Actions and populate the results below.
basic_term_benchmark:
- Julia array basic_term:
minimum time: TrialEstimate(30.279 ms)
result: 1.4489630534602132e7
Julia recursive basic_term:
minimum time: TrialEstimate(84.812 ms)
result: 1.4489630534602132e7
- Python array numpy basic_term_m:
minimum time: 83.45314299992879 milliseconds
result: 14489630.534603368
Python array pytorch basic_term_m:
minimum time: 51.54931000004126 milliseconds
result: 14489630.534603368
Python lifelib basic_term_m:
minimum time: 618.0503439999256 milliseconds
result: 14489630.534601536
Python recursive numpy basic_term_m:
minimum time: 63.08064000006652 milliseconds
result: 14489630.534603368
Python recursive pytorch basic_term_m:
minimum time: 75.6699999999455 milliseconds
result: 14489630.53460337
basic_term_me_benchmark:
- Python heavylight numpy basic_term_me:
minimum time: 354.6492100000478 milliseconds
result: 215146132.0684811
Python lifelib basic_term_me:
minimum time: 1191.792300999964 milliseconds
result: 215146132.06848112
Python recursive numpy basic_term_me:
minimum time: 309.30894900006933 milliseconds
result: 215146132.0684814
exposures:
- Julia ExperienceAnalysis.jl:
minimum time: TrialEstimate(29.659 ms)
num_rows: 141281
- R actxps:
min: 486.248656 ms
num_rows: 141281
mortality:
- Julia MortalityTables.jl:
minimum time: TrialEstimate(229.507 μs)
result: 1904.4865526636793
- Python PyMort:
minimum time: 9.425531999909253 milliseconds
result: 1904.4865526636793
savings_benchmark:
- Julia Benchmarks savings:
minimum time: TrialEstimate(119.226 ms)
result: 3.507113709040273e12
- Python lifelib cashvalue_me_ex4:
minimum time: 596.2715760000492 milliseconds
result: 3507113709040.141
Python recursive numpy cashvalue_me_ex4:
minimum time: 543.3022800000344 milliseconds
result: 3507113709040.124