search

SciML / SciMLBenchmarks.jl

Scientific machine learning (SciML) benchmarks, AI for science, and (differential) equation solvers. Covers Julia, Python (PyTorch, Jax), MATLAB, R
https://docs.sciml.ai/SciMLBenchmarksOutput/stable/
MIT License
319 stars 84 forks source link

Fix NBodySimulator Benchmarks #271

Open ChrisRackauckas opened 3 years ago

ChrisRackauckas commented 3 years ago

Continuouing https://github.com/SciML/SciMLBenchmarks.jl/pull/245 here.

@SebastianM-C these time out. How long do they take?

SebastianM-C commented 3 years ago

It is possible to be over an hour, but I'm not sure, I'll have to run them and see. Should I try to make it shorter? What should be the benchmark duration?

ChrisRackauckas commented 3 years ago

They are failing at 6 hours.

SebastianM-C commented 3 years ago

Oh... That's unexpected.

SebastianM-C commented 3 years ago

Are there any logs available to see where it hangs / what happens?

ChrisRackauckas commented 3 years ago

https://buildkite.com/julialang/scimlbenchmarks-dot-jl/builds/242#df9f5bc4-7f12-416e-8583-7e8e6088387b

SebastianM-C commented 3 years ago

I ran them locally and it looks like it's not getting stuck, but that the benchmarks take very long image

I'll try to have a look at some concrete examples to see if there are any performance regressions, but it may also be due to how many runs we have at different tolerances.