search

JuliaReach / ARCH2020_NLN_RE

Repeatability Evaluation package for the ARCH2020 NLN Competition
3 stars 0 forks source link

Add further benchmarks and output to a single file #6

Closed lbenet closed 4 years ago

lbenet commented 4 years ago

One possible way of having all benchmarks written in a single csv file

mforets commented 4 years ago

Awesome, thanks for putting this up :clap:
I've ran startup.jl in my machine and it is giving:

JuliaReach, PRDE20, I, 1, 1.07556081, 3.3e-20
JuliaReach, PRDE20, P, 1, 2.6764912525, 6.5e-21
JuliaReach, PRDE20, IP, 1, 2.067613997, 1.0e-20
JuliaReach, CVDP20, μ=1.0, 1, 0.29260537600000003
JuliaReach, CVDP20, μ=2.0, 1, 25.045381359
JuliaReach, LALO20, W=0.01, 1, 0.730844083, 0.0042
JuliaReach, LALO20, W=0.05, 1, 0.9436897405000001, 0.017
JuliaReach, LALO20, W=0.1, 1, 0.9245671240000001, 0.033
JuliaReach, QUAD20, Δ=0.1, 1, 1.7125344150000001
JuliaReach, QUAD20, Δ=0.4, 1, 1.221724344
JuliaReach, QUAD20, Δ=0.8, 0, 1.230236394
JuliaReach, LOVO20, , 1, 134.525143724, 0.000825, 0.176

Looks good, i have two observations:

1) The runtime for coupled van der pol and mu = 2.0 is twice as fast as the time i had reported in the paper (48s).

2) The runtime for lotka-volterra is 134sec, but i had reported ~3.8sec. I'll check if somthing has changed.

lbenet commented 4 years ago

Thanks for taking a look on this @mforets ! I guess the differences are related to the actual abs_tol used, which may be different. Please feel free to change (directly) anything you consider it worth, e.g., to make the internal functions more julian.

Incidentally, do you know if it is possible to test the docker container with travis?

mforets commented 4 years ago

Incidentally, do you know if it is possible to test the docker container with travis?

Sounds like a good idea, i've opened https://github.com/JuliaReach/ARCH2020_NLN_RE/issues/7

lbenet commented 4 years ago

All except the space rendezvous are ready: benchmarks are produced as well as the plots.

I'll go ahead and merge this.