search

NREL / PRAS

NREL's Probabilistic Resource Adequacy Suite (PRAS)
https://www.nrel.gov/analysis/pras.html
Other
46 stars 11 forks source link

Dramatic slowdown after upgrading to v0.6.0 #30

Closed ianfiske closed 3 years ago

ianfiske commented 3 years ago

To keep up with changes in v0.5.4 to v0.6.0, I made the following change to our code

      result_classic = assess(Convolution(), Temporal(), singlenode)

became

      result_classic, = assess(singlenode, Convolution(), Shortfall())

However, runtime increased by a factor of about 100x. This is an approximate timing because timing that specific line is a bit tricky -- and since it takes so long to run, I am seeing if I can avoid iterating just to measure for now.

Is this a known issue? Or did I inadvertently request additional computations alongside my syntax change? We have some comments in our code indicating that we tried to stick with "classic" because it was much faster than "modern", but I don't see anything in the NREL PRAS docs about classic vs modern. Does classic still exist?

Thank you!

ianfiske commented 3 years ago

To answer part of my own question, https://github.com/NREL/PRAS/pull/11 shows where the Classic/Modern distinction originally referred to Convolution / SequentialMonteCarlo. Since we are still using convolution here, I don't think that is the source of the slowdown. So I'm wondering if the Convolution method itself changed in a way that requires more computation.

ianfiske commented 3 years ago

Doing a bit of profiling this morning and it looks like the culprit may not be PRAS at all, but instead I'm running into https://github.com/jump-dev/Cbc.jl/issues/172 earlier in our code. Didn't see that coming, but I suppose updating packages can lead to surprises. Will update again here after I limit the Cbc version and verify that this fixes.

ianfiske commented 3 years ago

Yes, it was a Cbc issue (linked above), not a PRAS usual at all. I apologize for the noise!

GordStephen commented 3 years ago

Not a problem, glad you were able to sort it out!