Last period constrain error when training

[GuilleHele]

Dear community:

We are using SDDP and it is an awesome library. But we are having a little problem when adding a condition for the last period, when we run the training code, arise a warning message saying "Warning: basis matrix is ill-conditioned" in some of the iteration, so we wanna ask you if there is a way to see this matrix or to know what makes that error. Sometimes that error continue for every iteration and the algorithm nevers stop.

Explicit message obtain: "Warning: basis matrix is ill-conditioned (cond = 6.55e+012)"

I thank you in advance for your help.

Sincerely,

PS: It is my first time posting in Github, so if there is anything else that I should provide, please let me know it the comments.

Guillermo Sepúlveda

[odow]

Hi there!

This warning comes from GLPK when the problem has some difficult numerical properties (likely cause: we are adding cuts with a high dynamic range, so very steep in some dimensions, and very flat in others).

Take a read of: https://odow.github.io/SDDP.jl/stable/tutorial/06_warnings/

There are a couple of things you can do:

1. Re-scale your problem. Pay close attention to the numerical stability report at the start of a solve. You want numbers in the range [1e-04, 1e+04] if possible. (Basically, the closer the two numbers are, and the closer they are to 1e+00, the better.)

2. Switch to Gurobi.jl. In my experience, Gurobi can be slower than GLPK or CPLEX, but it is much more robust to these numerical issues.

If your code is small enough, and you don't mind sharing publicly, you can also post it here, or on https://gist.github.com and I'll see if anything obvious can be improved.

[GuilleHele]

Dear Oscar Dowson:

Thanks so much for your help, I follow your advice and use Gurobi as an optimizer for the algorithm and it works wonders.

For the numerical stability problem, we consider that issue from the beginning in order to discard it, so I guess it was only the optimizer as you said it.

About the code, my Professor and I wanna make a publication about it, so we can´t post the code for now, but it will be awesome to have your supervision in order to improve it once it is finished.

Thanks again for your help.

Sincerely,

Guillermo Sepúlveda

[odow]

The numerical stability is probably still an issue, it is just that Gurobi is much better at handling poorly scaled problems.

p.s. who are you working with? They probably know my personal email if you have follow-up questions.

[GuilleHele]

I´m working with Lorenzo Reus from Adolfo Ibáñez University, he has done some researches using your library before. He has your mail, but it the same that appears in your papers, don´t know if that is the one you use. My academic mail is gusepulveda@alumnos.uai.cl, I would be happy if you could write me so I can save your mail.

p.s.: My apologies for not writing before.

Sincerely,

Guillermo Sepúlveda