Response from mosek affected by non-model changes

[pgkirsch]

I've been having some issues with some of my larger models (12000+ free variables) recently, where if I make a change that has no effect on the model (e.g. calling from a different script, making an inconsequential change to the high level script etc.), mosek returns unknown, or conversely is able to solve a model for which it previously returned unknown.

For the most recent instance, I tried comparing the two A-matrices and I believe they are different (In [16]: np.allclose(A1.todense(), A2.todense()) Out[16]: False), which makes sense if mosek is giving different responses.

Does anyone have any suggestions for how to better understand what is going on? And are there tolerances I can tune in mosek via GPkit?

[1ozturkbe]

Which version of mosek and solver are you using? I'm assuming it is an SP, so does it use a naive localsolve?

[1ozturkbe]

(fyi, I have no idea why this might be, I'm just trying to see if extra info might help. however, the default localsolve method has been modified recently so I'm wondering if this can be a difference.)

[pgkirsch]

I'm using mosek 8 (sadly don't think an upgrade is on the cards for now), GPkit c7efc4f8dce23ef742e511869ce39de7488517fd, and yes naive localsolve. I only have one signomial constraint with two free variables but I could test to see if an intelligent x0 helps (although 1 for both isn't too terrible).

[1ozturkbe]

I don't think you should mess with x0. I would try modifying the localsolve through kwargs pccp_penalty=2e2 which is default. This is a penalty used in the new penalty convex-concave method that replaced old naive localsolve.

[1ozturkbe]

If you want to revert to old localsolve, you can also pass use_pccp=False and this should give you the same results as you had before.

[1ozturkbe]

But tbh these are all model-related hacks. I don't know why non-model changes would affect the solution otherwise.

[pgkirsch]

True. Good to know regardless. And just to be clear I don't think any changes on the GPkit side "broke" anything. I will update to latest master though because it seems like there were some relevant improvements since the beginning of March.

[1ozturkbe]

That would be good. Hope you can figure it out!

[pgkirsch]

Thanks!

[pgkirsch]

Wow ~40% speed-up on my largest test case, nice work @bqpd!

[pgkirsch]

I found some promising mosek parameters I would like to test out, but I can't quite figure out if it's possible to pass those through solverargs for mosek_cli. Is it possible @bqpd?

[aszekMosek]

@pgkirsch Why not get a MOSEK trial license and use Mosek 9 with native exponential cone support? If you have some hard task files it would be interesting to check them, too. Feel free to write to write to support@mosek.com

[pgkirsch]

Thanks @aszekMosek! I've been meaning to do both of those things.

[bqpd]

I found some promising mosek parameters I would like to test out, but I can't quite figure out if it's possible to pass those through solverargs for mosek_cli. Is it possible @bqpd?

Unfortunately mosek_cli as written doesn't pass args through into the created gpkit_mosek file; MSK_readparamfile might allow for doing so, but it doesn't seem particularly documented.

[bqpd]

I've been having some issues with some of my larger models (12000+ free variables) recently, where if I make a change that has no effect on the model (e.g. calling from a different script, making an inconsequential change to the high level script etc.), mosek returns unknown, or conversely is able to solve a model for which it previously returned unknown.

Does the file run sometimes and not others even in the face of no changes at all? If so it's probably somewhere I forgot to ensure deterministic dictionary ordering, but if not it's more likely something else, since the random salt is per Python session and shouldn't be affected by which script you call from.

[aszekMosek]

@bqpd Not sure if this is relevant but the parameter file is better documented here https://docs.mosek.com/9.2/cmdtools/mosekcomtool.html#using-the-parameter-file

[whoburg]

If so it's probably somewhere I forgot to ensure deterministic dictionary ordering

Isn't dict ordering deterministic in python >= 3.7? Or is our insertion order non-deterministic?

[bqpd]

If so it's probably somewhere I forgot to ensure deterministic dictionary ordering

Isn't dict ordering deterministic in python >= 3.7? Or is our insertion order non-deterministic?

Yeah but a lot of Py3 installs are still 3.5...

[bqpd]

@bqpd Not sure if this is relevant but the parameter file is better documented here https://docs.mosek.com/9.2/cmdtools/mosekcomtool.html#using-the-parameter-file

oh great! @pgkirsch looks like this is implementable; are you still using mosek_cli?

[pgkirsch]

Does the file run sometimes and not others even in the face of no changes at all?

No, when I last observed this, code changes were necessary to cause changes in the outcome. And for what it's worth I'm using python 3.7.

are you still using mosek_cli?

Yes.

I still need to follow @aszekMosek's advice and see if his team can help shed some light on what is causing these more challenging problems to fail.

[bqpd]

ah bizarre. If you're 3.7+ you should be able to diff the problem_output file across runs; if that's changing while the model print remains the same that would be good to catch.

[pgkirsch]

Sorry I don't follow. Which problem_output file?

[bqpd]

oh, the one generated by the mosek_cli solve process! mskexpopt takes a file as input, so we solve by making a text file and then passing its location.

if you solve with solve("mosek_cli", path="/home/pgkirsch/") (or similar) it'll generate the file at a more visible location and not delete it after solving.