search

sanurielf / kvxopt

KVXOPT -- Python Software for Convex Optimization containing more wrappers suite-sparse.
Other
10 stars 1 forks source link

Allocation-free in-place add and set methods for spmatrix #2

Closed cuihantao closed 4 years ago

cuihantao commented 4 years ago

Hi,

We'd like to contribute two methods for adding/setting values to existing spmatrix elements. They are implemented as ipadd and ipset methods of kvxopt.spmatrix. These functions are convenient for writing elements to a known Jacobian pattern, say, for power system simulations.

The code is contributed by Nichlos West, @TheHashTableSlasher. The code was originally part of our CVXOPT fork, but I think KVXOPT is a nice repository to get it eventually merged. Let us know what you think.

Thanks.

cuihantao commented 4 years ago

I also found an issue with the wheel file for macOS, which did not pack SuiteSparse libraries like other wheels do.

Not sure if this is intended behavior. It causes issue when using kvxopt in continuous integration platforms, which is difficult to install a system-wide SuiteSparse.

sanurielf commented 4 years ago

@cuihantao I've integrated your changes. Please also test the new wheels for macOS which now include the *.so files to run without installing anything extra.

cuihantao commented 4 years ago

I just tested v1.2.5.4 on macOS. It works flawlessly. I will report it here should I run into any issue.

cuihantao commented 4 years ago

Would you like me to make a conda-forge feedstock for it? I should be able to reuse CVXOPT's without modification.

cuihantao commented 4 years ago

The macOS path issue might not have been fixed. Although it passes on my laptop, CI on Azure Pipeline gives the following error message.

dlopen(/Users/runner/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/kvxopt/umfpack.cpython-37m-darwin.so, 2): Symbol not found: _aligned_alloc
  Referenced from: /Users/runner/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/kvxopt/.dylibs/libgomp.1.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/runner/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/kvxopt/.dylibs/libgomp.1.dylib

Not sure how further I can help with the troubleshooting.

sanurielf commented 4 years ago

It seems that we have different versions of OSX between the GH-actions used to built the wheel and your CI. Are those CI/Pipelines public? I would like to take a look at the whole specs and logs of the virtual machine.

Would you like me to make a conda-forge feedstock for it? I should be able to reuse CVXOPT's without modification.

I already have a private repo with some tests for a conda-forge. Let me update it and that can be the solution.

cuihantao commented 4 years ago

It seems that we have different versions of OSX between the GH-actions used to built the wheel and your CI. Are those CI/Pipelines public? I would like to take a look at the whole specs and logs of the virtual machine.

The pipeline is public. The logs are at https://dev.azure.com/hcui7/hcui7/_build/results?buildId=1228&view=logs&jobId=93056758-5bfb-5750-f113-e720ddefdb4c&j=93056758-5bfb-5750-f113-e720ddefdb4c&t=5614d57b-723a-5015-8888-62d21c9db85c

The error message is under "Captured stdout".

The pipeline file is at https://github.com/cuihantao/andes/blob/master/.azure-pipelines/azure-pipelines-macos.yml

sanurielf commented 4 years ago

I've modified your pipeline and updated the mac-os version to 10.15 and now it works. It seems that /usr/lib/libSystem.B.dylib may be different between 10.14 and >10.15. So we may have this problem even if we use a conda-forge

Is there a reason to maintain version 10.14?

Check the pipeline output here:

https://dev.azure.com/sanurielf/andes/_build/results?buildId=17&view=logs&j=93056758-5bfb-5750-f113-e720ddefdb4c&t=ce0d52b1-8769-5d0d-4451-1919f7eefce7

And also my changes to your andes project here:

https://github.com/sanurielf/andes/commits/develop

cuihantao commented 4 years ago

Uriel, thank you for checking this!

There is not any reason to keep 10.14 - I forgot to update it.

I will make a PR from your changes. When your conda feedstock is out, I can edit andes's conda recipe to point to kvxopt. That's the last thing I need to do for the migration.

sanurielf commented 3 years ago

Hi @cuihantao,

I've updated kvxopt to version 1.2.6.0. Could you check if the new version fixes your error when using OSX?

Thanks!!

cuihantao commented 3 years ago

@sanurielf Just tested. It works on macOS 11.2.3 running on M1.

sanurielf commented 3 years ago

Great. I've included a C-level interface to the OSQP QP solver, which fits very well for big-size Economic Dispatch problems. I've also created the kvxopt-wheels to support more linux/osx versions and now I'm focussed in the conda recipe.

cuihantao commented 3 years ago

Do you need any help on the conda recipe? We would like to see kvxopt available on conda-forge.

sanurielf commented 3 years ago

Thanks for reminding me. As soon as i finish the final tests of the OSQP and Gurobi interfaces I will go with this conda recipe.

cuihantao commented 3 years ago

Thanks for reminding me. As soon as i finish the final tests of the OSQP and Gurobi interfaces I will go with this conda recipe.

Great. Very much appreciated!!

sanurielf commented 3 years ago

I've created the KVXOPT recipe here. I hope we can have it in conda soon.

cuihantao commented 3 years ago

Thanks!! I will be able to fully swap out cvxopt when this is in.

Regards, Hantao Cui On Oct 14, 2021, 2:21 PM -0500, Uriel Sandoval @.***>, wrote:

I've created the KVXOPT recipe here. I hope we can have it in conda soon. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.