Lpsolver 3.0

[AFMac]

Adds mixed integer linear programming extension to NetLogo.

[AFMac]

Jeremy-

Thanks for the feedback. I have no access to a Mac, so am running in the blind for that OS. The lpsolve distro page currently only has 32-bit libraries, and the current developer on the project doesn't have access to a Mac so was unable to help me with a 64-bit version. I was able to find a user community where someone had compiled what was supposed to have been the library file for a 64-bit architecture, but I may have grabbed the wrong file, or it wasn't done correctly.

I thought I'd give it a shot with this file, but if its a no-go, I may just throw in the towel and leave it as a community extension living on the wiki.

[LaCuneta]

@AFMac Thanks for trying the user-built 64 bit library!

I went to capture a more detailed error message of why the extension failed to load, and what I found was the lpsolver.jar file included in this PR doesn't look like a valid jar file - NetLogo threw an error trying to load it. I tried to unzip it, as a jar should just be a zip file, and got the message End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive.

If you can get things re-packaged with a valid jar, I'm happy to try re-testing for you on macOS and getting you any diagnostic info I can if things don't work.

[AFMac]

@LaCuneta - appreciate the additional look at this. I'm not sure what happened there, but was able to see the same thing on this end. I recompiled, updated the download link in the libraries.conf file, and fixed the lack of newline in the manifest along the way. Fingers crossed...

[LaCuneta]

Thanks for getting things fixed up. This time the extension loaded, no problems or errors. I then tried to run a primitive and got this error:

java.lang.UnsatisfiedLinkError: no lpsolve55j in java.library.path
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
 at java.lang.Runtime.loadLibrary0(Runtime.java:870)
 at java.lang.System.loadLibrary(System.java:1122)
 at lpsolve.LpSolve.<clinit>(LpSolve.java:275)
 at org.nlogo.extensions.lpsolver.LPMaximize.report(LPMaximize.java:69)
 at org.nlogo.prim._externreport.report(_externreport.java:31)
 at org.nlogo.prim.etc._show.perform(_show.scala:12)
 at org.nlogo.nvm.Context.stepConcurrent(Context.java:107)
 at org.nlogo.nvm.ConcurrentJob.step(ConcurrentJob.scala:65)
 at org.nlogo.job.JobThread.runPrimaryJobs(JobThread.scala:133)
 at org.nlogo.job.JobThread.$anonfun$run$1(JobThread.scala:68)
 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
 at scala.util.control.Exception$Catch.apply(Exception.scala:228)
 at org.nlogo.api.Exceptions$.handling(Exceptions.scala:41)
 at org.nlogo.job.JobThread.run(JobThread.scala:66)

I confirmed all the files from the zip were in the extensions/lpsolver folder under NetLogo's application directory. Maybe it's a misdirected path?

[AFMac]

@LaCuneta - is it verified that the path includes the folder with the extension? I've had to manually make the adjustments on Windows, so this may be an added step on Mac. For Windows/Linux, there's a scan to ensure the path includes the lpsolver extension directory...but we weren't sure of the file structure on a Mac, this check currently is just commented out in the code and doesn't do anything. If you manually adjusted the path to point to the lpsolver extension folder, does that enable functionality?

[LaCuneta]

I added some debug code to check the java.library.path after lpsolver loads and saw that it was pointing to the user-relative folder that the extension manager would use for macOS: /Users/USER_NAME/Library/Application Support/NetLogo/6.1/extensions/lpsolver. So I extracted the extension files there and re-ran and still got that same exception, java.lang.UnsatisfiedLinkError: no lpsolve55j in java.library.path. Just to make sure, I added the hard-coded a path to the development folder extensions/lpsolver directory the extension was actually being loaded from to the java.library.path, and still got the exception. So it seems like it's still not finding it for some reason. Let me know if there is anything else I can check on.

[AFMac]

@LaCuneta - First and foremost...I appreciate your willingness to continue to push this noodle uphill, and am sorry for the back and forth. I did a little more digging and think I may have included the incorrect library file for the MacOS. I'd grabbed the .a file, which seems to be a static library file, when I should have included a .dylib file since this extension uses dynamic linking to the lpsolve library. I've swapped out the files and re-committed a new zip file with the (hopefully) right library. If you could give this a shot, would appreciate it.

Mac

[LaCuneta]

@AFMac No problem, I don't mind testing at all. Unfortunately I don't have good news with the .dylib file, I still get the same error, java.lang.UnsatisfiedLinkError: no lpsolve55j in java.library.path. I extracted to both the NetLogo application folder as well as the user-specific folder the extensions manager would use (after clearing both locations). Same result with hard-coding the java.library.path, too. Anything else we can try?

[AFMac]

@LaCuneta

...Apparently I hadn't been reading some of the initial documentation close enough. It seems that there was yet another platform-specific library I was missing that was needed specifically for the Java interaction (JNI wrapper). Linux and Windows distros had this included, but looking closer it seems the MacOS side of the house just included a build file. Long story short: I was able to dig around and believe I've found the right file. This has been added to the zip file, which has summarily been committed and is ready for a test...again. :)

[LaCuneta]

So there was some progress! This time when loading the extension, I got a macOS GateKeeper warning about the liblpsolve55j.jnilib file being a downloaded executable. I manually removed the quarantine bits from it so it could run, but unfortunately then I got a new exception, see below.

Just for kicks I tried moving the liblpsolve55.dylib file to a bin/osx64 folder under the lpsolver folder (NetLogo/extensions/lpsolver/bin/osx64/) as well as under the NetLogo root (NetLogo/bin/osx64/), but neither helped; in fact, they gave the exact same error message.

java.lang.UnsatisfiedLinkError: $PATH_TO/NetLogo/extensions/lpsolver/liblpsolve55j.jnilib: dlopen($PATH_TO/NetLogo/extensions/lpsolver/liblpsolve55j.jnilib, 1): Library not loaded: bin/osx64/liblpsolve55.dylib
  Referenced from: $PATH_TO/NetLogo/extensions/lpsolver/liblpsolve55j.jnilib
  Reason: image not found
 at java.lang.ClassLoader$NativeLibrary.load(Native Method)
 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1861)
 at java.lang.Runtime.loadLibrary0(Runtime.java:870)
 at java.lang.System.loadLibrary(System.java:1122)
 at lpsolve.LpSolve.<clinit>(LpSolve.java:275)
 at org.nlogo.extensions.lpsolver.LPMaximize.report(LPMaximize.java:69)
 at org.nlogo.prim._externreport.report(_externreport.java:31)
 at org.nlogo.prim.etc._print.perform(_print.scala:12)
 at org.nlogo.nvm.Context.stepConcurrent(Context.java:107)
 at org.nlogo.nvm.ConcurrentJob.step(ConcurrentJob.scala:65)
 at org.nlogo.job.JobThread.runPrimaryJobs(JobThread.scala:133)
 at org.nlogo.job.JobThread.$anonfun$run$1(JobThread.scala:68)
 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
 at scala.util.control.Exception$Catch.apply(Exception.scala:228)
 at org.nlogo.api.Exceptions$.handling(Exceptions.scala:41)
 at org.nlogo.job.JobThread.run(JobThread.scala:66)

[AFMac]

@LaCuneta Well, progress at least! OK...a couple other sites indicate the following:

The file liblpsolve55j.jnilib should be copied to ~/Library/Java/Extensions and liblpsolve55.dylib to /usr/local/lib and chmod to executable.

And another site indicates you should: set your DYLD_LIBRARY_PATH to /usr/local/lib

Do these make sense?

[LaCuneta]

We had more success! Thanks for researching more things to try. Here is what I did:

• Tried just setting +x for the libraries already in the folder(s) without copying anyhing, but no luck.
• Then I did the full set of instructions, and it worked!
• The I tried without the shell variable. Works, don't need it.
• Then I tried without the liblpsolve55.dylib, removed from /usr/local/lib. It failed, so I re-copied it.
• Then I tried without the other copied lib - removing liblpsolve55j.jnilib from /Library/Java/Extensions, and it still worked. So just the liblpsolve55.dylib to /usr/local/lib did it.
• Removed liblpsolve55.dylib from /usr/local/lib/ and made a symlink with ln -s ~/Library/Application\ Support/NetLogo/6.1/extensions/lpsolver/liblpsolve55.dylib /usr/local/lib/. This didn't require root/sudo perms or anything, and matches how Homebrew apparently handles things when installing .dylib files.

So it looks like there is a solution there if you have some way to add a symlink to the .dylib file for macOS users? Or if you've got another idea that might work, feel free.

[AFMac]

@LaCuneta Awesome! I'll get to work on adding something to add the link. Can you confirm the file structure though? Your command above seems to have three separate arguments:

-- ~/Library/Application\ -- Support/NetLogo/6.1/extensions/lpsolver/liblpsolve55.dylib and -- /usr/local/lib

Should the first two have been concatenated? So: ln -s ~/Library/Application/Support/NetLogo/6.1/extensions/lpsolver/liblpsolve55.dylib /usr/local/lib/?

Thanks again!

[LaCuneta]

So the second argument is just the path to the where the extension manager stores the extension libraries, the backslash \ is just escaping the space in the path for the shell. You could rewrite that as ln -s ~"/Library/Application Support/NetLogo/6.1/extensions/lpsolver/liblpsolve55.dylib" /usr/local/lib/ using double-quotes to handle the space in Application Support instead.

[AFMac]

@LaCuneta - some modifications made, in an attempt to establish the required symbolic link when the extension is loaded on a mac...fingers crossed.

[LaCuneta]

Unfortunately no luck with the new one. As before, I cleared the previously downloaded files to be sure I wasn't using an old version. The new version did not create a symbolic link, and I confirmed once I added the symbolic link manually things started working again. Is there any diagnostic info I can get for you?

[AFMac]

@LaCuneta Dang it. First: thanks again for the continued slog through this. I looked back and believe that I may have actually had the arguments backwards in the code to establish the symbolic link...I've swapped that, so that may do the trick. Either way, the code attempts to catch IO exceptions and push the accompanying message to System.err. If this ends up as another no-go and you can provide whatever output that is producing, that would be helpful.

[LaCuneta]

This test also failed, but the exception output was useful (thanks for adding it): java.nio.file.NoSuchFileException: ~/Library/Application Support/NetLogo/6.1/extensions/lpsolver/liblpsolve55.dylib. I think the issue is that ~ is a special shell keyword, so Java isn't going to expand that into the user-specific folder for macOS. One workaround I can think of is to import org.nlogo.FileIO and use the FileIO.perUserDir() method to get the user-specific folder; it should handle the macOS logic as well. So FileIO.perUserDir("extensions") should get you the extensions directory that you can use to construct your paths for the symlink. I should've thought to suggest this before, my apologies!

Here is the definition of the function, and here is a usage of it by the R extension, for reference. R is doing something slightly different, making its own configuration folder "r", you'd want to get the "extensions" folder that lpsolver will be installed to and work from there.

[AFMac]

@LaCuneta - Doh! That makes total sense, and the function you'd referenced was actually used earlier in the code. I grabbed the value I'd already pulled and used that instead for the symlink creation logic. Looks like it works on Windows and Linux, so will see if this is the one. 12th time is the charm? :)

[LaCuneta]

Hey, thank you for your perseverance! I think we're almost there! I believe something is off with the symlink parameters, though, as this was the error this time: java.nio.file.FileAlreadyExistsException: /usr/local/lib. Maybe they're reverse again? Or the link path needs to include the file name?

[AFMac]

@LaCuneta - I think that was the issue. I'd inadvertently left out the file name as I was trying this out on the Windows side since the library names are different on win/mac. Snatching defeat from the jaws of victory as it were...

Another commit pushed, thanks again for the help. Sorry to have dragged this on so long.