Consider hosting or replacing STEPWISE and LHS source code

[jdpipe]

Version 5.5 of lp_solve mentions (http://lpsolve.sourceforge.net/5.5/) that it is released under the LGPL which means that it is required that ssc include the LGPL license conditions with your own code -- eg by adding a LICENSE.txt file in the lpsolve directory in ssc.

The same is true for nlopt -- https://nlopt.readthedocs.io/en/latest/NLopt_License_and_Copyright/

[nickdiorio]

Thanks for bringing this up, I'm not sure how it was missed, but we'll get those licenses added today.

[jdpipe]

Thanks Nick. To lp_solve and nlopt I would add STEPWISE and LHS. I googled STEPWISE and found that it lives here: https://dakota.sandia.gov/content/packages/stepwise

Seems that it would be reasonable to include the source code in the SAM repository as well, in case people would like to compile for other architectures etc?

***Also regarding STEPWISE (since there is no contact information at the URL above) I note here that the scanned SAND79-1472 report is missing Page 1, and that the source code tarball seems to have a different user's manual embedded in the ZIP file than that which is given at the link to SAND79-1472. -- Perhaps you might know who to contact about that.

Regarding LHS, https://dakota.sandia.gov/content/packages/lhs, there is no standalone source code release; one has to download the whole Dakota package eg version 6.7 from https://dakota.sandia.gov/sites/default/files/distributions/public/dakota-6.7-release-public-src.tar.gz. Perhaps you have some instructions for how you created the 'lhs.bin' blob?

[cwhanse]

Nick, let me know if you need help with those STEPWISE and LHS issues.

[nickdiorio]

@jdpipe, we originally decided not to host STEPWISE and LHS so as not to create confusion about the origin of those code bases, which are Sandia National Laboratory produced libraries, and hosted at dakota.sandia.gov as you mention. I believe LHS was originally available as standalone source code, before it was integrated into the Dakota package. We compiled the binaries from the standalone source to distribute with SAM and comply with the LGPL licenses.

@cwhanse , do you have any guidance or suggestions? We would certainly like to ensure we're complying with the intended usage of LHS and STEPWISE.

Thanks!

[cwhanse]

I looked at the codebase for both LHS and STEPWISE. Dismaying. Both are legacy programs written in Fortran, that have been adapted to work within DAKOTA. The source code is obtainable although not conveniently, as @jdpipe points out. And I would not want to have to build an executable, as neither code is packaged for independent use (e.g., the STEPWISE distribution includes a bunch of extraneous code to link to other Sandia-written applications). There are no plans to distribute or maintain either LHS or STEPWISE separate from DAKOTA. I don't see any practical alternatives to distributing compiled versions with SAM.

For STEPWISE, there is a more recent user manual with the code: https://dakota.sandia.gov/content/packages/stepwise

For the longer term, I would advise finding other sources for Latin hypercube sampling and stepwise regression capability. Happy to help look, if you can provide some guidance in terms of language, etc.

[nickdiorio]

Thanks @cwhanse for the guidance and additional reference. From the SAM perspective I think we're okay using the binaries compiled from Fortran, as we rarely need to look at, let alone modify it. But, if you do find alternates or re-write them in C/C++, we'd certainly look at replacing. The other popular choice these days seems to be Python, which I think we could work with or adapt.

[jdpipe]

I would like to encourage you to reconsider the idea of hosting the code within SAM/SSC, or as a separate Github repository. If the Sandia guys aren't hosting in a convenient form then it's OK to take out the bits you are using and host them somewhere else. LGPL is fine with that AFAIK -- just keep acknowledging the source, and try to merge the upstream changes whenever possible. You will already be doing that anyway, just not sharing your process. The bonus of sharing your process like this (hosting the code and build process that you are using to create your binary) is that others will be able to compile, eg for another platform/architecture other than the one you're using, and possibly even to create a package that can be bundled into the operating system.

[nickdiorio]

@jdpipe , sorry for the delay. I think that's something we could do once we have a little more bandwidth. If it enhances the user experience in a way that doesn't make it too much more difficult to compile SAM, then I'd say we're open to it. Perhaps we would still distribute the compiled binaries, but could also include the source code (while acknowledging that it is Sandia derived) if people wanted to re-target for another platform.

[nickdiorio]

@jdpipe, @cwhanse , I finally took some time to dive more into this issue. We determined that it would make the most sense to open-source the underlying code used to build these binaries, which I've done for LHS and STEPWISE. We hope that doing so enables others to build on different platforms or swap in modified libraries.

We fully acknowledge the origin of these libraries as Sandia National Laboratory products, packaged with the Dakota framework (here and here, licensed as LGPL code.

Down the road, if there are suggestions for incorporating more actively maintained libraries with the functionality of LHS or STEPWISE, we'd be happy to consider them. At present, we've found no solutions with similarly licensed terms.