Pre-built macOS releases

[rkoeninger]

All releases 2.1-2.5 are missing pre-built binaries for macOS.

Need someone with a mac to build these. Should be as easy as checking out the tag, fetching kernel sources, and running make release. See the readme.

[tizoc]

I made these homebrew formulas a while ago, but didn't submit them to Homebrew back then because the repository didn't have enough stars back then (they ask for 50+, this repository has 54 already):

https://github.com/Shen-Language/homebrew-shen

Homebrew provides pre-built binaries for all the accepted formulas, but this requires people to be homebrew users.

What I'm thinking is that it may be possible to automate this all using something like Azure pipelines, that way binaries would not have to be built manually anymore (I have no experience with azure pipelines, but I saw a project that handles things this way a few weeks ago).

[vrescobar]

I installed my sbcl-shen using homebrew:

 $ brew search shen
==> Formulae
shen-language/shen/shen-sbcl ✔                                                  shen-language/shen/shen-scheme

I guess to have a pipeline is better and cleaner than any other solution, especially since personal environments might be contaminated with other artifacts (other env variables or whatever). As soon as there is a global build we can also do other neat things like generate docs, releases, test performance etc.

[tizoc]

Other possibilities:

• https://docs.travis-ci.com/user/deployment/releases/
• https://www.appveyor.com/ (Linux and Windows, no OSX)

Travis sounds like a good option because Shen/CL already uses it for CI, so a big part of the work is done already, and it also supports OSX environments.

Still investigating this, just sharing what I have found so far.

Btw, both Shen/CL and Shen/Scheme repos are past 50 stars already, I will update the formulas and submit them to Homebrew later, that will give us pre-compiled binaries through Homebrew.

[tizoc]

Just submitted the updated shen-sbcl formula to Homebrew.

I also integrated travis CI into Shen/Scheme's repo, I will keep investigating it as an option for building binaries, being able to automate this for Shen/SBCL and Shen/Scheme would be great.

[vrescobar]

I confirm that 2.5.0 works properly also when downloaded from brew. Regarding CI/CD travis si the most popular option nowadays and it seamlessly integrates with github, I would just go with it (be mainstream in everything that is not your core business).

[tizoc]

It was not easy, but I managed to have Travis complete the Shen/Scheme build for Windows, from here on releasing binaries should be easy. But first I have to make it work for OSX (what I'm missing is a prebuilt Shen/SBCL binary for OSX, because I use it to compile part of Shen/Scheme, in the future I will just use binaries for earlier versions of Shen/Scheme itself).

Travis doesn't support language: generic when building under the Windows environment, but using language: shell works the same.

[tizoc]

Ok, good news, I got it all working for Shen/Scheme, with binaries for Linux, OSX and Windows: https://github.com/tizoc/shen-scheme/releases/tag/0.18

In the case of Shen/SBCL, I have to figure out how to obtain the dependencies on Windows and OSX, also, I guess for those platforms is better to skip all the non-SBCL targets.

[vrescobar]

That was surprisingly fast especially not having previous experience with travis!

Regarding OSX, what about just using brew? Example:

brew install git make wget sbcl  shen-sbcl

At some point we should be able to just use the previos shen release in order to bootstrap the following one (I guess the klambda generated source is created with library of the new runtime not the old).

Regarding non-sbcl targets, perhaps a metarepo for all of them or let each one has its own travis... but I believe that is going to cost slightly some money.

EDIT:

for windows there seems to be another extraofficial package manager supported by Travis CI, but I ignore how it works and I did not get any experience there

[tizoc]

@vrescobar brew works for OSX, but sadly chocolatey doesn't provide SBCL packages, so for windows something else will have to do (I didn't investigate much yet).

Regarding using the previous version, yes, all is needed is a single version that works, I used Shen/SBCL to bootstrap the process in Shen/Scheme, and once I was successfully able to make a release I switched to using binaries for that release of Shen/Scheme, so future releases will just use that.

But Shen/CL doesn't have to do this because a while ago @rkoeninger ported all the Shen code to Common Lisp. If we ever go back to using Shen code in the Shen/CL compiler, then yes, using binaries from previous versions will make things work.

[tizoc]

Something that may work is installing one of the .msi files from https://github.com/sbcl/sbcl/releases using msiexec.

[vrescobar]

Right now I am looking exactly at that with wine (the linux/mac wine, not the drink). I got so far and it is not working well... tomorrow I will have access to a windows env and I will try a bit how that could be done.

wine msiexec /i sbcl-1.4.14-x86-64-windows-binary.msi/qb TARGETPATH="~/.wine/drive_c/home"

[tizoc]

Great. I'm not going to start working on this yet, but when I do I will do OSX first because it is what is easier for me to test, then Linux, and Windows last because it seems the most complicated. Please share anything you find when testing that on a Windows environment, it will be helpful.

[vrescobar]

Try this commands, I am not an expert on windows but all of them did work on my env (which is also non standard env).

:: Assumes that chocolatey is preinstalled on travis, as well as the microsoft powershell.
:: Assumes that we start on the git cloned folder.
:: If no chocolatey is preinstalled use a cmd.exe session with admin rights
:: @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

set SBCL_URL="https://netcologne.dl.sourceforge.net/project/sbcl/sbcl/1.4.14/sbcl-1.4.14-x86-64-windows-binary.msi"

:: Install small deps
choco install git.install make -Y
refreshenv

:: now git and make are in path and that would be possible: git clone https://github.com/Shen-Language/shen-cl
:: but I assume travis starts the shell on the checkout dir with git's HEAD the pushed commit
set CHECKOUT_DIR=%cd%
cd %TEMP%

:: Download MSI using the powershell (choco's wget didn't work for me)
@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -Command "Invoke-WebRequest -Uri %SBCL_URL% -OutFile sbcl.msi"

:: install quiet with no ui:
msiexec /i sbcl.msi /qnb
refreshenv

:: now "sbcl" is already on path and you can execute as in any shell. e.g.
sbcl --version

cd %CHECKOUT_DIR%
:: During testing just create a checkout of your own
:: git checkout https://github.com/Shen-Language/shen-cl
:: cd shen-cl

:: Proceed with the compile
make pure
make fetch
make sbcl
make release

:: Check return codes:
if %ERRORLEVEL% NEQ 0 {
    echo "Error during last command, RC: %ERRORLEVEL%"
    exit 1
}
if %ERRORLEVEL% EQU 0 {
    bin\sbcl\shen.exe -v
    echo "Release successful!"
    exit 0
}

At the end there is not just the release zip but you could also try to use bin\sbcl\shen.exe:

Shen, copyright (C) 2010-2015 Mark Tarver
www.shenlanguage.org, Shen 21.1
running under Common Lisp, implementation: SBCL
port 2.5 ported by Mark Tarver
exit REPL with (cl.exit)

(0-)

An alternative would be to install bash on windows and just throw a shellscript calling the 2 exe files needed (msinstaller and sbcl). I guess you could pack all of them in a bat file.

[tizoc]

@vrescobar thank you. Don't worry about trying to make this all work as if it were a .bat file, Travis environment for building Windows stuff is just bash.

I'm giving this stuff a try in a new branch.

Did this work without any extra interaction? msiexec /i sbcl.msi /qnb Because it is just sitting there (not sure if it is taking a long time or waiting for some input)

Edit: SBCL install seems to work, it is something else that is making it hang Edit2: it does indeed hang there, commands I added after the call to msiexec don't happen, but only hangs inside Travis' environment, a friend tested this on his computer and it didn't hang Edit3: running msiexec like this worked: powershell -Command "Start-Process msiexec.exe -Wait -ArgumentList '/i sbcl.msi /qn'"

[tizoc]

WIP PR here, we can continue the discussion there: https://github.com/Shen-Language/shen-cl/pull/32

[tizoc]

Solved by #33, just tested the process and it works. Will tag a new ~2.5.1~ (2.6.0, there are more changes and the port version is a float) release so that we have binaries for all platforms now.

[rkoeninger]

@tizoc Does the version have to be a float? I think all the other ports make it a string.

[tizoc]

@rkoeninger nope, it doesn't, but since there were enough changes to justify a minor version bump instead of a patch I just did that, but I guess it is better to just change it to a string for the next version (and include major, minor and patch in the version)

[hakujin]

If helpful, https://github.com/hakujin/nix-shen provides a cross-platform way to build and install shen for systems that have the nix package manager.

[tizoc]

Good to know, thank you.