Haxe 4.2.5 installation terminates with error pointing to Neko

[rat-moonshine]

@JustinProminic reported the following error while trying to install Haxe 4.2.5:

Installer path: /Applications/MoonshineSDKInstaller.app/Contents/Resources
Installer version 4.5.0 (mac)
Using Locale: en_US
Fetched the SDK download mirror URL from the CGI.
Downloading Haxe SDK v.4.2.5
Starting to untar: /Users/justinhill/Library/Application Support/net.prominic.MoonshineSDKInstaller/Local Store/haxe-4.2.5-osx.tar.gz
Completed untar Haxe SDK
Moving files to SDK directory /Users/justinhill/Downloads/MoonshineSDKs/Haxe/haxe-4.2.5
Downloading Neko SDK v.2.3.0
Starting to untar: /Users/justinhill/Library/Application Support/net.prominic.MoonshineSDKInstaller/Local Store/neko-2.3.0-osx64.tar.gz
Completed untar Haxe SDK
Moving files to SDK directory /Users/justinhill/Downloads/MoonshineSDKs/Haxe/neko-2.3.0
Deleted temp files.
Installation complete
Now continues with Haxe/Neko setup
Please enter haxelib repository path with write access
Hit enter for default (/usr/local/lib/haxe/lib)
Path : 
haxelib repository is now /Users/justinhill/Downloads/MoonshineSDKs/Haxe/lib

A new version (4.1.0) of haxelib is available.
Do `haxelib --global update haxelib` to get the latest version.

Downloading feathersui-1,2,0.zip...

Installing feathersui...

  Current version is now 1.2.0

Done

Installing dependency actuate 1.9.0

Downloading actuate-1,9,0.zip...

Installing actuate...

  Current version is now 1.9.0
Done

Installing dependency hxargs 4.0.0

Downloading hxargs-4,0,0.zip...

Installing hxargs...

  Current version is now 4.0.0

Done

Installing dependency openfl 9.3.2

Downloading openfl-9,3,2.zip...

Installing openfl...

  Current version is now 9.3.2

Done

A new version (4.1.0) of haxelib is available.
Do `haxelib --global update haxelib` to get the latest version.

You already have openfl version 9.3.2 installed

A new version (4.1.0) of haxelib is available.
Do `haxelib --global update haxelib` to get the latest version.

You already have actuate version 1.9.0 installed

A new version (4.1.0) of haxelib is available.
Do `haxelib --global update haxelib` to get the latest version.

Downloading lime-8,1,1.zip...

Installing lime...

  Current version is now 8.1.1

Done

Installation error: dyld[13132]: library not loaded: '@rpath/libneko.2.dylib'
  referenced from: '/users/justinhill/downloads/moonshinesdks/haxe/haxe-4.2.5/haxelib'
  reason: tried: '/users/justinhill/downloads/moonshinesdks/haxe/haxe-4.2.5/libneko.2.dylib' (no such file), '/users/justinhill/downloads/moonshinesdks/haxe/haxe-4.2.5/libneko.2.dylib' (no such file), '/usr/local/lib/libneko.2.dylib' (no such file), '/usr/lib/libneko.2.dylib' (no such file

[rat-moonshine]

While testing Haxe 4.2.5 installation in a fresh environment (no /usr/local/lib/haxe or /usr/local/lib/neko), I couldn't able to reproduce the problem at my side.

The installation completes successfully adding following folder/files under MoonshineSDKs folder:

I wonder why I didn't receive the error that @JustinProminic pointed to; since the error related with dyld I remember we had such problem in past.

While looking into the installer script, I see we have a portion that pointed to dyld fix, but I can't recall now why we comment-off the section - https://github.com/Moonshine-IDE/Moonshine-SDK-Installer/blob/master/MoonshineSDKInstaller/src/installer/haxe-installer.xml#L180 .

We also needs a system where we can reproduce the problem and possible fixes.

[JoelProminic]

I see we discussed a similar error in the Moonshine-IDE repository: https://github.com/Moonshine-IDE/Moonshine-IDE/issues/967

In particular, see this test procedure.

[rat-moonshine]

I finally able to reproduce @JustinProminic problem while running in IDE-debug mode. It's possible that something influencing while in installed-run and the Haxe installation completes successfully; But in IDE-debug run, installation proceed without any influence.

I able to overcome the original problem by providing Neko symlinks to Haxe path - things that we did in Moonshine - https://github.com/Moonshine-IDE/Moonshine-SDK-Installer/issues/13#issuecomment-1030416821 .

But when installation proceed, it failed in another place:

haxelib run openfl setup -y --quiet

Do you want to install the "openfl" command? [y/n/a] y
Installation error: cp: /usr/local/bin/lime: permission denied

I didn't found any corresponding discussion over internet, unfortunately. So I think I can bring only to @joshtynjala for any suggestion, I'm trying to explain here:

1. As part of the Moonshine SDK Installation, Haxe, Neko and other items all installed under /Users/$user/Downloads/MoonshineSDKs/Haxe folder:

2. I renamed all these existing items from default locations, to ensure they do not influence the MSDKI Haxe installation:

   • /usr/local/lib/haxe
   • /usr/local/lib/neko
   • /usr/local/bin/haxe
   • /usr/local/bin/haxelib
   • /usr/local/bin/lime
   • /usr/local/bin/neko
   • /usr/local/bin/openfl

3. At a point after installing OpenFL, Neko, Lime, the installer script tries to run haxelib run openfl but return with an error:

   
   haxelib run openfl setup -y --quiet

Do you want to install the "openfl" command? [y/n/a] y Installation error: cp: /usr/local/bin/lime: permission denied

@joshtynjala can you give me an idea what step I'm missing here (?)

[joshtynjala]

I can run haxelib run openfl setup from a terminal, and it successfully copies the openfl script to /usr/local/bin. I wonder if the macOS sandbox is preventing Moonshine from writing to /usr/local/bin. It might require sudo or to request permission from the user to access that file/directory before running the command.

[rat-moonshine]

It might require sudo or to request permission from the user to access that file/directory before running the command.

But why it requires /usr/local/bin when the installation targeted to ~/Downloads/MoonshineSDKs/Haxe/ - and installed to the directory (as in earlier screenshot shown) as well.

sudo is not an option while run from MSDKI, unfortunately. I tried to run our installation script on Terminal, and I see following additional output, when the particular command triggered:

layout is up to date
Do you want to install the "openfl" command? [y/n/a] y
cp: /usr/local/bin/lime: Permission denied

To finish setup, we recommend you either...

 a) Manually add an alias called "openfl" to run "haxelib run openfl"
 b) Run the following commands:

sudo cp "/Users/devsena/Downloads/MoonshineSDKs/Haxe/lib/lime/8,1,1/templates/bin/lime.sh" /usr/local/bin/lime
sudo chmod 755 /usr/local/bin/lime
sudo cp "/Users/devsena/Downloads/MoonshineSDKs/Haxe/lib/openfl/9,3,2/assets/templates/bin/openfl.sh" /usr/local/bin/openfl
sudo chmod 755 /usr/local/bin/openfl

Set MAC_USE_CURRENT_SDK to 1

[joshtynjala]

But why it requires /usr/local/bin when the installation targeted to ~/Downloads/MoonshineSDKs/Haxe/ - and installed to the directory (as in earlier screenshot shown) as well.

It's hard-coded by Lime/OpenFL's command line tools to copy their executables to /usr/local/bin on macOS and Linux.

They do not use the value of the HAXEPATH environment variable on macOS or Linux to figure out where to copy their executables, like they do on Windows. On Windows, it's expected that HAXEPATH will have been added to the PATH environment variable (by the Haxe installer, or manually, if customized). On macOS or Linux, that's not the case. Instead, the Haxe installer creates symlinks to haxe and neko in /usr/local/bin. So there's no way for Lime/OpenFL to detect a better directory to copy the executables on macOS and Linux than the default.

We could try to skip running haxelib run openfl setup in Moonshine. Technically, openfl build commands can be run by prepending haxelib run. For example: haxelib run openfl build html5. I think I may have even had Moonshine do this by default so that things would still work for someone who manually installed Haxe outside of MSDKI (like I do), but chose not to (or forgot to) run haxelib run openfl setup. So maybe things will "just work" without the setup command.

There is no way to skip copying those executables when running the setup command. However, it's strange is that it's not displaying the same "To finish setup, we recommend you either..." suggestion in Moonshine. I don't know why it wouldn't. The Lime/OpenFL command line tools should catch errors in the cp command automatically, and it appears that they are when you run them in the terminal. So why not Moonshine? I can't explain that.

I tried to run our installation script on Terminal, and I see following additional output, when the particular command triggered:

It's interesting that it's working successfully on my machine, but not yours. However, based on some research, it appears that some non-standard macOS tools may make /usr/local/bin writable as part of their installation (including, possibly, the Intel version of Homebrew, while the Apple Silicon version of Homebrew uses a different installation path instead). It seems that /usr/local/bin is probably read-only on your machine (unless using sudo), but some tool I ran made it writeable on my machine.

[JoelProminic]

Now I see we are getting the same error I saw in #113.

I tried adding a symlink for lime in the haxe-4.2.5 directory (which is on the PATH), but that didn't help (which makes sense with @joshtynjala's clarification).

cd ~/Downloads/MoonshineSDKs/Haxe/haxe-4.2.5
ln -s ../lib/lime/8,1,1/templates/bin/lime.sh lime
chmod 755 lime

Then I started testing some of the suggestions from the command output (I was also wondering why these instructions are not displayed in MSDKI. I though that they might be on STDERR, but that doesn't seem to be the case):

# My test command.  -quiet didn't seem to hide much in this case anyway:
haxelib run openfl setup -y

# I tested this in isolation, but there was no difference
export MAC_USE_CURRENT_SDK=1

# This didn't help either
alias openfl='haxelib run openfl'

# Then I tried the sudo commands as a sanity check, but they also didn't work!
sudo cp "/Users/joelanderson/Downloads/MoonshineSDKs/Haxe/lib/lime/8,1,1/templates/bin/lime.sh" /usr/local/bin/lime
sudo chmod 755 /usr/local/bin/lime
sudo cp "/Users/joelanderson/Downloads/MoonshineSDKs/Haxe/lib/openfl/9,3,2/assets/templates/bin/openfl.sh" /usr/local/bin/openfl
sudo chmod 755 /usr/local/bin/openfl

@rat-moonshine saw the same errors with the sudo commands. I restarted my Terminal (from MoonshineDevelopment) to rerun a test with the sudo commands, but this still didn't work. I confirmed that /usr/local/bin/lime exists and can be run from the same Terminal.

My Haxe environment variables look like this:

% set | grep -i haxe
DYLD_LIBRARY_PATH='/Users/-snip-/Downloads/MoonshineSDKs/Haxe/neko-2.3.0:/Applications/HCL Notes.app/Contents/MacOS'
HAXE_HOME=/Users/-snip-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.5
NEKO_HOME=/Users/-snip-/Downloads/MoonshineSDKs/Haxe/neko-2.3.0
PATH='...:/Users/-snip-/Downloads/MoonshineSDKs/Haxe/haxe-4.2.5:/Users/-snip-/Downloads/MoonshineSDKs/Haxe/neko-2.3.0:/usr/local/bin:/Users/-snip-/Downloads/MoonshineSDKs/Haxe/neko-2.3.0:/Applications/HCL Notes.app/Contents/MacOS:...:/usr/local/bin:...

[joshtynjala]

I'm guessing that this output is on stderr:

cp: /usr/local/bin/lime: Permission denied

I tracked down HaxeInstallHelper in Moonshine-SDK-Installer, and I see that it aborts if there is any output on stderr. However, I suspect that the openfl setup command would finish successfully with exit code 0, if it were allowed to complete.

[JoelProminic]

Aborting on any STDERR output does seem like a bad idea, but I think it was required for one of the installers at some point.

I confirmed that the manual command finishes with exit code 0, but did it actually work? Is there a good test command we can use to confirm this?

% haxelib run openfl setup -y
...

layout is up to date
Do you want to install the "openfl" command? [y/n/a] y
cp: /usr/local/bin/lime: Permission denied

To finish setup, we recommend you either...

 a) Manually add an alias called "openfl" to run "haxelib run openfl"
 b) Run the following commands:

sudo cp "/Users/joelanderson/Downloads/MoonshineSDKs/Haxe/lib/lime/8,1,1/templates/bin/lime.sh" /usr/local/bin/lime
sudo chmod 755 /usr/local/bin/lime
sudo cp "/Users/joelanderson/Downloads/MoonshineSDKs/Haxe/lib/openfl/9,3,2/assets/templates/bin/openfl.sh" /usr/local/bin/openfl
sudo chmod 755 /usr/local/bin/openfl

Set MAC_USE_CURRENT_SDK to 1
% echo $?
0

If this does work, then we still have the issue where the error is displayed in the MSDKI console (#113).

[joshtynjala]

I confirmed that the manual command finishes with exit code 0, but did it actually work?

As a maintainer of Lime/OpenFL, I am saying that yes, it did actually work. Any sub-command launched by by haxelib run openfl setup will result in a non-0 exit code if that command fails — unless that failure is specifically caught and ignored by openfl setup. The copying of lime.sh/openfl.sh is one case where it is caught intentionally (and the message is displayed instead) and so is not considered a failure.

Copying of lime.sh/openfl.sh is mainly for convenience (and because some other IDEs expected it in the past). As I mentioned, haxelib run openfl build html5 also works instead of openfl build html5, and I think Moonshine already uses the haxelib run version for safety). However, if we want to be extra careful, Moonshine could manually copy lime.sh and openfl.sh to somewhere it knows will be on the PATH, and that will be equivalent.

Aborting on any STDERR output does seem like a bad idea, but I think it was required for one of the installers at some point.

I don't doubt that some commands always return an exit code of 0, even for failures, but that is uncommon for developer tools. We should probably assume that stderr is not a failure if the exit code is 0 unless we know/learn otherwise for a specific command. Then, we can make a special case for that command only.

If this does work, then we still have the issue where the error is displayed in the MSDKI console (https://github.com/Moonshine-IDE/Moonshine-SDK-Installer/issues/113).

The "Installation error:" message should not be displayed anymore if HaxeInstallHelper.shellError stops dispatching EVENT_INSTALL_ERROR when detecting stderr output.

[rat-moonshine]

As we had two possible option to overcome this problem -

1. Omit haxelib run openfl setup from MSDKI installation process
2. Skip the stdErr that triggers during the particular command run

As I do not likes the idea to remove the command from running - as this anyway installs/configure different things, I keep the command. But I overlook the stdErr during this command execution, though. That completes the MSDKI installation process successfully.

I noticed one odd sequence during the installation process, though; I thought to keep that in document:

1. MSDKI Haxe setup completes successfully
2. Opened Moonshine-IDE and created a new OpenFL/Feathers project
3. Console prints some trouble during the project open - unfortunately, I didn't copied that message at that time, that is my bad; And, I noticed that code-intelligence not working
4. I ran "Build & Run"
5. Moonshine compiles the project successfully, and starts the application
6. I tried to re-open the project to see the error again - but this time project opens properly and code-intelligence also works
7. In despair to get that once seen message again, I removed everything from the Haxe installation, Even restarted the system
8. Re-installed Haxe from MSDKI
9. But I couldn't able to get the once seen error message anymore - project opens correctly with all its code-intelligence

I commit my changes, please check if someone can see an error.

[JoelProminic]

I ran a test, and the Haxe SDK install did not trigger any errors in MSDKI. I saw that the cp: /usr/local/bin/lime: Permission denied message was missing in the output. The STDERR output could contain important errors or warnings, so we still need to display it:

Installing layout...

  Current version is now 1.2.1
Done

Do you want to install the "openfl" command? [y/n/a] y

To finish setup, we recommend you either...

 a) Manually add an alias called "openfl" to run "haxelib run openfl"
 b) Run the following commands:

sudo cp "/Users/-snip-/Downloads/MoonshineSDKs/Haxe/lib/lime/8,1,1/templates/bin/lime.sh" /usr/local/bin/lime
sudo chmod 755 /usr/local/bin/lime
sudo cp "/Users/-snip-/Downloads/MoonshineSDKs/Haxe/lib/openfl/9,3,2/assets/templates/bin/openfl.sh" /usr/local/bin/openfl
sudo chmod 755 /usr/local/bin/openfl

I tried creating a Haxe OpenFL Project and a Haxe Feathers UI Project, but I didn't get a language server error on either.

[JoelProminic]

The STDERR output should be fixed now. I'll retest it when I have a chance.

[JoelProminic]

I didn't see any problems with this for my last test here: https://github.com/Moonshine-IDE/Moonshine-SDK-Installer/issues/111#issuecomment-1939775181