Running Spiral on Linux

[mrakgr]

Continued from the PL monthly thread.

[mrakgr]

I'm not having much luck accessing the extension in VS Code. I used Ctrl+P and pasted in "ext install mrakgr.spiral-lang-vscode" (following instructions from here) but that just loaded a huge list of things that don't seem related at all. If I type "spiral" in the extension search box, I get just one result, which is "ClassiCube Script", but that doesn't seem like it's what I want. Any suggestions? (By the way, I am not at all familiar with VS Code. Also, in case it matters, I'm running it on a computer that's running Arch Linux.)

It is strange that you cannot see Spiral in the VS Code marketplace. This happened to me once while I was trying out Purescript, but as for myself, I can definitely see the plugin. Indeed, ClassiCube also shows up when searching for Spiral, but that is not it.

Did you install VS Code just recently? Maybe the package list it is using is stale...

Also can you see it right now if you type search for it? Sometime I push patches and it takes a few minutes for the new version to get verified. During this time it might be the case that the plugin does not show up. I am not sure. If this issue is persistent I'll have to open an issue on the VS Code side.

[mrakgr]

I used Ctrl+P and pasted in "ext install mrakgr.spiral-lang-vscode" (following instructions from here)

Hmmm, I don't see don't the instructions in the link. Did you mean to link to something other than the Spiral plugin? Also, Ctrl + P opens up the palette of previous files. Ctrl + Shift + P (or F1) is for the actual commands. You can also do Ctrl + P and then start the command with >.

But I am not sure where ext install mrakgr.spiral-lang-vscode is supposed to go, it is not in the usual command palette.

[mrakgr]

It would be good if you could install Spiral the usual way through the extension marketplace, that way it will get auto-updated in the background.

[mrakgr]

However, when I try to use it, I see a notification: "Spiral: The server has aborted with an error."

This happens when the process gets started by the plugin and then aborts due to an exception. The plugin shows that error if the process aborts with a return code that is not 0. It should not have anything to do with port numbers.

Unfortunately, the most likely case is some kind incompatibility between Windows and Linux. I've only tested Spiral on Windows, so I have no idea how it will behave on Linux.

It would help if you could start the process by hand from the shell and tell me what the output is. On my Windows system, VS Code unpacks the extensions raw into c:\Users\Marko\.vscode\extensions\mrakgr.spiral-lang-vscode-2.0.26 (this is the latest version at the time of writing.) The way to run it is to go into the compiler directory and run Spiral.exe. This is actually what the plugin itself is supposed to be doing.

Alternative, this error could be because the process is not being run in the first place. Maybe the command to run it outright fails for some reason on Linux due to some path differences.

[mrakgr]

Edit: Note: When I run dotnet --list-sdks on the command line, I see "3.1.108 [/usr/share/dotnet/sdk]". And dotnet --version produces "3.1.108".

This should be fine. On my system I have a few different 3.1 versions, but unless you are getting an error specifically due to this then that should not be a problem. On Windows, the OS offers to install the runtime during app start if it is missing. I am not sure how it is on Linux.

[mrakgr]

Ok, here is how I am going to handle this. Since the people I am going to be showing Spiral to will most likely be using Linux, getting it to run on Linux should be a high priority for me.

Right now I am on the very last stretch of the documentation; I just want to get this last segment out of the way so I can make room in my head. This should be done today. After that I'll try installing the Linux subsystem for Windows and give it a shot at running the plugin myself.

[ebb]

(I am ericbb on Reddit)

Did you install VS Code just recently?

Yes. I installed it yesterday in order to try Spiral.

Also can you see it right now if you type search for it?

No. It doesn't show up for searches of "spiral" or "the spiral language".

Hmmm, I don't see don't the instructions in the link. Did you mean to link to something other than the Spiral plugin?

That's the intended link. I see the instructions immediately below a heading "Installation", near the top of the page.

It would help if you could start the process by hand from the shell and tell me what the output is.

I found the following directory ~/.vscode-oss/extensions/mrakgr.spiral-lang-vscode-2.0.26 and I found compiler/Spiral.exe within it. Note that the path uses .vscode-oss instead of .vscode. I'm not sure what is the best way to execute Spiral.exe since it is not a native Linux executable file. I tried using Wine:

$ wine64 Spiral.exe
A fatal error occurred. The required library hostfxr.dll could not be found.
If this is a self-contained application, that library should exist in [Z:\home\eric\.vscode-oss\extensions\mrakgr.spiral-lang-vscode-2.0.26\compiler\].
If this is a framework-dependent application, install the runtime in the global location [C:\Program Files\dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location or register the runtime location in [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x64\InstallLocation].
...

I found the following file on my system: /usr/share/dotnet/host/fxr/3.1.8/libhostfxr.so. So based on some web searches and guesswork, I set the environment variable export DOTNET_ROOT=/usr/share/dotnet.

After that, attempting to run Spiral.exe with Wine still generates an error saying that hostfxr.dll could not be found. Note that the library I did find uses the .so extension rather than the .dll extension.

[mrakgr]

I'm not sure what is the best way to execute Spiral.exe since it is not a native Linux executable file.

Hmmm, I did compile it to be portable. It should not be run using an emulator line Wine. What should happen is that the .NET runtime should JIT it for Linux and then run it. Are you sure you can't run Spiral.exe on Linux directly?

It has been a while since I used the OS. I forgot what extensions it uses for its executable. According to this, anything.

No. It doesn't show up for searches of "spiral" or "the spiral language".

I wonder if this has anything to do with the fact that you can't run it on Linux. Maybe the extension cannot be found because it cannot be installed on Linux?

[ebb]

Are you sure you can't run Spiral.exe on Linux directly?

I have confirmed that the Spiral.exe file is not a Linux executable that just happens to use the Windows file naming convention. It is a "PE32+ executable (console) x86-64, for MS Windows" according to the file utility program, which is a tool that looks at file contents and guesses their type. Also, the file's permissions do not include the execute bit for user, group, or all (the ls command shows -rw-r--r-- as the permissions). It may be that .Net for Linux is able to execute such a file for me but I don't know how to make that happen (.Net for Linux is new to me).

Maybe the extension cannot be found because it cannot be installed on Linux?

Seems quite plausible.

[mrakgr]

Right now, I am playing with it. I tried running the Spiral.exe using WSL (Ubuntu 20.4) and it actually works without problem. Could you try running it with ./Spiral.exe?

mrakgr@Lain:/mnt/e/compiler$ dir
AsyncIO.dll              Spiral.deps.json                    it
FParsec.dll              Spiral.dll                          ja
FParsecCS.dll            Spiral.exe                          ko
FSharp.Core.dll          Spiral.pdb                          pl
FSharp.Json.dll          Spiral.runtimeconfig.json           pt-BR
FSharpx.Collections.dll  System.Net.Http.WinHttpHandler.dll  ru
Hopac.Core.dll           System.ServiceModel.Primitives.dll  runtimes
Hopac.Platform.dll       cs                                  tr
Hopac.dll                de                                  zh-Hans
NaCl.dll                 es                                  zh-Hant
NetMQ.dll                fr
mrakgr@Lain:/mnt/e/compiler$ ./Spiral.exe
Server bound to: tcp://*:13805

It starts, run for 2s and exits without error. This should be the expected behavior, as it is configured to smoothly quit if it does not get any contact from the editor without the specified limit. Normally, if there is no input, but the editor is up it gets a heartbeat message every second. Right now I am very sure that the problem is not with the server itself, rather the plugin is not starting it properly.

Maybe the extension cannot be found because it cannot be installed on Linux?

Also, I do see it from the extensions tab, so I am not sure why that problem is happening.

[mrakgr]

I tried compiling the compiler specifically for Windows and running it from WSL, and unfortunately it works. Now I can't assume that the compiler really is portable. I'd appreciate it even more now if you could run Spiral.exe and show me the output. Thanks.

[mrakgr]

Here is the current status:

• The shell is being misleading in how it can run Windows programs from within Linux.
• I've tried WSL Linux to Window TCP communication via ZeroMQ and it does not work. WSL Linux to WSL Linux TCP communication works.

This last point does not matter since for Linux compatibility I just need Linux to Linux communication to work. I spent 4h today clearing up 300Gb on my SSD so I might want to try installing Arch Linux tomorrow if there is no other choice.

[mrakgr]

mrakgr@Lain:/mnt/c/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/compiler$ ./spiral
A fatal error occurred. The required library libhostfxr.so could not be found.
If this is a self-contained application, that library should exist in [/mnt/c/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/compiler/].
If this is a framework-dependent application, install the runtime in the global location [/usr/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location or register the runtime location in [/etc/dotnet/install_location].

The .NET Core runtime can be found at:
  - https://aka.ms/dotnet-core-applaunch?missing_runtime=true&arch=x64&rid=ubuntu.20.04-x64

When I compile it for Linux specifically and try to run it from WSL I get this error. This is the framework dependent compilation. Let me try self contained.

[mrakgr]

mrakgr@Lain:/mnt/c/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/compiler$ ./spiral
Server bound to: tcp://*:13805

When compiled in self contained mode it works.

[ebb]

I'd appreciate it even more now if you could run Spiral.exe and show me the output.

$ pwd
/home/eric/.vscode-oss/extensions/mrakgr.spiral-lang-vscode-2.0.26/compiler
$ ./Spiral.exe
bash: ./Spiral.exe: Permission denied

I suppose that the error message only indicates that the file does not have execute permission.

As an experiment, I tried giving the file execute permission (using the chmod 755 Spiral.exe command) and trying again. Interestingly, it did run then but once again gave me the "libhostfxr.dll could not be found" error.

[mrakgr]

Interestingly, it did run then but once again gave me the "libhostfxr.dll could not be found" error.

This is actually a good sign. I started to think that compiling it to the portable platform makes it Windows specific and that WSL was just tricking me, but if you are getting the same error I did when it is compiled to the Linux-x64 platform, then that means that most likely that the executable is in fact taking the Linux path on Linux.

Anyway, today I'll investigate all these issues.

Yesterday I got it to run by compiling it in self contained mode and the linux-x86 platform, but the ZeroMQ subscriber socket keeps throwing an error. I'll start by looking into that.

[mrakgr]

I resolved all the issues and have just published a new version on the VS Code marketplace.

Under the hood, what I thought were issues with subscriber sockets were really Uris being invalid on Linux. They had extra / in places and this kept causing exceptions.

As for the portability issues, I found this.

https://docs.microsoft.com/en-us/dotnet/core/deploying/#publish-framework-dependent

Publishing an app as framework-dependent produces a cross-platform binary as a dll file, and a platform-specific executable that targets your current platform. The dll is cross-platform while the executable isn't.

The cross-platform binary of your app can be run with the dotnet command, and can be run on any platform.

For this to work I had to install the .NET 3.1 SDK, just the runtime was not enough. But after that I could run Spiral.dll using the dotnet command just as the docs say. Right now, I've made it so it runs the .exe file on Windows, and the .dll file on everything else.

Right now Spiral should be properly multiplatform, please give it a try.

Before I can close this issue I still have some more work to do though. It occurs to me that the Linux users have no way of accessing the server shell anymore. It opens a shell window on Windows, but nothing happens on Linux. I am going to make it so that the shell can be optionally opened in VS Code natively and push the commands through the VS Code API to it, that will be a proper multiplatform solution. This will allow me to simplify things and have everything go through the dotnet command. This will make Spiral's experience consistent for everybody compiling it from source.

[mrakgr]

Also, if possible I need to do something about Spiral not showing up on the VS Code marketplace for other people. I'll look around to see if I can find something tomorrow.

[ebb]

It worked! I was able to install the new version of the extension (using a downloaded VSIX file for now) and I was able to use the "Spiral: Build File" command to convert some sample Spiral code to F# code.

[mrakgr]

@ebb Does the plugin show up in search when you try ext:spiral?

[mrakgr]

Published a new version.

The way Spiral launches now is through the VS Code terminal rather than Node's child_process.spawn. While Spiral is in alpha I'll have it be visible by default, but there is a config option to hide it on startup in the Settings. This will make getting the feedback from the server a lot easier. I should have done it like this from the start.

I think this should be it as far as the issue of running it on Linux is concerned. But there is still the matter of the plugin not showing up for some people. And now that Spiral opens in a VS Code terminal the docs are out of date in places already. I'll deal with the docs next.

[mrakgr]

@ebb In the issue I opened it was suggested that your editor version might be configured to use OVSX rather than the Microsoft marketplace. I've published Spiral to OVSX. Does it show up in search for you now?

[ebb]

Does it show up in search for you now?

Yes it does! When I did the search and saw Spiral in the results, I also noticed an "Installing" label in the search result. After some time, it changed to "Reload Required", which I clicked on. So I suppose it noticed that I already had installed the extension through VSIX and automatically updated it for me.

[mrakgr]

Glad to hear it! Right now I am looking in the Arch Linux wiki and it does say that VS Code is configured to use Open VSX registry for it, so it can only be because of that. I'll maintain the plugin on both registries from here on out.

This should be good enough to close this issue. Thank you for taking the time to report this. Don't hesitate to open another issue if you see something else amiss.