Scripts under bin/ should run on Windows

[cmnrd]

This is closely related to #225 but I think this discussion is complex enough to deserve its own issue.

I installed the LF IDE in my Windows VM and after fixing a few things it turned out to work very well. Due to the magic of cmake, we can compile C++ programs from the IDE using MSVC (The native C/C++ compiler on Windows). The Windows Subsystem for Linux (WSL) is not required to get this working. However, now I am stuck because I can only compile LF programs from the IDE. All of our scripts in bin do not run on Windows. Most important, I cannot use the script for running tests.

I think we should carefully consider how we can make our scripts work on Windows. As far as I can see, there are 3 options that all come with their own set of drawbacks:

1. Keep everything as is and use bash scripts. This implies that the Windows Subsystem for Linux would be one of our requirements. I think this is quite a big dependency and I don't think it is safe to assume that everyone has access to it. For instance, I am stuck with an older Windows build that does not support WSL yet because the function upgrades are hold back by my administrator.
2. Provide separate powershell scripts. This is probably the simplest solution, but also doubles the effort required for writing and maintaining scripts.
3. Switch to a portable scripting language such as Python. This is my preferred option, but obviously it also adds a big dependency. However, I think python is easier to install and use on most machines than WSL (Python can even be installed without administrator privileges)

What do you think?

[Soroosh129]

There is a 4th option to add something like Cygwin as a requirement on Windows.

[cmnrd]

Right, this is also an option. However, I haven't made good experiences with Cygwin in the past. But maybe it is sufficient for running our scripts.

I think it is a question of whether we want to support Windows natively or only through a layer of tools that bring Linux compatibility. There is also the question of how we test this Windows support. I am not sure if we can easily run something via WSL or Cygwin on Travis.

[Soroosh129]

I think it is a question of whether we want to support Windows natively or only through a layer of tools that bring Linux compatibility. There is also the question of how we test this Windows support. I am not sure if we can easily run something via WSL or Cygwin on Travis.

This is a very interesting point.

I think luckily currently Travis supports WSL 1.0.1 (in fact I think it is pre-installed), as well as bash natively. But for end-users this would add an extra level of indirection. We also rely on POSIX APIs such as pthreads and sockets in the C runtime so a native Windows support could be quite a bit of work.

[cxbrooks]

Summary: If you want Windows users to use your product, then you need to provide an installer that has everything necessary. Otherwise, the support burden and level to entry can be quite high.

With Ptolemy II, we went down the Windows path quite a bit. If people will install Cygwin, then there lives will be better, but Cygwin takes configuration, see https://ptolemy.berkeley.edu/ptolemyII/ptII11.0/cygwin.htm

If you are shipping Windows installers and expect them to just work, then you probably want to not assume that Cygwin is present and provide a .bat file that starts up your application. Build scripts etc that are expected to be run by Windows users would need to be rewritten and use a tool that is part of the Windows or the install. Requiring CMake might be easier than requiring Cygwin. Ant is not that bad to ship if you are shipping a JDK anyway. Full C-based Python or is more painful, than Ant. Java-based Python (Jython) might be worth a look if you are shipping a JDK.

Writing Powershell scripts might not be that bad and might be a reasonable alternative. In the end, it might be less work than supporting Cygwin, Ant, CMake, Python or writing Java code to do the build.

The way Ptolemy II handled this might be of interest. We used configure to determine paths and what should be compiled. At the end we read in $PTII/bin/ptinvoke.in and generate ptinvoke. The ptinvoke bash script can be invoked as "$PTII/bin/makebat vergil.bat" where it will generate $PTII/bin/vergil.bat with the proper classpath. We then shipped vergil.bat with the Windows installers. One key thing is that under Unix and macOS, scripts like $PTII/bin/vergil are symbolic links to $PTII/bin/ptinvoke. The ptinvoke script can determine which link it was invoked from with:

commandname=`basename "$0"`

I'm not saying you should use configure etc, but being aware of how to have a common script could save you a lot of trouble is all your scripts have a lot in common.

While I'm at it, if you use Cygwin, then paths are a nightmare. Under Windows, you definitely need to support spaces in filenames and pathnames, so everywhere you use variables that are paths probably need to be double quotes. You also need to watch out for different path separators. Under Cygwin, the c: drive is \cygdrive\c, which can cause no end of trouble.

If it were me, I'd look long and hard at using Powershell and having the nightly build test it.

[cmnrd]

We also rely on POSIX APIs such as pthreads and sockets in the C runtime so a native Windows support could be quite a bit of work.

Yes, it would probably require much more work to get the C runtime work natively on Windows. However, for the C++ runtime and probably also TS, this is not such a big deal. C++ uses the std::thread API and relies on cmake, which makes it portable. So while it might be hard to get native Windows support for the C runtime, I don't think we should let this block native support for other targets.

Thanks @cxbrooks for the insights. I agree, ideally we want to have an installer somewhere in the future. Leveraging a portable build system such as Cmake or Ant would also be an option for building lfc and running the tests. The C++ target requires cmake anyway, so requiring it to be installed for building lfc or shipping it with the installer shouldn't be too much of a problem.

[lhstrh]

The lfc script in bin has been drastically simplified. As a result, it should be easy to write a Windows-compatible counterpart. Gradle is using a very similar approach; we should have a close look at their wrapper script gradlew. I don't have access to a Windows installation at all. Would this be something you could take on, @cmnrd, perhaps with help from @efsanesoyer?

[cmnrd]

Yes, I can give this a try.