Build is possible, but overly difficult, on Windows

[tommyettinger]

I think this qualifies as a legitimate issue, but feel free to close if you don't wish to support Windows as a target. I managed to build Thorin on Windows 7, 64-bit, using VS Community 2015 (version 14 compilers, 64-bit). I don't know how to test it yet, and it may not work as-intended, but it does compile. However, it required multiple installations of unrelated software packages, mostly available through Chocolatey/NuGet/OneGet (on Windows 7, I use Chocolatey to manage packages for C/C++-built tools), but not universally (xz needed extra work). I'm not sure if half actually needs to be built, since it's header-only, so I am unsure why it includes a (rather out-of-date) Visual Studio solution. Some extra work was needed to build half because I didn't have doxygen, and without it the whole build mysteriously failed, giving the error "cmd.exe" exited with code 3. After updating the version-11 VC++ project files to version 14, hand-editing the one relevant vcxproj file to omit the custom build step that invoked doxygen, and then re-opening in VS and building, half compiled (though I suspect I didn't need to do any of this).

Then came the significantly more-frustrating task of navigating cmake for thorin. I was able to point it to the include directory of half without issue, but setting LLVM_DIR was completely ignored, and this consistently repeated no matter how explicitly I specified that variable. The solution was to put the bin folder of the already-built LLVM installation on my PATH env var (presumably this allowed llvm-config to be run), and give the llvm_install directory as LLVM_DIR (though this may have been overwritten anyway). I had built LLVM using the AnyDSL/anydsl scripts, which needed the manual download of xz to extract LLVM and the package installation of svn to acquire half (as well as some fiddling to get svn to actually download any files), and then some changes to how LLVM built so it used the correct settings for msbuild to successfully compile.

The closest thing to an actual bug here, as opposed to a mind-melting layer-cake of frustrations that are, still, solvable given enough patience, is that the advice cmake gives to set LLVM_DIR is incorrect. So is the description of LLVM_DIR as the directory that contains the cmake files for LLVM (setting it to that directory doesn't seem to do anything differently). What would be nice is if some of this build was done in a more unified way, like forking half so it's on GitHub instead of hoping that SourceForge won't be as unreliable today as it has been recently, and including half, Thorin, and maybe Impala as git submodules of anydsl.

Other than the work needed to get the code operational, the science that is the backbone of Thorin looks really excellent. I've linked the Thorin paper to a friend who is working independently on a graph-related programming language/platform, and he's impressed, as am I. I found this project through the Bangra IR project, which doesn't seem to use your code but is based off the paper, and apparently Leonard Ritter found very significant reductions in code size/effort when switching Bangra's closures to the method used in Thorin, as your team had found in the paper. I'm hoping to at some point implement a hybrid imperative/functional/array language using Thorin or, once it's more-ready, Bangra, but Impala looks like an excellent idea for easing DSL implementations.

So, sorry for venting a bit, but I hope there's a way to ease the Windows build process for other potential users. Thanks.

[madmann91]

There is also the less painful option to use MinGW to build AnyDSL on Windows. If you do not need CUDA, it's the way to go as it simplifies pretty much everything.

[wizzard0]

well, @tommyettinger thanks for venting! I got stuck on LLVM_DIR too, will continue again this week then @madmann91: including dependencies as submodules, possibly from forks, with proper versions, could simplify things a lot

[stlemme]

For CMake's find_package(XYZ), which programmatically adds the XYZ_DIR to CMake variables, you must set the path to the directory containing one of the files XYZ-config.cmake or XYZConfig.cmake. Usually, libraries that support CMake ship with such a file in the root directory of the lib. However, LLVM places the respective file at %LLVM_SOURCE_DIR%/my_build_dir/share/llvm/cmake when building LLVM yourself or %LLVM_INSTALL_DIR%/share/llvm/cmake after building the install target.

[madmann91]

I see your point that the installation is rather difficult on Windows. Apart from going through the difficult manual installation steps, as you did, the only fallback at the moment is to use MinGW + MSYS + Git and clone the AnyDSL/anydsl repository. This gives you a minimal UNIX-like environment which allows to run the bash setup script setup.sh (which downloads and sets up everything). We will try improve the support for Windows and will probably add the dependencies as submodules once we find the time. Meanwhile, if you want to contribute, feel free to add your own setup script or anything that makes your life easier as a pull request, we will review it. We will keep this issue open until it is solved.

[leissa]

The problem is that our core team currently consists of 2-3 persons who all work with Linux. So it's a bit difficult for us to maintain windows-specific stuff. That being said I second @madmann91 's comment.

[tommyettinger]

I'll try starting fresh-ish with MinGW, maybe making a list of things I needed to install to get it going. It might be useful to have a list like that on the wiki.

Out of curiosity, is there a primary intended use for DSLs made with AnyDSL? It seems like if there's a Linux/Unix focus, then server-side applications would fit well, especially as a sort of "bridge" that allows code written by a domain expert in the DSL to be sent to some server that can evaluate the AnyDSL-based language, then either run directly (altering a database or giving feedback to the expert) or translated to some other format that would be too difficult to write by hand. An example of this last approach already in use is Shapeways' DSL for 3D model authoring, which allows a simpler way to procedurally generate 3D model files, without needing a dedicated 3D modeling program for some tasks. I believe it is or was called ShapeJS and was JSON-based (might be a JS library), but I haven't used it yet. Their DSL is, as far as I am aware, meant to be run using their servers, but might have alternate ways it can be run. There are probably lots of similar cases in production all over various technical fields.

Speaking of graphics programming, I'm not sure if I need to build Stincilla to use Impala, but AnyDSL seems to need it. I'll try doing a completely-MinGW build of Thorin, since it seems to be at the core, and maybe try an MSYS2 build if I can't get far enough with MSYS version 1 and MinGW. If that gets working, I can move on to Impala...

[leissa]

Stincilla is a DSL built with AnyDSL. You don't need it.

It's not like Linux/UNIX is our focus, it's just that all of us are currently Linux users :)

Anyway, if you want to have a sketch on how to setup things, you can orientate yourself on the setup script we use under Linux: https://github.com/AnyDSL/anydsl/blob/master/setup.sh