Support Native Windows

[jwoertink]

With Crystal almost at 100% support on native Windows, it's about time to start looking in to making Lucky run natively as well. This will most like require a minimum of Crystal 1.6 to work.

These are the shards that will need to be ported:

• [x] - This shard
• [ ] - LuckyCLI
• [x] - Avram
• [x] - Authentic
• [x] - LuckyFlow
• [x] - Carbon
• [x] - Pulsar
• [x] - LuckyCache
• [x] - LuckyRouter
• [x] - Habitat
• [x] - LuckyEnv
• [x] - LuckyTask
• [x] - Wordsmith
• [x] - LuckySecTester
• [x] - LuckyTemplate

[mdwagner]

I think another hat to throw in the ring is eventual WebAssembly (or WASI) support. Probably partly related to this, but also might be a little more off, not sure.

[diego-crespo]

For what it's worth I just tried building Lucky on Windows 10 manually and it failed at the shards build step with this error. I know this work isn't done so hopefully this is a useful datapoint. Looking forward to seeing Windows support soon!

Error target lucky failed to compile:
Showing last frame. Use --error-trace for full trace.

In lib\teeplate\src\lib\file_tree.cr:8:10

 8 | {{ run(__DIR__ + "/file_tree/macros/directory", dir.id) }}
        ^--
Error: error executing macro 'run':

https://imgur.com/a/fksmfzl

Crystal 1.8.1 [a59a3db] (2023-04-20)

LLVM: 15.0.7 Default target: x86_64-pc-windows-msvc

[mdwagner]

For what it's worth I just tried building Lucky on Windows 10 manually and it failed at the shards build step with this error. I know this work isn't done so hopefully this is a useful datapoint. Looking forward to seeing Windows support soon!

Error target lucky failed to compile:
Showing last frame. Use --error-trace for full trace.

In lib\teeplate\src\lib\file_tree.cr:8:10

 8 | {{ run(__DIR__ + "/file_tree/macros/directory", dir.id) }}
        ^--
Error: error executing macro 'run':

https://imgur.com/a/fksmfzl

Crystal 1.8.1 [a59a3db] (2023-04-20)

LLVM: 15.0.7 Default target: x86_64-pc-windows-msvc

Yeah, I'm already aware of this issue. Still working on a fix.

At this point, we just need these shards working on Windows first, then we can start fixing things internal to Lucky.

[jwoertink]

Working on support and one major roadblock we have is the postinstall scripts. Avram, Lucky, and a few other shards have a postinstall script which will precompile some tasks (i.e. lucky watch, lucky exec, lucky gen.migration, etc...). These all get built by running a bash script that will run shards build and take the generated binary and move it from the shard's bin directory to your app's bin directory (via cp bin/* ../../bin/).

There's a few issues going on here.

1. The postinstall on Windows https://github.com/crystal-lang/shards/issues/468 has a few issues
2. The postinstall hook could go away in the future https://forum.crystal-lang.org/t/shards-postinstall-considered-harmful/3910/92
3. We can't just run bash in Windows, but we also can't tell postinstall script to run Powershell when on windows and bash elsewhere.

So with all of this, it seems getting rid of the postinstall completely might be the way to go. This brings on new challenges.

1. If a task lives in Avram, how do you get access to it from your app?
2. How would you precompile this if you wanted to?
3. What does the "install" path for extensions look like? (e.g. installing breeze now requires 1 extra step)

One solution would be that LuckyCli would build your app's shard.yml with the extra targets of all the shards.

targets:
  app:
    main: src/watcher.cr
  lucky.gen.migration:
    main: lib/avram/src/precompiled_tasks/gen/migration.cr
  lucky.exec:
    main: lib/lucky/src/precompiled_tasks/exec.cr
  lucky.watch:
    main: lib/lucky/src/precompiled_tasks/watch.cr

Then we could have a new task lucky precompile_task which just grabs all targets that start with lucky. and then runs shards build lucky.gen.migration lucky.exec lucky.watch.

The downside to this approach is going to be existing apps having to update that file, and then also update deployments (like mine that just runs shards build would need to specify the exact target). Then developers would have to keep this list up to date. If we remove/rename a task, then that will be part of your upgrade flow.

I'm open to alternate ideas here...

edit another option that was brought up is doing what Ameba does and use executables.

https://github.com/crystal-ameba/ameba/blob/f72f0b13303da87483cccea075ef5884e175fdef/shard.yml#L12-L18

This would make the postinstall easier, but if postinstall goes away for Crystal 2.0, then we have to change all of this anyway.

[jwoertink]

Adding in some more info to this:

I do sort of like this idea. It solves the issue of having to keep this extra list up to date, and removes the need for the postinstall. If we added some sort of --build flag to the lucky cli, then we could just run and save that executable in to bin at any time.

[robacarp]

Is this an area where we could use a script which is run by the interpreter instead of compiling files?

[jwoertink]

Is this an area where we could use a script which is run by the interpreter

Most likely, yes. This is something I think would be good for the interpreter, but until it gets shipped natively with the compiler, I'm gonna hold off on attempting any sort of integrations. We could probably even do some slow roll integration to test with a CLI flag --interpret -i