Support for cross-compiling

[sagebind]

Is there any plans on supporting cross-compiling? I'm writing a program right now that I would like to cross-compile to many different targets, but I'm having trouble getting the lua crate to compile. I see the other issues about compilation, but this is a slightly different problem.

I tried re-writing build.rs myself to see if I could get cross-compiling to work, but I ran into a problem (which is more of Cargo's fault). First, the build script itself is always is compiled for the host as the target, so we can't rely on cfg!(target_os) to determine how to compile Lua. Instead we can use the TARGET environment variable. However, the glue program needs to be compiled natively as well, since we run it immediately. I tried compiling Lua twice; once for glue, and once for the crate, but glue.rs is put in the wrong place because the value of env!("OUT_DIR") in build.rs does not match env!("OUT_DIR") in src/ffi/mod.rs since they are under different target directories.

Any advice on how to solve this? If I do the compilation steps by hand, I have successfully compiled the crate for Windows from on Linux with mingw-gcc, for example.

Also, there needs to be a mechanism of specifying the gcc to pass to make, since in many cases the system default gcc will not be used during cross-compilation.

[jcmoyer]

I'm not really familiar with the needs or process of cross-compiling. Would it be possible to use a different environment variable than OUT_DIR if it exists?

[SpaceManiac]

the value of env!("OUT_DIR") in build.rs does not match env!("OUT_DIR") in src/ffi/mod.rs

I did a little bit of investigation - the use of env!("OUT_DIR") is the problem. When build.rs is built, OUT_DIR is the non-CC directory, but when it's run, the result of std::env::var("OUT_DIR") is the desired CC directory. The buildscript needs to be changed but this isn't impossible.

Also, there needs to be a mechanism of specifying the gcc to pass to make, since in many cases the system default gcc will not be used during cross-compilation.

I believe the gcc crate is meant to be used for this purpose - I see a route Config::new().get_compiler() to produce the information on what compiler to pass to the Makefile. Looks like it might even handle the -fPIC flag automatically?

However, the glue program needs to be compiled natively as well, since we run it immediately.

This is the trickiest one. Compiling the glue program natively is likely to produce incorrect results! The whole point is to extract certain constants and types from the compiled Lua binary. I'm not sure how to best make this cross-compile amenable.

[sagebind]

I successfully modified the build script to support Linux to Windows cross-compilation at least. I haven't tested other platforms yet, but I kind of made a mess of the build script in the process. I'll make a branch with the new build script so I can show you what I did.

Admittedly, the constants' values came from a different build o Lua so there's a chance that they have the wrong value in the resulting lib, but I ran a program using the cross-compiled lib on Windows and didn't notice any immediate issues.

Edit: here's the link to the working build.rs. It's a lot messier than it was. :disappointed:

[sagebind]

You're right that the gcc crate would probably be a better choice for the gcc compilation part -- that doesn't help us tell make which gcc to use when building Lua though, does it?

[SpaceManiac]

It looks like it provides a way to get the path to the compiler that it has autodetected.

[sagebind]

Nice! It's possible to clean up the build script to be much nicer then. I'm not incredibly experienced with the ins- and outs- of this crate, but would you rather have me try and update the script?

[SpaceManiac]

I was planning to take a stab at it, but you're welcome to as well. I have the necessary to test cross-compiling.

[sagebind]

Great! I'll let you tackle it for now. :smile:

But I will stay posted and watch for updates. Cross-compilation is a must-have for my dependent project.

If it is any help, I'm working on a Docker image with a Rust and GCC cross-compiler set up. It's not perfect yet, but I used it to test cross-compilation on this crate.

[SpaceManiac]

I opened a pull request at #32 which should allow basic cross-compilation in a clean way, based partially on your branch. It'd be great if you could verify it works for you. The glue issue still needs addressing, but I'll have to think on a solution.

[sagebind]

Yes, I can confirm that #32 works very well for me. I tested it with host x86_64-unknown-linux-gnu and I could cross-compile for targets x86_64-pc-windows-gnu, i686-pc-windows-gnu, and arm-unknown-linux-gnuabihf with success.

Thanks for your time spent on this solution @SpaceManiac!