Publishing `libindigo-sys` FFI bindings on crates.io

[chrsoo]

Hi,

As mentioned in #561, I have been experimenting with wrapping INDIGO in a Rust library and am now ready to share this highly experimental code with others, even if it is not yet ready for using in any real project.

It is available in my public chrsoo/libindigo-rs git repository.

There are many open issues. One is the very large size of the INDIGO code base.

Rust's package manager only distributes source code packages that must be compiled locally before use, and when publishing Rust crates that wrap a C-library on crates.io, all C source code files are required to be included in the crate.

The alternative is to assume that libraries and headers pre-exist on the local platform where the Rust crate is used. Given that Rust does not exist locally on most platforms, this means that a Rust user would have to download and compile the upstream project before use. Ideally this is avoided and the source code is included in the Rust crate.

I have added INDIGO as a git submodule of the libindigo-sys crate. Cargo builds the INDIGO C-library using make all invoked from the build.rs Cargo build script. This works fine though it would be good to remove the need for tools apart from the Rust tool-chain.

When publishing the crate to [crates.io] we hit a snag as the INDIGO C-library is a whopping 300 MB of compressed source code and the limit is 10 MB...

The solution is to pair down the source code included in the libindigo-sys crate to the bare minimum required for the Rust library, I think we safely could exclude all drivers included in the base distribution. Ideally we would only include the INDIGO base library, possibly it could make sense to include the CLI as it would mean that you could easily compile and install it using Cargo.

Could you please help me define which source code files must be included and what combination of make commands would build the relevant libs?

Apart from the source code, we must of course also include LICENSE.md and other files. Note that naively invoking make indigo_libs does not work for building the indigo libs... but I could experiment with building directly from Rust.

[polakovic]

If I do understand correctly what are you trying to do, you probably need just source files under indigo_libs folder. But you will need to write new Makefile as the current one is very complicated for this purpose. It among other downloads some 3rd party libraries as binaries, builds other 3rd party libraries from source, builds all drivers etc.

To build just INDIGO libraries, invoke make in indigo_libs folder. But it needs ../Makefile.inc to be present and this file is created by top level Makefile.

[chrsoo]

Thanks, yes this was my guess as well. I am familiar with make, but the challenge is working with C targeting multiple architectures... Ideally I would just skip make and compile from inside Rust.

Also I am starting to think that it is perhaps best on requiring that any Rust user just have to download and compile the base INDIGO distribution and then rely on the compiled system libraries and header files. It would be a pity though, because it could be self-contained by including the source in the Rust crate published on crates.io.

Anyhow, we will see where I end up...

Right now I first need to figure out why building vanilla INDIGO suddenly stopped working for me. I think it might be related to an OS/Xcode upgrade but I was messing around with Homebrew as well, so I am not 100% sure.

Prior to that all worked fine, but now I get the following error building on Mac:

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [fax2ps] Error 1
make[2]: *** [install-recursive] Error 1
make[1]: *** [/Users/csoop/src/astronomy/libindigo/sys/externals/indigo/build/lib/libtiff.a] Error 2
make: *** [all] Error 2

Not sure what that x86_64 is doing in there as I am on Apple silicon (M1/arm64).

Any idea how to get around this hurdle?

Is the build targeting universal binaries? Is there a way of forcing arm64 only?

[rumengb]

If you do not intend to warp drivers you may want to compile only the client library only. it will be much smaller and it would not require many dependencies. We build it for windows and you can see how it is built and which files are needed here: indigo_windows/gcc/Makefile