GDNative: support multiple OS builds for plugins

[willnationsdev]

Plugins created through GDNative will need to have a particular build for any given desktop platform.

This means that creators will need to be uploading versioned, OS-specific builds for their GDNative plugins. It also means that users will need to be able to automatically download the latest stable release of a build for their current platform (which may not even exist!) as well as optionally download other versions / OS's libraries should they desire to manually.

The Asset Library doesn't currently support this variability, but it WILL be needed if 3.0 is to truly take advantage of GDNative's potential. Otherwise no one will easily benefit from others' incredible work.

[Calinou]

How are multiple compilers going to be handled (MSVC versus MinGW, GCC versus Clang)? Which platforms will we require pre-built libraries for (as building for all platforms is not a straightforward process)?

[reduz]

As far as I understnand, you just download the asset with all the versions for all OSs bundled.

[BastiaanOlij]

I don't think you'd want people who download GDNative module from the asset store to go through compiling. If they want the latest and greatest compiled there is far more involved.

The way I see it is that you would store the compiled binaries as part of the asset and when you download it you get the binaries for all platforms with a GDNLIB file setup correctly. The exporter will then pick out the right binaries for deployment.

It will however be important that we know the Godot version for which the binary was made. While the GDNative API should be frozen once Godot 3.0 is released and a 3.0 GDNative library should work in 3.1, the other way around is unlikely to work.

[BastiaanOlij]

Just to add a remark from Akien into the mix, possibly this should be an issue in its own right but maybe having one issue for getting GDNative modules into the asset store makes things easier.

As I understand it right now the asset store requires you to put your asset into its own Github repository which the asset store then uses. This involves putting binaries into that repository which is frowned upon.

Need to think about how we'll address that one or if we just accept that as the way it is.

[Calinou]

GitHub Releases could be used by the developer to upload compiled libraries, which the asset library would then use. This means that the asset library (and the developer) would also have to use tagged releases.

Another problem is that not all developers are able or willing to compile for all platforms and architectures out there.

[BastiaanOlij]

@Calinou that is indeed a problem for me. I compile the openvr libraries for windows, and Ben does the linux one for me, but I don't have the setup to do all cross platform ones. Now with openvr that would only add Mac and I only can't do that because I don't have the hardware to test if it works

[bojidar-bg]

Thinking about it, for every plugin which has a runtime gdnative libraries (as opposed to editor-only libraries), the user will need to download it for all platforms when exporting to them anyway. Since editor-only plugins are rare, I think it would be better if the asset library does not care about OS-es.

[herman-rogers]

We could potentially leverage this in a continuous integration system that compiles all needed platforms (I'm assuming the server editor can do this) and then upload each binary to a location the asset store could then access.

[Calinou]

Compiling code on the server would be too resource-intensive (we only have a TuxFamily instance provided for free) and possibly insecure.

It's possible to point people to existing CI platforms, but it would probably end up being difficult to set up.

[herman-rogers]

Well theoretically this could be done with the headless server mode correct? Take the project code, compile it for Linux, Mac, Windows, zip the binaries up to a storage location and have it available for download. Then when downloaded and extracted you'd have all three binaries there ready to use.

As for security, if you're using a service like AWS and GCE then you should be fine, plus a lot of the assets will be publicly available (we can also make sure that its just code rather than binaries being stored in their repos).

Edit: I understand that the current services don't have that sort of resource allocation for compilation - just discussing this academically.