search

Radfordhound / HedgeLib

A C++ library and collection of tools that aims to make modding games in the Sonic the Hedgehog franchise easier.
MIT License
88 stars 24 forks source link

Generate C# bindings using CppSharp #41

Closed Radfordhound closed 5 years ago

Radfordhound commented 5 years ago

As stated in the HedgeLib++ announcement video, I wish to have AppVeyor generate C# bindings for HedgeLib using Mono's CppSharp. I haven't looked into how to do this specifically yet, though I don't imagine it's too hard.

Radfordhound commented 5 years ago

Well, actually, it turns out this is going to be a major nightmare. (This reply is long, so I've written some bolded TL;DR versions at the end of each section)

Alright, so HedgeLib++ has been going really well so far; I'm really happy with the benefits C++ provides, such as direct loading, compiling to native binaries (so no requirement to install the .NET framework or .NET core and very optimized machine code), the ability to use any C/C++ library (Qt is super nice), etc.

However, I of course still want people to be able to use HedgeLib from a C# environment. C# has some major benefits, too, especially now with .NET core and such. While I believe the end result is way nicer in C++, the development experience is way nicer in C#. If I just want to quickly slap together a personal tool for a specific use-case I'd much rather do it in C# as I'll quite literally get the same task done, just way faster.

TL;DR: I still want to have a way to use HedgeLib from C# in addition to C++.

The Problem

I've been aiming to make HedgeLib++ a C++17 library, not a C library that's just being compiled from a C++ environment. Because of this HedgeLib++ frequently uses certain C++ features such as constant expressions, smart pointers/RAII objects, and ofc stuff like templates and vectors.

Templates

Turns out CppSharp doesn't really support templates despite its wiki saying otherwise. Therefore I can't generate C# bindings using it - I just get tons of errors.

CppSharp is probably the best method of generating C# bindings out there, so this is quite a problem. Though I don't say that to diss CppSharp - it's a very understandable limitation, actually! I didn't really think about it until I ran into errors, but honestly, you can't really make template bindings like that.

See, C++ generates templates at compile-time based on all the types used in the code. If a variant of a template isn't used in the code (let's say Foo<int>) the compiler won't include it in the compilation.

CppSharp generates C# bindings from C++ headers and compiled C++ objects. How could it possibly generate a C# generic for Foo<int> if Foo<int> isn't used anywhere in the C++ source and therefore isn't available in the compiled C++ object? It can't just compile a new version of Foo and add it to the compiled object - it's already been compiled!

Even if it generated from C++ source code instead of C++ objects, it's impossible for it to know exactly what types the C# users are going to use ahead of time. Just think about it. Like legit, just stop and think about how CppSharp could even do this. It honestly melts my brain lol.

TL;DR: CppSharp doesn't really support generating C# bindings for C++ templates, because of course, how could it possibly even do that?

.NET Core

.NET Core is Microsoft's new cross-platform implementation of the CLR. Note the "cross-platform" part.

If you compile a purely C# library you end up with a purely managed DLL containing CIL that can be executed by .NET Core on any platform. If you compile C++ bindings, however, you need to also include the native DLL the C# library is going to call upon.

With the .NET Framework this is a minor inconvenience, but it's nbd. Just include a native Windows DLL and a C# binding. But with .NET Core, to work properly cross-platform, you'd need to have a separate set of bindings for each target platform as you need to call upon separate DLLs, each with their own name-mangling as well since they're going to be compiled by different compilers.

So that means 3 native DLLs and 3 managed bindings for them (one for Windows, Mac, and Linux). That's just bad lol. Like I don't even need to say anything else about it.

TL;DR: Can't do a cross-platform .NET Core version unless you have a separate set of C# bindings + native DLLs for each target platform. Ew.

Alternatives

So basically, we can't use CppSharp as it doesn't support what we need (mostly templates) and can't without jumping through major hoops.

What can we do instead? Here's a few alternatives I thought of:

Write a C wrapper for HedgeLib++

Pros:

Cons:

By far the worst option imo. Just listed it for completeness.

Write a C++/CLI binding

Pros:

Cons:

If we don't care about cross-platform support, .NET Core support, making a NuGet package, or future-proofing the bindings, this could work. Otherwise... no.

Just port it to C# lol

yes I know what you're thinking inb4 the jokes

hear me out

Pros:

Cons:

kill me but I'm actually kinda leaning towards the third option...

I know that sounds insane lol but I don't think it would actually be that bad once it's caught up to the current HedgeLib++, since from there all we'd have to do is make the same changes to HedgeLib# that we just made to HedgeLib++ - which is bad, yes, but you have to do that no matter what approach you use, even CppSharp is technically just automating this.

Maybe we could create a tool to automatically generate a port rather than just bindings to get around the issues I listed previously with CppSharp/other types of bindings? In that case it really wouldn't be bad.

Sorry for the massive reply here lol. My question is: what are your thoughts? I'd really like to hear them before I go and do something like this.

IchigoDemandsFurryBoobs commented 5 years ago

Honestly i think going back to HedgeLib# is a pretty stupid(no offence) thing to do at this point because it literally means taking a big step backwards(that is if im getting this right on my sleep deprived hours™), i personally think the best option is the CLI binding, since it is not a big deal to download the library from appveyor and not having the posibility to use NuGet since you need internet connection anyways, and to be fair i can not imagine someone trying to mod HE games on a linux system, since to begin with, no unix-based OS can run HE games at all, probably not even with Wine from what i know(i have had a linux system years ago, i tried to run gens), and as for mac, kinda the same lol; Anyways this is just my personal opinion, this is your project so i believe you can go for whatever option you want to, have a good day, BlackMail Boi™.

Radfordhound commented 5 years ago

Thanks for the feedback! Just for clarification, I don't plan on going back to what I was doing before - the third option would mean basically having two libraries, HedgeLib++ and HedgeLib#, with the latter being pretty much a direct port of HedgeLib++ except with "simulated" direct memory loading using new C# features like Span rather than actual direct memory loading (so it'd be faster than old HedgeLib but not quite as HedgeLib++).

You make a good point about cross-platform support, though. If people don't mind the cons with the C++/CLI route I mentioned above then that route would work well

KenzieMac130 commented 5 years ago

I'm not quite sure why such a binding is necessary and that you might be accidentally introducing unnecessary complexity for the sake of solving a theoretical problem (one that hasn't been truly encountered in development) which may be slowing down progress and taking up development time where it could be better utilized. But if you still think a C# binding is a necessary goal, I'd go with writing a lightweight public facing C-Style API of some of the most sensible things to expose (editor debug drawing, file serialization, etc) "encapsulating" most of the behind the scenes stuff the end user wouldn't need to see or interact with.

Radfordhound commented 5 years ago

Well it's certainly not a requirement, it's just something I wanted to do so people could continue to use HedgeLib from a C# environment like they were able to before this C++ rewrite, as not everyone's comfortable with shifting over to C++ and some can't due to their tools, etc. already being written in C#.

This would simply cover HedgeLib, not HedgeEdit or the HedgeTools (those will still be purely C++), so there'd be no need for callbacks for drawing and such, just file serialization.

You're probably right with me overthinking things, though - I just assumed people wanted a way to access the rewrite from C# like they were doing before with old HedgeLib, but if I'm wrong and no one actually cares if there's a way to use it from a C# environment or not than this whole issue is just completely unnecessary.

I realize when I say it out loud that this all probably sounds a little silly. Again, maybe I'm overthinking things.

I guess the question should first be: Is this really a thing people want? Or am I over-engineering/misjudging things?

I'd really like to hear what people who used HedgeLib before I started this C++ rewrite think since those are the people mostly impacted by this decision.

KenzieMac130 commented 5 years ago

I don't personally use the library but I've watched some of the development Livestreams so that's where I'm coming from full disclosure. I think if someone has a use for Hedgelib++ they will probably be fine interacting with it and extending it in the language it's written in (C# and C++ are in the same family of languages so it might not be too difficult for people to adapt). Although if you do create C wrappers for stuff whenever you have the opportunity that will open up the possibilities for users to more easily generate bindings using to their language of choice should they actually want/need to. (that's atleast what I've seen be the most successful approach from other C/C++ libraries). Just don't stress about it too much and weigh down the development schedule.

KenzieMac130 commented 5 years ago

Have you considered or tried using SWIG for creating bindings yet? It seems promising. http://www.swig.org

blueskythlikesclouds commented 5 years ago

I would like to look at this realistically. Who really is going to use HedgeLib##?

Plus, the API and features are still in development and they are bound to change. I would worry about making C# bindings when you finally get to make a stable release, not when you are still factoring and implementing things.

Radfordhound commented 5 years ago

@astrand130 Thanks again for the input. I haven't tried SWIG actually, but it's my understanding that one of the reasons CppSharp was created was actually to get around limitations with SWIG. It does look good though!

@blueskythlikesclouds I was only concerned with C# bindings because I thought others actually wanted them/would rather use them than C++, especially given some comments I've received over time, but judging from the replies already it seems I was completely wrong. I really thought people would actually be upset if I made no bindings haha.

Radfordhound commented 5 years ago

Alright, so knowing you all anyway, several people telling me an idea of mine is bad on GitHub (where I don't even usually get a lot of interaction) in the span of a day is a good sign that my idea is really bad lol.

Being honest, I was only concerned with C# bindings because I said I would make them in the HedgeLib++ announcement video as I genuinely thought people would be upset without them/wouldn't use this just because it wasn't available in C#, but judging from the replies (so far anyway), it seems I was wrong.

With that said, you all are right. I'm overthinking things. Sorry. I'll close this here. If you're someone who really wants C# bindings feel free to reply.