Plugin system to allow 3rd party maintainership of large or specialist mods

[pinkestt]

I've been using Kanan for a while and I'd love to start developing some mods for it. However, I've noticed from looking through the source that the current mods are either patches loaded from a JSON file (not very powerful) or are compiled in (not very extensible).

I'm wondering if you would be interested in a system where external mods can be loaded from DLLs rather than needing to be compiled in. This would allow people to develop mods without needing to have them merged into upstream, which could be better for some more specialist or larger mods, as well as reducing the amount of recompilation required during development.

Looking at Mods.cpp, this doesn't seem like it would be a particularly significant change. I would be happy to submit a pull request that adds code to automatically load each DLL in a directory at the end of Mods::loadMods.

Let me know if this is wanted and I'll get hacking! ^_^

[Rietty]

There is already a feature in the Loader that loads/injects dlls that are listed in a specific text file. The one drawback is that you won't be able to really use the Kanan stuff like that.

On Mon, Nov 26, 2018, 11:40 AM pinkestt <notifications@github.com wrote:

I've been using Kanan for a while and I'd love to start developing some mods for it. However, I've noticed from looking through the source that the current mods are either patches loaded from a JSON file (not very powerful) or are compiled in (not very extensible).

I'm wondering if you would be interested in a system where external mods can be loaded from DLLs rather than needing to be compiled in. This would allow people to develop mods without needing to have them merged into upstream, which could be better for some more specialist or larger mods, as well as reducing the amount of recompilation required during development.

Looking at Mods.cpp, this doesn't seem like it would be a particularly significant change. I would be happy to submit a pull request that adds code to automatically load each DLL in a directory at the end of Mods::loadMods.

Let me know if this is wanted and I'll get hacking! ^_^

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cursey/kanan-new/issues/82, or mute the thread https://github.com/notifications/unsubscribe-auth/Ahm-VLa9YsBZ9Y2G15yRfA73168NSqNuks5uzBl5gaJpZM4YzZ6V .

[pinkestt]

Exactly. That's the motivation here: to allow using Kanan APIs without needing to compile your mods into Kanan itself.

[Rietty]

I mean you're welcome to do if you'd like to? It does sound neat. How do you plan on using the Kanan API with your external mods however?

On Mon, Nov 26, 2018 at 11:58 AM pinkestt notifications@github.com wrote:

Exactly. That's the motivation here: to allow using Kanan APIs without needing to compile your mods into Kanan itself.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cursey/kanan-new/issues/82#issuecomment-441714952, or mute the thread https://github.com/notifications/unsubscribe-auth/Ahm-VOhYLOPm23-Uvx_c-7N1ayRJPwJGks5uzB3KgaJpZM4YzZ6V .

[cursey]

I often see plugin systems like what your suggesting to be an example of over-engineering. Most users don't care about a plugin system they just want to download the mod and have it work. Most developers have no problem forking kanan and developing their own changes on top of it.

Personally I want to encourage upstream contributions and having a plugin system runs counter to that idea. Private mods don't interest me. People developing private mods with kanan is fine but it's not something I want to go out of my way to encourage. If you're interested in developing mods for kanan you shouldn't have any issue with doing it publicly where everyone can benefit.

Specifically, you say you want to start developing mods for Kanan. What about kanan is currently stopping you from doing that? A plugin system wouldn't make it any easier to get started than just forking kanan as it is.

[cursey]

I'm not totally against the idea I just need a better understanding of what the point is.

[pinkestt]

My main objective is less to avoid needing to fork Kanan, and more about modularity. I love how small and simple Kanan is and I'd really love it to stay that way, however I have a few ideas for mods that would: a) be significantly larger than the mods currently included with Kanan, possibly requiring multiple source files to keep tidy and b) not be particularly useful to many users.

Including this kind of mod upstream means that the Kanan maintainers would be responsible for keeping them updated and working, which is both not a priority (since they have a small userbase) and a large amount of work (since there's a lot of code). This means that the mods would fall into disrepair unless the original creator stepped in to update them. However, this maintainer would either need full access to the Kanan repository or need to go via pull requests for everything, once again creating unnecessary work for the Kanan maintainers.

With a plugin system, Kanan could become less a modpack and more a mod system for Mabinogi, providing a simple platform and set of APIs and allowing people to develop their own mods on top of it.

An example of a very successful mod system that does this is Minecraft Forge.

[cursey]

I won't turn down a PR for a plugin system if it doesn't complicate things. How were you planning on implementing it?

[pinkestt]

Here's how it'd work in C++-style pseudocode:

// Inside Mods::loadMods, or a method called from there
for (string& filename : listDirectory(PLUGIN_DIRECTORY)) {
    auto modLib = loadDLL(filename);
    auto mod = getSymbolPointer(modLib, "_kananModExport");
    addMod(mod);
}

Inside a header that's exposed to plugins, there would be a macro for creating the _kananModExport variable, which would look something like:

#define KANAN_MOD(modClass) std::unique_pointer<modClass> _kananModExport = std::make_unique<modClass>();

So in your plugin, you'd do something like this:

#include <Kanan.h>

class MyMod : public Mod {
    // ...
}

KANAN_MOD(MyMod)

[pinkestt]

If all goes well, it shouldn't require any changes other than adding a few lines in Mods::loadMods and possibly moving some header files around to make them easier to access from plugins.

[pinkestt]

I had a play around with loading DLLs using LoadLibrary and GetProcAddress from the Windows API, and it seems to work well. I'll try integrating it with Kanan later on.

[LoneDev6]

I've been using Kanan for a while and I'd love to start developing some mods for it. However, I've noticed from looking through the source that the current mods are either patches loaded from a JSON file (not very powerful) or are compiled in (not very extensible).

I'm wondering if you would be interested in a system where external mods can be loaded from DLLs rather than needing to be compiled in. This would allow people to develop mods without needing to have them merged into upstream, which could be better for some more specialist or larger mods, as well as reducing the amount of recompilation required during development.

Looking at Mods.cpp, this doesn't seem like it would be a particularly significant change. I would be happy to submit a pull request that adds code to automatically load each DLL in a directory at the end of Mods::loadMods.

Let me know if this is wanted and I'll get hacking! ^_^

It's a really cool idea, totally support this

[cursey]

How are you planning on exposing kanan's API to the plugins? You have to be careful with sending data across dll boundaries. It's probably enough to just dllexport everything in kanan core, dllexport the important bits in kanan, and see if there is an option to dllexport imgui. Then each plugin could just link against kanan.lib and maybe it'd all work as long as everything was compiled with the same compiler version.

[pinkestt]

Yes, that will likely be the biggest challenge. I'll need to refresh my memory on C++'s calling conventions, as well as read up on how Windows handles DLLs since I'm more used to Linux.

Hopefully, it'd simply be a matter of linking Kanan into the plugins as a library, but as you say, the compiler will probably play a big role in whether that works or not.

[pinkestt]

~From a quick look at StackOverflow, it looks like the biggest problem will be the fact that C++ doesn't specify the structure of classes. One solution to this could be to expose Kanan's API through a C interface, which is more standardized.~

~To make the API usage from plugins more similar to the internal API, some header-only wrapper classes could be created on the other end.~

My mistake - it's standard library classes that don't have a defined ABI, not classes in general. This means that my original design of having the plugin define a std::unique_ptr won't work, but other than that everything should work fine.

[cursey]

I would rather not expose a C API for kanan if at all possible. You can export C++ class methods, and like I said as long as the compiler matches it should work without much fuss (as long as care is taken passing data across dll boundaries). I don't particularly care about supporting other compilers. On Windows it's a decent expectation that a developer will be able to use MSVC and kanan only comes with build files for Visual Studio.

[pinkestt]

Sounds reasonable. Personally I use MinGW as I'm more familiar with GCC, but like you say, provided the same compiler is used to compile both parts it shouldn't matter.

[cursey]

Well, I just think it'd be a lot less work than creating a C API.