Support native plugins

[thejoshwolfe]

I'm thinking something like:

var native_library = mf.include_native("libhax.so");

or even:

mf.include("libhax.so");

It's a wild idea, I know. Just thought I'd start a discussion on it. To start, there's an obvious portability issue with the above examples.

[thejoshwolfe]

Or, another idea is for a directory to contain plugins (as .so or .dll files), and they're all loaded if possible.

Possible mineflayer_plugin.h interface:

#include <QScriptEngine>

// called before the main .js file has been run
void mineflayer_plugin_preload(QScriptContext *context, QScriptEngine *engine);

This would allow plugins to setup a presence in the global context for .js file initializers to check for and respond to (like included .js files). A native plugin's real interface would be a javascript interface. With this idea, any access to the mineflayer core would need to go through the mf object.

[thejoshwolfe]

Another idea is to provide a Game object to plugins.

#include "Game.h"
#include <QScriptEngine>

// called before the main .js file has been run
void mineflayer_plugin_preload(QScriptContext *context, QScriptEngine *engine, Game *game);

That way, any Game-level optimization would be available to native plugins. This would allow a plugin to build block location caches to speed up "find" like we talked about a long time ago.

[thejoshwolfe]

Here's some demo C and C++ code that shows that the Game* idea should be possible. http://wolfesoftware.com/shared_object_demo.tar.bz2

[thejoshwolfe]

I'll note that the above mentioned strategy is Linux-only, and there isn't even an equivalent in windows (according to here).

[thejoshwolfe]

branch 'path' has not even a hint of windows support. I don't know what to do. The plugin framework is so promising, yet so awesome-dependent (read "linux-dependent").

[thejoshwolfe]

According to the link above, we can get plugins to work properly in windows, but we need to make the entire program a dll (with a minimal bootstrapping exe). Potential problems with this:

• a mineflayer binary is no longer a single file (was it anyway?)
• how to find the .dll/.so?
  • in development, we can use a path relative to argv[0]
  • in deployment, the .so should be installed; the .dll should resolve relative to the exe automatically
• subtle drawbacks of using a .so.
  • i think just performance due to PIC. whatever.
• subtle drawbacks of using a .dll.
  • ???
• development is more complicated. I think we'll need another .pro for Qt to build the bootstrapping exe.

Note that this does not entail dumbing everything down to C. We can still rely on all sorts of 1337 C++ across library boundaries as long as everyone's built with the same compiler.

[andrewrk]

http://doc.trolltech.com/4.7/qlibrary.html#details

this should solve your cross-platform problem.

[thejoshwolfe]

Well, alright. QLibrary does solve the dlopen vs LoadLibrary problem, but it doesn't solve the issue of plugins being able to access host code, such as calling functions of the Game object. I don't think Qt can add that functionality to Windows.

I read from http://doc.trolltech.com/4.7/plugins-howto.html that Qt's plugin solution forces plugins to access their host code through pure virtual classes (like java interfaces). This is effectively solution 1 from the link above. This is not as simple a solution as the linux way because we'll need to duplicate any Game declarations in an IGame interface for plugins to be able to use them. The benefit of this is that it provides a nice layer of encapsulation in case we wanted to hide some core stuff from plugins.

[thejoshwolfe]

My vote is Solution 2 below

Requirements

1. Plugins must be in separate files (duh) like a .dll or .so. Static linking is not acceptable.
2. Plugins must be able to interact with the javascript environment.
3. Plugins must be able to call Game::blockAt, Game::getMapData, etc. without going through javascript.

Solution 1

Link the executable with -rdynmaic/-export-dynamic. This allows plugins to access all the symbols of the host. Load the library using QLibrary and call a single special function by name (such as "mineflayer_plugin_init"), and have the rest of the plugin accessible through the object returned by the function. This is currently working (except for the QLibrary part) in branch 'path' (at least commit 4d861ee5f8e0eddc724eab631d3202bebe52c625).

drawbacks

-rdynmaic/-export-dynamic doesn't work in Windows.

Solution 2

Make mineflayer a library with a minimal bootstrapping exe. Link plugins dynamically against the library (load-time linking) so that plugins can call code in the mineflayer library. Plugin loading and initialization would be the same as in solution 1.

drawbacks

Ugly organization.

Solution 3

Use QPluginLoader to load plugins. Expose the interface of the Game class to plugins as purely virtual classes. Give plugins the object that implements the interface so that the plugin can call the functions of the Game class.

drawbacks

Bulky interface means expensive maintenance.

[Darthfett]

I don't understand much/any of this, but I vote for cross-platform support. Also: the fact that mineflayer would be a library and/or support plugins is nice, and might bring in more developers/users.

[thejoshwolfe]

Well, mineflayer being a library is a tricky issue. I wasn't thinking of making it a proper library in the sense of it having a public api and being accessible from another serious application. I was planning for the minimum change necessary to accomplish our goals, which would effectively (perhaps literally) be the library having an int main() and taking control of the main thread just like mineflayer does now. It being a library would just be a technicality to accommodate Windows' lack of an -rdynmaic/-export-dynamic equivalent.

[Darthfett]

I just figured I would offer my opinion on plugins and libraries. I guessed that was what you meant when you called it minimal. As far as solution 2 vs solution 3, I don't know enough about either to offer my opinion.

[andrewrk]

Done! We get this for free with node.js.