ability to use third party libraries with behaviours

[blurymind]

We now have this fantastic ability to create behaviours in the event sheet. We also have the ability to write js code- straight in the event sheet.

But what use is writing js code for your game, when you cannot import any third party js libraries? This is literally the biggest weakness in our api atm imo. Compared to this: https://www.construct.net/en/make-games/addons/1/javascript

Including third party libraries to extend a GD game is very difficult.

So the request here is simple- make it easy. As easy as typing import MyClass from "myscriptsFolder/MyClass" in the monaco editor. Or as easy as an event sheet action.

WIthout third party libraries, our extension ecosystem is not going to get any of the cool things happening in opensource html5 gamedev world.

That is kind of where construct3's js is still ahead of gd. Most of their extensions are based on some library that the dev simply exposed to construct: https://www.construct.net/en/make-games/addons/plugins

[Bouh]

In gd4 I've copy paste socket.io inside js events , why I can't do same in GD5 ? If socket.io is global (window.socket) I should have the possibility to get in all JS events

Le sam. 1 juin 2019 18:15, Todor Imreorov notifications@github.com a écrit :

We now have this fantastic ability to create behaviours in the event sheet. We also have the ability to write js code- straight in the event sheet.

But what use is writing js code for your game, when you cannot import any third party js libraries? This is literally the biggest weakness in our api atm imo. Compared to this: https://www.construct.net/en/make-games/addons/1/javascript

Including third party libraries to extend a GD game is very difficult.

So the request here is simple- make it easy. As easy as typing import MyClass from "myscriptsFolder/MyClass" in the monaco editor.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/4ian/GDevelop/issues/1077?email_source=notifications&email_token=AAMX4DUJBRAZ63RSVTYGPN3PYKOCBA5CNFSM4HSA2GOKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GXDICMA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMX4DTHHLUWRVGSQJEBVWLPYKOCBANCNFSM4HSA2GOA .

[4ian]

So while I agree that adding support for "raw" JavaScript file is interesting, there is something that I want to clear:

That is kind of where construct3's js is still ahead of gd. Most of their extensions are based on some library that the dev simply exposed to construct

All of these plugins require JavaScript to be written, aka, they are the same as GDevelop extensions right? And of course it's possible to add any arbitrary JS library/file to a GDevelop extension, that's how Physics or Tween extensions are working 😊 See:

https://github.com/4ian/GDevelop/blob/master/Extensions/TweenBehavior/JsExtension.js#L56

So I think that you're slighlty confusing the fact that GDevelop allow you to use JavaScript (which is not yet the case for C3 - afaik it's being added right now) and the fact that C3 have plugins with third party libraries? Or is there something that I am missing?

Most of their extensions are based on some library that the dev simply exposed to construct

So tell me again if I'm mistaken, but it's perfectly possible to re-develop all the extensions, that are on the page you linked, for GDevelop. Or did I miss something and it's somehow easier for C3?

I know that this is not the main point of the topic, but I want to get our facts straight 😊It's important because there are so much important things to do on GDevelop that I don't want to develop things "because C3 is ahead of GD" to discover later than in fact GD is on the same point (and even more advanced as allowing JavaScript in your events).

To get back to what you're asking, is the solution of Bouh potentially usable? Best way to have me work on something for GD is to actually show me a proof of concept using for example a library embedded in a JS event that works. At which point I will have a valid, proven use case to allow arbitrary JS files to be included - which will need rework of the exporters and make things like minification more complex.

(Again, I understand that this would be an interesting feature, it's just that I don't want arguments that are biased because it's a dangerous way to prioritise things)

[4ian]

Said in other words: if tons of JS plugins exist in Construct to re-use JS libraries, that they are done by written plugins in JS and knowing that we can do the same in GDevelop, then the question is:

• Why don't we have people doing the same for GDevelop?

Question for which there are multiple hypothesis:

• 1) Writing an extension for GDevelop is work difficult?
• 2) We have less developers in the community?
• 3) Or the fact that we can add JS libraries like I shown (https://github.com/4ian/GDevelop/blob/master/Extensions/TweenBehavior/JsExtension.js#L56) is not known and so you think it was not possible?

If it's 1, we can surely make it easier to write extensions - or see why it's harder and how to improve that. If it's 3, we can enhance the doc. If it's 2, GDevelop is growing so hopefully we'll get more people contributing :)

I know that I am a pain, but I really want us to explore all available options before risking to bloat the software. Simplicity is something that is hard to achieve, and supporting things like "import MyClass from "myscriptsFolder/MyClass"" is actually a very complex feature involving JS bundlers, syntax transformers and all kind of things that will make me spend 2 months of my time doing this - while maybe a line in the doc or a way to include files to Extensions made in GD will be enough :) But for now I'm not sure what is the proper solution, that's why I'm asking all the questions 😉

[4ian]

Turns out there is already a card on the roadmap: https://trello.com/c/jnT2J2Td/230-add-support-for-embedding-arbitrary-js-files-in-the-game

I'm linking the discussion here.

[blurymind]

There is a card yes. Sorry I totally forgot this was requested in the past. It will really help with prototyping extensions if we could include js files directly in the event sheet, yes.

The motivation behind this is to make development of extensions in GD more and more enjoyable and encourage the development of extensions. From my point of view the ability to import js libraries is very important for creating extensions. Almost everything I am interested in making an extension of depends on it

[4ian]

Fair enough, for now I have no better solution than asking you to put the whole file in a JS event but I see the idea. Could be part of extensions that you can create in GD (so basically, that would call addIncludeFile, like in extensions).

[blurymind]

If I create 2 js code events, one after the other, can I call functions declared in event A inside event B? I will give this a try with a simple js library. Pasting it's minified contents in an event and trying to call a method from it in event B

[4ian]

Not out of the box because declarations in JavaScript are scoped to the function where they are running (in other words, what you declare in event A stays in event A). The solution is to:

• Attach what you want to share to an object that will be shared between event A and B functions. In GDevelop, this is for example runtimeScene. Be super careful and name your variable with a very explicit name like "runtimeScene._extensionXXXMyStuffThatIsShared" to be sure that it won't collide and destroy something (method or variable) that belongs to runtimeScene.
• OR Attach what you want to share to a global. For example gdjs. Same safety rule applies
• OR Attach what you want to share to window variable, which in JS make the variable global. i.e: window.MyLibrary = function() { /*...*/} means that I can then do MyLibrary() anywhere in the codebase.

Historically, JS libraries used the third approach, exposing themselves in a global variable. Things have improved with the introduction of JavaScript bundlers, allowing you to require files and bundle everything inside a single JS file that you import in your index.html (this is what is done for GDevelop IDE). This requires libraries to "export themselves" using something called AMD or CommonJS. See https://medium.com/computed-comparisons/commonjs-vs-amd-vs-requirejs-vs-es6-modules-2e814b114a0b for what seems a good explanation.

In practice, most libraries are still doing a dual approach of both exposing themselves to be used in a CommonJS environement (i.e: "require") AND can also expose themselves as a global on the window object (this is called UMD: universal module definition. If the library says it support UMD, then it can be used as a global in the browser. For example, shifty.js is UMD)

The game engine and GD events/JS code is still using the "good old" approach of having a bunch of files without bundlers or anything - notably because this is simple (for preview it's especially important to be fast) and avoid bundlers adding weird stuff or applying transforms to the JS that would be unwanted. This means that everything is located under gdjs object. Any extension using a third party library must include the file containing the library (using addIncludeFile) and be sure that the library is exposing itself as a global.

So at the end, what does all of this has to do with using libraries in a JS code event? If the library is exposing itself on window, you should be able to:

1) paste it in a JS code event 2) be sure that this JS code event is only executed once when the game launch/scene is started. 3) be sure that the library is available on window (depends on the library, but usually it's explained in the README) 4) if it's not exposed on the library, you can still console.log it in the JS code event (after you pasted the library minified code) and expose it to window:

// Code of the library

console.log(MyLibrary); // Will crash if it's not here
window.MyLibrary = MyLibrary; // MyLibrary is now available anywhere
// You can also expose it on GDJS, but be careful:
gdjs.ExtensionXXXMyLibrary = MyLibrary.

5) Enjoy :)

Let me know if you can get something working this way.

[blurymind]

Thank you @4ian :) I will give this a try tonight

[blurymind]

It works for something simple I wrote. For bondage.js I get this error:

Uncaught TypeError: Cannot read property '_babelPolyfill' of undefined
    at Object.<anonymous> (code0.js:94)
    at Object.<anonymous> (code0.js:94)
    at t (code0.js:94)
    at Object.<anonymous> (code0.js:94)
    at t (code0.js:94)
    at code0.js:94
    at Object.gdjs.New_32sceneCode.userFunc0x8254e8 (code0.js:94)
    at Object.gdjs.New_32sceneCode.eventsList0x825424 (code0.js:111)
    at Object.gdjs.New_32sceneCode.eventsList0xb22f0 (code0.js:427)
    at gdjs.RuntimeScene.gdjs.New_32sceneCode.func [as _eventsFunction] (code0.js:1089)
websocket-debugger-client.js:21 WebSocket connection to 'ws://127.0.0.1:3030/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

https://raw.githubusercontent.com/jhayley/bondage.js/master/dist/bondage.min.js

This is a very old minified compile of https://github.com/jhayley/bondage.js

I assume this is related to babel, which is used in ALOT of libraries, to compileto a minified file for including in other projects

[4ian]

Do you have this issue when simply running the preview without using bondage at all or do you have some code calling bondage? Can you paste it here if so?

[blurymind]

when calling bondage. I find that in order to execute nested functions, you have to do this

window.myFunc = function(evt="err"){
    console.log(evt)
    window.myFunc.sub = function(){
        console.log("subfunc")
    }
}

when usually the case for libraries you import is:

window.myLibrary = function(evt="err"){
    sub = function(){
        console.log("subfunc")
    }
}

You can try with the js version of http://depts.washington.edu/madlab/proj/dollar/pdollarplus.html which is not minified. Try executing a function or logging it to console

[blurymind]

Actually I get it with a black screen, simply by declaring bondage as

window.bodagejs=function(){
 ...minified contents...
} 

no need to call it

[4ian]

What if you paste directly the minified content without adding the window.bodagejs=function(){, this seems strange to me? This seems strange to me to do this, you wont get bondage in it, just a function executing a bunch of stuff.

Can you share the project with me? Will be easier to debug.

[blurymind]

black screen. I will share a project tonight :)

Minified libraries seem to be all transpiled to nested functions emulating classes with methods. So you have something like this:

myClass = function(){
  aMethod=function(){
    ..stuff happens
  }
  anotherMethod=function(){
    ..stuff happens
  }
} 

My Observation is that GD's monaco code can't handle that at all. As noted previously, you can't execute a nested function, unless it has been declared like this:

myClass = function(){
  myClass.aMethod=function(){
    ..stuff happens
  }
} 

This kind of makes it impossible to import and use the code - even if you copy and paste it. Nested functions don't work as they are supposed to in minified js

[4ian]

No it's normal, classes don't exist in JavaScript and are transpiled to functions. GD code editor has nothing to do with execution of JS itself.

Most libraries are wrapped into a self executing function that is called immediately after being defined. This is to avoid leaking variables into the outside. This is common for UMD/libraries running in browsers

[blurymind]

I know, my point was that when I wrote a tiny test in monaco as

myClass = function(){
  mySubfunc=function(){
    ..stuff happens
  }
} 

I couldn't execute the subfunction. This suggests that declaring a minified library in monaco by pasting it like this

window.bondagejs = function(){ ...pasted minified contents here }

will still have the problem of not being able to access any of the pasted contents

[4ian]

Right, but we must not do this, right? You must paste the minified content and that's all. Re-read https://github.com/4ian/GDevelop/issues/1077#issuecomment-498176041 => I never said to put the library in a function right? Or am I missing something?

[4ian]

I'll give it a try as soon as I can, that will be easier :) Still give it a try as I explained:

• in a js event, paste the lib
• do it only at the beginning of the scene
• add a console.log at the end, depends on the library here you actually have to read the Readme and see what's the name it is exposed with. Most libs are registering themselves as a global on window, meaning that you can access them from anywhere. Otherwise, you'll have to attach it to runtimeScene or window manually (as I explained in the message I linked ;)). This is more an issue of the library usage than GDevelop there.

[Bouh]

• in a js event, paste the lib
• do it only at the beginning of the scene

This return a error : Uncaught TypeError: Cannot read property '_babelPolyfill' of undefined

I have a great deal of trouble too, adding libraries. But I have found this way to do why it works I don't know, and knowing why the other way doesn't work is just as important.

@blurymind You need start this JS code once at beginning of the scene Dont keep my link (witly.fr/GD11/bondage.min.js) i will delete it in few days

window.loadscript = function(url){
    var script = document.createElement("script");  // create a script DOM node
    script.src = url;  // set its src to the provided URL

    document.head.appendChild(script);  // add it to the end of the head section of the page (could change 'head' to 'body' to add it to the end of the body section instead)
};

window.bondage = loadscript('http://witly.fr/GD11/bondage.min.js');

[4ian]

I did a try with Shifty.js and it works. In the case of bondage, can you give me an example game?

[blurymind]

@Bouh thank you! :) window.bondage = loadscript('http://witly.fr/GD11/bondage.min.js'); wouldn't that require internet connection to play the game? I want the script to be distributed with the game and working out of the box- with or without internet

[blurymind]

I am attaching a test project- its just a copy of the js example: bondageJSinGD.zip

@Bouh 's method works, @4ian 's method results in black screen- so I commented it. I would prefer @4ian 's method to work though as it doesnt require internet connection :)

It' strange as monaco displays some of the regular expressions in the minified script as commenting out of code, yet if you load it with @Bouh 's approach it works regardless. Bouh's method does not work when I try loading it with github's host raw file url - so it also requires for the minified scripts to be uploaded to a different host too.

[4ian]

Found the issue in my "solution". GDevelop wraps all the JS code in a function using "use strict":

function(runtimeScene, objects) {
  "use strict";
  // .. Your code here ..
}

This allow to use JS strict mode, which prevents basically "bad" things like using undeclared variables (that would be implictely global) and other things that make JS stricter, which also means much much safer: https://www.w3schools.com/js/js_strict.asp

The good news is that GD does this for you to protect you against old JavaScript caveats, that gave a bad reputation to this language when it was first created. The bad news is that bondage.js is not compatible with this an so the error with e. _babelPolyfill (e should be window, but is undefined in strict mode).

With @Bouh solution, you're loading a script from another website, bypassing strict mode, so it kind of works (I say kind of, because it's not "officially" supported and you can get some hairy problems with CORS) (though it's a good start to experiment - but it's only a temporary solution).

Strict mode was added in GDevelop, but turns out that you can disable it in the JSON of the game:

BUT turns out that then the script fails to be evaluated, most probably because it's not loaded as UTF8 and so contains invalid character:

So I'll have to dig more, probably loading all scripts with charset="utf-8" in the index.html... for now, experiment with Bouh method of loading scripts (but don't expect it to work to well in an exported game).

Also, side note: isn't there any alternative/similar libraries to Bondage.js? Not to criticise anything, just that I don't know this field and how many libs are doing this kind of work.

It' strange as monaco displays some of the regular expressions in the minified script as commenting out of code

Don't pay too much attention to syntax highlighting of minified scripts. As they are minified, all whitespace and line breaks are removed, so that everything is more or less on a single line. Editors have a hard time parsing this properly and at some point, when the line is too long, they give up and stop syntax highlighting, so that you have the rest of the code that is in the color they were using before giving up. (this is actually a good thing, as this is protecting the editor (and so GD) to blow up because of a memory issue).

[Bouh]

When you set the url with github you can't use because MIME type of file is "text/plain" and with this type chromium inside electron cannot read the script.

[blurymind]

@4ian is there anything that can be done to make it's minified script more compatible? The author of bondage made a new web build just a few minutes ago: https://github.com/jhayley/bondage.js/pull/48 https://github.com/jhayley/bondage.js/blob/ec1b2d0957a91b6c26a8938d242edb33e702537f/dist/bondage.min.js

Bondage is using gulp to generate the files https://github.com/jhayley/bondage.js/blob/ec1b2d0957a91b6c26a8938d242edb33e702537f/gulpfile.js

is there some option that we can pass or avoid passing to make it more compatible with GD?

[Bouh]

It's possible to wrap the script as text and inject this text in Githubissues.

Githubissues is a development platform for aggregating issues.