Add Plugins System

[julien-moreau]

The plugins system must allow to develop external plugins. A plugin must contain at least:

• package.json (that points the main file to require)
• the according JS files
• dependencies
  • electron-rebuild (same version of the editor for native modules)
  • babylonjs-editor (for typings)

A plugin should return an object that can:

• Add a menu to the toolbar
• Register inspectors
• ???

[jkeys-ecg-nmsu]

@julien-moreau we are looking forward to this immensely, also to re-iterate blueprint.js is fine in terms of component libraries.

[julien-moreau]

Hey! Just released the 4.0.0-beta.1. You can download the prebuilt here: https://github.com/BabylonJS/Editor/wiki/BabylonJS-Editor-v4.0.0-beta

You can of course build your own using this repository.

For your plugin, you can start from the sample plugin which is available here (just copy/paste in your own directory): https://github.com/BabylonJS/Editor/tree/release/4.0.0/sample-plugin

For the "babylonjs-editor" module version, don't forget to change from:

"devDependencies": {
    "babylonjs-editor": "file:../module",
    "typescript": "3.7.5"
},

to:

"devDependencies": {
    "babylonjs-editor": "4.0.0-beta.1",
    "typescript": "3.7.5"
},

Then install the dependencies using npm :)

To debug your plugin, once the editor is running, you can type "CTRL+ALT+i". This will open the devtools of the editor and you'll be able debug your plugin, see errors etc.

Important: the editor doesn't provide hot reload for plugins at the moment. That means you should refresh the editor (using F5 in the devtools focused) so the plugin will be reloaded with the editor.

To add a plugin, simply open the editor's preferences in the main toolbar "Edit -> Preferences" and add your plugin using the "Plugins" section. The editor will ask you a folder, just add the one that contains the package.json in your plugin's directory.

And that's all!

Let me know if it works for you :)

Thanks!

[julien-moreau]

Also, the new branch containing the V4 is « release/4.0.0 »

[jkeys-ecg-nmsu]

Fantastic, I'll port my code over to implement the new IPlugin registration function.

I suppose our contribution will be the first plugin for the new plugin platform -- what are your thoughts on distribution?

Would another section on this guide be helpful? https://doc.babylonjs.com/how_to/load_from_any_file_type Specifically I'm thinking of adding a section like "Load Scene Over Network" to show how content would be downloaded and appended to a scene with a URL. I'd also contribute some tips on ideas for layering scenes in order to achieve cool effects without having to redownload the whole bundle at runtime. Shout-out to @prestomation for the idea of layering scenes dynamically to cost-effectively achieve complex dynamic scenes.

(beginning of this conversation here: https://forum.babylonjs.com/t/hook-into-export-functionality-or-add-publish-functionality/11756/16)

[julien-moreau]

@jkeys-ecg-nmsu just fixed a bug and pushed to release/4.0.0, you should pull and work with that version of the editor :)

For the guide, I'm not sure to understand what you mean. In other words, not sure to understand the link between the SceneLoader and the plugin of the editor ^^

For the distribution of plugins, I haven't really though about it but everything is possible today:

• Add way to pack a plugin (zip for example) and give it directly to the editor ? So the editor uncompresses and adds it in an internal folder?
• simply reference all plugins (like today) on the HDD of the user?
• Create a plugins store ? (looks harder to do ahah)

I'm open to any other suggestion :)

[jkeys-ecg-nmsu]

Let me think more about your distribution ideas, creating a store does seem a bit ambitious. I'll think more about my documentation suggestion and get back to you on that also.

When I try to npm run build off of upstream/release/4.0.0 I'm getting a lot of TypeScript errors, is there anything special I need to do to setup my local environment to build the release/4.0.0 branch? It also looks like the install script was removed. It helps to npm install the correct dependencies before trying to build.

It seems like it's only building for Linux at the moment, how would I get a Windows build of the editor? Edit: I got it to build for Windows by running the build script from a Windows (Powershell) environment instead of Linux (WSL Ubuntu).

[julien-moreau]

I think too for the store ahah. Thanks, can't wait for feedbacks :)

Oh, you are right, the build will be related to the "OS" that is running the command. And electron detects Ubuntu On Windows as a linux one. On my side I use git bash or powershell to build the windows version. I can add an argument like "npm run build -- --os=win32" to force an OS

[julien-moreau]

@jkeys-ecg-nmsu any news? Is that working for you? :)

[jkeys-ecg-nmsu]

Hi @julien-moreau sorry for being so slow to respond.

I've got the code for the plugin mostly built out but I'm confused as to how to currently load a plugin -- whenever I try to load the sample plugin, the main toolbar is unaffected. Am I loading the plugins incorrectly?

Also whenever I add my plugin to the npm postinstall script (e.g. extend it to be electron-rebuild && electron-builder install-app-deps && cd sample-plugin && npm i && cd .. && cd publish-plugin && npm i && cd ..) the executable doesn't contain the plugin.

I got stuck at this point and had some other stuff to accomplish this sprint, hence the delay. Thanks again for all the help!

[jkeys-ecg-nmsu]

@julien-moreau please let me know what I'm doing wrong! I'd like to drive this on through to completion.

[julien-moreau]

Hey @jkeys-ecg-nmsu so sorry for the delay! I Was not available for holiday purposes :)

The plugins are not packed with the editor, they are dynamically registered/loaded at runtime (when the editor runs). No documentation exists so I apologize. To add a plugin (previously built), just:

• Go to the Editor's preferences and see the "Plugins" section.
• Click "Add" to a new plugin to the list
• Select the folder that contains the plugin (I mean the root folder that contains the package.json file). The package.json file must contain the "main" key that points to the main javascript file.
• Click "Apply" on the bottom-left of the preferences window.

The plugin is now loaded! The remove a plugin, go to the same plugin's section in preferences and click "Remove" then "Apply". The plugin is now unregistered.

Note: when you remove a plugin, the Javascript source is still available in the VM of Electron. That means if you re-add the same plugin, the sources of the plugin will be reloaded but will be taken from the "require" cache of Electron/node.js.

Here is a little Gif that shows how to enable the sample plugin in the Editor:

[jkeys-ecg-nmsu]

@julien-moreau thanks for the response! I forgot to compile the typescript for the plugin D: thanks for the gif, the "build" and "declaration" folders existing is the kick I needed. I'm now able to load the sample plugin. Thanks again for all the help.

[jkeys-ecg-nmsu]

Do you have a recommended way of clearing the electron require cache? It's hard to tell if my iterations on the plugin are being accepted.

[julien-moreau]

@jkeys-ecg-nmsu right, I mentionned that because yes, we have to find a way to clear the cache properly :) I'm investigating the "require.main.cache" property right now and I think I can clear the cache of ALL plugin's loaded files, filtering by the absolute path of Javascript files. Before doing that I prefer to be sure it also clears the compiled module in node.js. Else we'll have huge memory issues.

Once you added the plugin, it is saved directly in the editor's settings. What you can do at the moment to clear the cache:

• type "CTRL+ALT+i" in the editor. This will open the devtools.
• Focus the devtools with your mouse and simply type "F5" to reload the Editor (refreshes the page like any other web browser)
• The editor restarts by loading the workspace you previously loaded.
• Once reloaded, the plugin is required. And because we reloaded the webpage, the Node.js cache has been fully cleared :)

Is it helping you to speed up the development ?

[jkeys-ecg-nmsu]

Awesome! I'll start using this workaround.

I definitely like working with React and development is fairly quick; the problem is that there are some intricacies around Blueprint/JS like how to launch a Dialog. Actually, on that: do you think it makes sense to have a

component rendered by the toolbar's MenuItem? I'm trying to figure out how to popup a "modal" to allow the users to enter their workspace preferences for just this plugin.

In terms of my team's artists' workflow, this will be a huge help once we've got it stable! For now we're just exporting our babylon scenes (as GLB) and manually uploading it to S3 / invalidating the cache.

[jkeys-ecg-nmsu]

I was able to get the Dialog to work, it seems that sometimes the Menu closes itself, which unmounts the nested Dialog but for the most part it seems to be working. The only difference is that with my plugin, there will be no MenuItem -- the toolbar button itself launches the Dialog.

Thanks again for all the help!!

[jkeys-ecg-nmsu]

@julien-moreau is there an easy way of getting the workspace directory via code? I'm looking at various solutions with relative pathing and __dirname but I'm not sure if you have a preferred solution (trying to grok the workspace code to find it).

I want to store the plugin preferences as a json blob in that directory.

[julien-moreau]

Hey, I'm back from holidays and I'm now 100% focused :) The toolbar's menu item(s) are mandatory today as this is the only way to draw a plugin but we can discuss about what a toolbar item can be.

For example, maybe you would like it to be a simple button ? For the modal, I can add an helper like:

import { Modal } from "babylonjs-editor";

Modal.Show(
    <MyReactStuff ... />
);

Are you interested in that kind of helper ?

[jkeys-ecg-nmsu]

I was actually able to simulate Modal.Show with the Blueprint Dialog component (which renders into a newly created element on the DOM that parents to <body>). Here is how it looks now:

It probably would be nice to have Modal.Show (something like the equivalent of CreateWindow from previous versions of the editor), but as you can see I was able to get past the modal problem for the time being so it's not critical.

The only remaining problem is saving and loading the plugin preferences but I think I was reading the documentation incorrectly -- I was treating getWorkspacePreferences and setWorkspacePreferences the opposite of how they are intended. So I'm trying to fix it with that insight, if I still need help I'll drop you a note though.

[julien-moreau]

Excellent !! Really nice !! 💯 I created a simple plugin that supports Quixel Bridge.

To export the settings: https://github.com/julien-moreau/babylonjs-editor-quixel-plugin/blob/master/src/index.tsx#L40 And the function that exports the settings to a JSON representation: https://github.com/julien-moreau/babylonjs-editor-quixel-plugin/blob/master/src/quixel/preferences.ts#L23

[jkeys-ecg-nmsu]

I was almost there! :D That's perfect, it shows that my understanding of the two functions was backwards. Thanks for all the help.

Here's how I'm simulating the modal:

https://github.com/electronic-caregiver/Editor/blob/feature/publish-plugin/publish-plugin/src/dialog.tsx

Also, I didn't put it into a branch on its own for a pull request b/c I wanted to run it by you, but what do you think about installing React Devtools into the main process?

https://github.com/electronic-caregiver/Editor/blob/feature/publish-plugin/src/main/main.ts#L12-L18

[julien-moreau]

Yeaaaaah we did almost the same ! That rocks !

That's an EXCELLENT idea ! We can think about a way to specify that "I want the react dev tools please" so the Editor will not try to install the devtools each time when in production mode. Like a section in the Editor's preferences window for developers. So they can toggle "enable react dev tools" in their context.

[jkeys-ecg-nmsu]

Everything works! I split it off into its own public repository. There are some TODOs like support for gltf and babylon formats (I was a little unclear on how this would work since they generate multiple files) but it re-establishes the pipeline for our artists. You're the man @julien-moreau!! https://github.com/electronic-caregiver/babylonjs-editor-publish-plugin

I'll work on a preference for devtools, then put the plugin install inside a conditional branching on that preference. Then submit a pull request to the main repository.

[julien-moreau]

Excellent !! You really rox!! I'm having a look to your plugin and I can see that Editor can provide you a lot of utilities to get less lines of code. Typically, for the .babylon file format, the Editor must be able to give you:

• the path to the .babylon file
• the path of ALL files associated to the .babylon file
• the path of ALL binary files in case of a binary export (not yet supported)

I'm creating a list of all utility functions and they'll be able in next releases :)

[jkeys-ecg-nmsu]

Those will be incredibly useful! Although I wonder if these are utilities that should be provided in the Babylon core, or babylonjs-loaders, rather than the Editor.

[jkeys-ecg-nmsu]

Eg change this contract:

const scene = await GLTF2Export.GLBAsync(this.props.editor.scene, name, {});

To this:

const { scene, exportedFilePaths } = await GLTF2Export.GLBAsync(this.props.editor.scene, name, {}); //do something with exported paths

[julien-moreau]

@jkeys-ecg-nmsu don't worry about the developer options in preferences, I just added them :) I think unfortunately that the GFLTF2Export class is maintained by the Babylon.JS Core team and has to work in all contexts (electron, web, etc.). And having a notion of "file path" would break it.

Anyway, if it was what you were talking about, I can provide helper functions that use the GLTF serializer and returns that kind of object you mentionned. Is that a solution working for you?

[jkeys-ecg-nmsu]

Yes, I think that'll work great. Thank you!

[julien-moreau]

@jkeys-ecg-nmsu, just released the beta.4 here: https://github.com/BabylonJS/Editor/wiki/BabylonJS-Editor-v4.0.0-beta :)

[jkeys-ecg-nmsu]

@julien-moreau we are not using the Editor as extensively as we previously thought, hence the lack of followup development on the plugin =) our main limitations are deficiencies in visual scripting. If the Editor's visual scripting system gets to the point where we need it to be, we will most likely resume planning around the Editor and I will continue active development of the plugin.

For reference here is the documentation for the visual scripting provided by our previous editor/engine. Particularly we need the animation nodes. I am trying to contribute the emote and Point of Interest nodes to your codebase (which rely on an open-source AWS Babylon-compatible feature library), but I find it intimidating to implement a tween node with easing functions and transforms. I'm hoping you can implement tweens and get us back on track! :D

[jkeys-ecg-nmsu]

That being said I will take a look at the helper functions and see about using them for exporting the multiple-file formats. Although I have come to learn that GLB can still have separate textures so I think my plugin needs to generally handle multiple file exporting.

[julien-moreau]

@jkeys-ecg-nmsu, I'm thinking about a way to register custom nodes, like we can register plugins.

For example, the node (emote) you shown me (for the visual scripts editor) looks enough specific to your environment to not be in the "official" list of nodes in the editor.

What I'd like (and I think you too) is to be able to create nodes and register them ONLY for your projects. Exactly like today you can create plugins and keep your plugins private.

Do you have any though about this?

[jkeys-ecg-nmsu]

@julien-moreau I like that; the emote node also wouldn't make sense without this dependency.

[julien-moreau]

@jkeys-ecg-nmsu just added a function in the ProjectExporter static class: ListExportedFiles When exporting to the .babylon format, you can call this async function to get the list of ALL absolute paths of files generated in the scene's output folder. That will be useful for your plugin :)

You can find the commit here: https://github.com/BabylonJS/Editor/commit/aecd6a5eef744f12cc6aad7c0e95dee3aac1c774

[julien-moreau]

@jkeys-ecg-nmsu If I understand well, using the window events (custom events on .window) nodes of the Graph Editor, you don't need to create emote nodes anymore?

[jkeys-ecg-nmsu]

Yes I think that's true, we can run the same logic behind the event listeners from the visual script generated code as we do behind the Hub listeners. Great point!

[julien-moreau]

Perfect! That will allow you continue the dev of your application until we get the way to register custom nodes :)