[$250+ bounty] Filters/Effects/Shaders for Individual Objects [Feature Request]

[Silver-Streak]

Description

PixiJS Filters/Effects/Shaders have been implemented for a while now, but they can only be applied per layer. This works for some filters (CRT, BulgePinch, etc), but many times users will want to only impact a single object.

There are workarounds such as moving the desired object to an empty layer that has the effect on it, but this becomes very complex and unmanageable once you get past 1-2 objects or different effects.

• As an example, if you want the glow effect on the player character as well as a chest, but you want them to glow different colors, you now need two completely separate layers that are following the main layer's camera and have the different layer effect setups.

A per-object effect/filter/etc would be much better suited in these cases, and could open up additional possibilities for effects that don't make sense for a layer-wide filter.

This is related to the existing trello card here: https://trello.com/c/5U5dWRSd/34-support-for-visual-effects-shaders-for-objects

Solution suggested

Allow for adding effects to objects just like we currently can for layers, in the object properties panel. While there could be a use case for adding/removing an effect via events, I think making it behave similar to layer effects where you add it to the panel and then can disable/enable it will be more uniform.

Wendigo has already provided a potential solution on the trello card:

If someone wants to give it a go, here is a working example as a behavior. Pixi also supports multiple filters per drawable object / container by adding all instances as an array to their "filters" property. https://github.com/Wend1go/GDevelop/tree/objectfilters/Extensions/ObjectFilterBehavior

Pixi's documentation on filters is available here: https://pixijs.download/release/docs/PIXI.Filter.html

While I cannot help implement this, I intend to open a bounty for this feature once confirmed we're good with this writeup.

Alternatives considered

1. Educate users on workaround of moving objects to specialized layers for just that object. Document and detail all downsides, including performance, complexity, etc. Relatively complex and different behavior that users are expecting.
2. Do not allow for effects on single objects at all. Long-term limiting for the engine and puts it behind things like Phaser as far as flexibility.

[Silver-Streak]

@4ian let me know if you have any concerns on the above or it looks good, and I can get a bounty opened up on it.

[HarsimranVirk]

@4ian we can use a similar approach for sprite objects as we do for layers (PIXI Containers and PIXI Sprites are equivalent as far as filters are concerned). I can set up a prototype to check performance hits.

[4ian]

@4ian we can use a similar approach for sprite objects as we do for layers (PIXI Containers and PIXI Sprites are equivalent as far as filters are concerned). I can set up a prototype to check performance hits.

Yes please!

• On PIXI side, I think it can boils down to "just" setting filters on the renderer object.
  • For performance we have to check, that will be dependent on Pixi... but maybe in the future we can integrate in the profiler the number of effects rendered, and warnings if you're using too much effects.
  • (Similarly, I'm thinking of adding something like gamestats.js (https://github.com/ErikSom/gamestats) or similar to enhance the profiler with real time info and be able to warn when there is too much usage of effects. But that's for another PR :))
• We should probably reuse the tools in pixi-filters-tools.ts to handle the filters set on an object.
• On the action/condition side, we should re-do the same exact actions/conditions as for layers, but for objects :)
• On the editor side, we should also reuse the same existing component (EffectsList) - and probably put it in an "Effects" tab next to "Behaviors" and "Properties" in the object editor (in the future, we can add the variables to have everything in the editor, but that's another PR).
• Finally, in the project structure in C++, we should augment Object.h so that it has an effects container. See HasEffectNamed/GetEffect/GetEffectPosition/GetEffectsCount/InsertNewEffect/InsertEffect/RemoveEffect/SwapEffects in Layer.h.
  • Note that I made the bad decision I think of putting all these methods inside gd::Layer directly. Instead, we should probably create a gd::EffectsContainer with only these methods. Then we: 1) Add a gd::EffectsContainer effects; member in gd::Object and 2) refactor gd::Layer to inherit from gd::EffectsContainer (to keep backward compatibility) (or we could even refactor to use composition, i.e: a gd::EffectsContainer effects;... that would be better!!).
• That's it?? :)

Because I think this is a very important and long over due feature, I'll put a substantial bounty on it.

[4ian]

I've put a $250 bounty on this because it's an important and long overdue feature :)

Anyone is free to contribute to it too! :) I'll be happy to give an additional $100 for proper refactoring (making a gd::EffectsContainer and using it as a member in gd::Layer), performance checking (so that we know how well it works) or general code quality improvements, and $50 for updating the wiki documentation to add/update pages about effects to explain their usage with objects.

In case this involve multiple persons, the bounty will go to one person but I'm happy to split the remaining $100+$50 according to the work put by each person. Just ask me if something is unclear! :)

[HarsimranVirk]

I'm just making a list here to point out major tasks: (Correct me if its wrong @4ian)

• [x] Decouple gd::Layer and effect functions -> Make a new class gd::EffectsContainer
• [x] Add a gd::EffectsContainer member in gd::Object
• [ ] Create a gdjs.PixiObjectEffectsManager and adapt runtimeobject.js accordingly
• [x] Add EffectsList in the editor.
• [ ] Create actions/conditions in BaseObjectExtension to manipulate effects during the game

[4ian]

That's the idea globally, apart on this:

Adapt runtimeobject.js accordingly. Add functionality in spriteruntimeoject.js and its pixi-renderer.

The idea is that effects are working with all objects :) That's why they are so powerful. On the IDE side, this will be handled, as you properly mentioned, by the fact that all gd::Object will have a gd::EffectsContainer (like a layer), so we can just pass object.getEffects() to the EffectsList and, here we go, we can edit the effects on an object.

Then at runtime, the base runtime object will store the effects data in the object, and will ask the renderer to create, remove an effect etc... whenever the object is created, or a method to manipulate an effect is called.

In practice, because the effects can work on any PIXI.DisplayObject, we don't need to implement this in each renderer :) We can probably make a single gdjs.ObjectEffectsManager, which will be called by gdjs.RuntimeObject any time an operation is to be made. It will pass to it the renderer object, (see getRendererObject in gdjs.RuntimeObject). This ObjectEffectsManager (which will be named PixiObjectEffectsManager for the Pixi version - and unimplemented for Cocos) will probably have methods very similar to what is in gdjs.LayerPixiRenderer: addEffect, removeEffect, setEffectDoubleParameter, setEffectStringParameter, setEffectBooleanParameter, hasEffect, enableEffect, isEffectEnabled. Except that it will act on the object (and its renderer).

Let me know if something is not clear.

You can replace Add functionality in spriteruntimeoject.js and its pixi-renderer by create a gdjs.PixiObjectEffectsManager and Adapt runtimeobject.js accordingly is done more or less at the same time :) Finally, you can also add a check "Create actions/conditions in BaseObjectExtension to manipulate effects during the game" (like the layers).

[HarsimranVirk]

@4ian, so we will be having, a class called ObjectEffectsManager and another for pixi, called PixiObjectEffectsManager. My understanding is that in the ctor of RuntimeObject, we'll initialize an object of class ObjectEffectsManager.

Right now, I pass the effect data and the runtime object to the new object of class ObjectEffectsManager. I have defined a function in class ObjectEffectsManager called objectRendererInitialized() which is called in the onCreated() of RuntimeObject. (Won't need this, if we pass the rendererObject as an argument to each function, as this is just to change a boolean variable to know that the renderer object is initialized)

I'm a bit confused about how should I work with PixiObjectEffectsManager. How does ObjectEffectsManager delegate tasks to the PixiObjectEffectsManager? Or, to put it another way, how should we figure out whether to use PixiObjectEffectsManager or CocosObjectEffectsManager?

Also, one unrelated doubt, whenever I define a new file in GDJS/Runtime (called objecteffectsmanager.ts), the class isn't available in the preview. Do I have to register it somewhere?

[4ian]

My understanding is that in the ctor of RuntimeObject, we'll initialize an object of class ObjectEffectsManager.

No I think this is too heavy, you don't want to have thousands of ObjectEffectsManager in your game, this would bloat the memory. Instead, the ObjectEffectsManager is a single object, much like gdjs.PixiImageManager. It's probably something that is accessible from the RuntimeGame (runtimeScene.getGame().getObjectEffectsManager());

Right now, I pass the effect data and the runtime object to the new object of class ObjectEffectsManager. I have defined a function in class ObjectEffectsManager called objectRendererInitialized() which is called in the onCreated() of RuntimeObject

This is ok, though see if you can do it:

1. using a single gdjs.ObjectEffectsManager, stored on the runtime game (const objectEffectsManager = runtimeScene.getGame().getObjectEffectsManager()).
2. by just passing what is necessary: the renderer object (i.e: the PIXI object) and the effect data. In other words, in the onCreated(), so something like objectEffectsManager.initializeEffects(this.getRendererObject(), effectsData)

Same for anything else that uses the effects :) When you want to manipulate/create effects, call a function of objectEffectsManager, passing it the renderer object and any data that is required.

t how should I work with PixiObjectEffectsManager. How does ObjectEffectsManager delegate tasks to the PixiObjectEffectsManager?

Basically, we do something like this inside the PixiObjectEffectsManager file:

  export const ImageManager = gdjs.PixiImageManager;
  export type ImageManager = gdjs.PixiImageManager;

So that the rest of the engine is not aware it's using a pixi specific class. It's just using a class called gdjs.ObjectEffectsManager with a set of methods (you can think of gdjs.ObjectEffectsManager as an interface, and you could do it with an interface in TypeScript by the way). If someone wants to implement a new renderer, they need to do the same interface: have at least the same methods used in the engine.

But I would say for now don't really care about this :)

Also, one unrelated doubt, whenever I define a new file in GDJS/Runtime (called objecteffectsmanager.ts), the class isn't available in the preview. Do I have to register it somewhere?

Yes in void ExporterHelper::AddLibsInclude ;)

[4ian]

....aaaand this is done!

@HarsimranVirk thanks for your earlier work on gd::EffectsContainer and the initial work to implement this in the game engine + editor :) I'll let you claim the bounty on https://www.bountysource.com/issues/98425556-250-bounty-filters-effects-shaders-for-individual-objects-feature-request?

Note that a few filters had to be deactivated because of rendering not at the proper position on the objects (I reported this on the Pixi filters repo, unsure if they will fix it soon though).

[Bouh]