CuraEngine plugin architecture

[jellespijker]

plugin system for CuraEngine

gRPC definitions: CuraEngine_grpc_defintions: https://github.com/Ultimaker/CuraEngine_grpc_defintions example plugins: CuraEngine_example_plugin_cpp: https://github.com/Ultimaker/CuraEngine_example_plugin_cpp CuraEngine_example_plugin_rust https://github.com/Ultimaker/CuraEngine_example_plugin_rust CuraEngine_example_plugin_python: https://github.com/Ultimaker/CuraEngine_example_plugin_python PullRequests on existing repo's: https://github.com/Ultimaker/CuraEngine/pull/1878 https://github.com/Ultimaker/Cura/pull/15563 https://github.com/Ultimaker/libArcus/pull/150 https://github.com/Ultimaker/pyArcus/pull/5

The Engine Plugin System uses Protobuf https://protobuf.dev/ and gRPC https://grpc.io/

Suggested Architecture

Cura container

Cura is responsible for starting the engine plugin service before the engine. In the local variant, this is done with a bash command adding the appropriate arguments (such as IP localhost and port number) these arguments are send along to the engine backend plugin which is responsible for starting engine, providing the settings, mesh and engine plugin (IP and port) data.

Engine plugin container

The engine plugin container can be written in any language, provided that it can handle Protobuf messages. The engine plugin service is started and will listen to any message on the specified socket. It should be able to respond to the following message types:

Identity: This should return at the very least the version and maybe in the future possible an authentication response, to ensure that we hook in to a valid plugin.

Process(data, args…): This is a request to process the data, with some additional arguments and return the modified data in the agreed upon format, if an error occurs it should respond with an error message instead.

Shutdown: This should stop listening and exit the program

CuraEngine container

CuraEngine is to be extended with a PluginRegistry, which will take care of the book keeping, coupling a slot, (which is a hook, defined by an certain operation, version range and call signature) with a PluginProxy (which provides an uniform API for the remote plugin service and acts like a regular C++ function object, with a default fallback behaviour if no valid plugin services is running.

Since we now have to take into account that the update cycle for engine plugins might differ from that of Cura and therefor CuraEngine, and that a user might still run an older version of a engine plugin, we also need to validate if a plugin is compatible; This is done with the PluginValidator which is a type that specifies the semver range. This validator can be extended in the future with additional checks, such as an authentication scheme, which would allows use to only run UltiMaker approved plugins (@Remco Burema will look into this further)

Because the data transfer between a plugin and the engine is defined as a “generic” Protobuf message the engine also needs a way to convert that Protobuf type to CuraEngine native types. This is done with a Converter which maps the protobuf type to the native types (e.q. Polygons etc.). The converter throws a runtime error when it receives an error response from the plugin.

A slot is defined by a PluginValidator, a specialisation of the Converter<proto::Request, proto::Response> and a SlotID (e.q. POSTPROCESS, SIMPLIFY, etc.).

Proto definitions container

The common types between a plugin and the engine are the Protobuf messages. These messages should live as a single source of truth in their own repository (with a very loose license e.q. MIT).

The Protobuf definitions are:

SlotID (e.q. POSTPROCESS, SIMPLIFY, etc.), these describe the specific functionality of the plugin

Identity_Request request detailed information from a plugin

Identity_Response provides the version of the plugin, can be extended in the future with additional verification methods

Error_Response provides an error message if the plugin encounters an error

Each type of slot should have a corresponding Request and Response message in the definition file, e.q. Simplify_Request , Simplify_Response

Sequence diagram

To use the plugins from the front-end for now use environmental variables as below, while running this Cura branch:

SIMPLIFY_ENABLE: default disabled, setting this to any value will enable the plugin SIMPLIFY_ADDRESS: defaults to localhost SIMPLIFY_PORT: defaults to 33700 POSTPROCESS_ENABLE: default disabled, setting this to any value will enable the plugin POSTPROCESS_ADDRESS: defaults to localhost POSTPROCESS_PORT: defaults to 33701

[github-actions[bot]]

Unit Test Results

26 tests  ±0   26 :heavy_check_mark: ±0   13s :stopwatch: -1s   1 suites ±0     0 :zzz: ±0    1 files   ±0     0 :x: ±0 

Results for commit f278ed95. ± Comparison against base commit ef6a3c79.

:recycle: This comment has been updated with latest results.

[fieldOfView]

[jellespijker]

Side note. It should be fairly easy to extend this system to allow for parallel execution of the same plugin over multiple hosts as a sort of microservices. Maybe not relevant for the current examples, but if we decide to write a "plugin" handling the complete support generation or infill etc. We could extend this logic such that multiple plugins are hooked up to a roundabout queue in a slot which will distribute the workload over multiple services/computers

[fieldOfView]

I see that - among other uses - plugins will be able to postprocess gcode output. Will they also be able to affect the layer data that is displayed in Cura? It would also be interesting if a plugin could preprocess geometry and settings (eg adding in custom helper meshes such as mouse ears, support meshes in marked places, etc).

I like the doubling down on Arcus communication, but it may require users to add further exceptions to overzealous firewalls.

[jellespijker]

@fieldOfView Yes one of the test slots is indeed postprocessing on the gcode, unfortunately that doesn't change the preview in Cura :-( But on a happy note: @casperlamboo and @nallath managed to run an engine plugin for Postprocessing, which was written in Python and ran on @nallath his computer, while CuraEngine ran on @casperlamboo his computer.

@casperlamboo also has a Rust plugin and I'm busy with a C++ plugin.

We would love to have your input where you think certain slots can be placed.

Side-note this is all very preliminary and subject to change

[jellespijker]

@fieldOfView we're not doubling down on libArcus btw. We will keep libArcus for now for communication with the front-end as is. But the plugin system uses gRPC https://grpc.io/

Which has support for way more languages then libArcus and also allows for secure communication.

[fieldOfView]

we're not doubling down on libArcus

Ah, I saw protobuf and my mind went to libArcus.

We would love to have your input

Since you asked... I don't think postprocessing gcode is useful unless there is also layerdata postprocessing. They can be one slot providing and returning both gcode and layerdata, or two independent slots.

Preprocessing the settings and meshes can also be very powerful. For example the whole trick of Blackbelt Cura (slicing for a belt-style printer) is about 70% preprocessing, 20% postprocessing layer data, and 10% frontend tweaks. It would be great (and powerfull) to be able to formalise that in a higher-performant plugin.

I am wondering if the "simplify" slot could not also be used for other purposes than simplification (eg quite the opposite: adding texture or a text embossing to a path). So I would probably call it something along the lines of path manipulation instead.

Could infill patterns be added with plugins? A slot that provides a path and then returns an infill path. Would there be a way for a plugin to be specifically called? Eg only when a certain infill pattern is set (say "hexagonal"), or are all plugins for a slot always called (in what order?)

Will plugins have access to settings sent to CuraEngine, or do these have to be explicitly passed via the slot? If the latter, that would make it a lot harder to be able to have plugins that have settings in Cura.

[jellespijker]

Yes I have asked ;-) and I'm loving the ideas here

I don't think postprocessing gcode is useful unless there is also layerdata postprocessing. They can be one slot providing and returning both gcode and layerdata, or two independent slots.

Agreed, but for now this will serve our Print Process & Material engineers, try out the plugin systems, modifying GCode in Python so that we Cura devs can finally work in piece with out them nagging us ;-)

Preprocessing the settings and meshes can also be very powerful. For example the whole trick of Blackbelt Cura (slicing for a belt-style printer) is about 70% preprocessing, 20% postprocessing layer data, and 10% frontend tweaks. It would be great (and powerfull) to be able to formalise that in a higher-performant plugin.

This requires some more thoughts and I think once the initial work is done, we might look into this in detail. We could invite you for a brainstorm session in due time for something like this.

I am wondering if the "simplify" slot could not also be used for other purposes than simplification (eg quite the opposite: adding texture or a text embossing to a path). So I would probably call it something along the lines of path manipulation instead.

Simplification is for now just a test slot, it is unlikely that it will end up as an actual slot.I like the idea of adding textures, but the simplification slots isn't the best candidate for this. Since this is also used to sanitize the input for the Boost voronoi algorithms. You know the algorithm which is currently responsible for 90% of the engine crashes.

Could infill patterns be added with plugins? A slot that provides a path and then returns an infill path. Would there be a way for a plugin to be specifically called? Eg only when a certain infill pattern is set (say "hexagonal"), or are all plugins for a slot always called (in what order?)

Infill patterns is in my opinion a very quick win.

The same as the earlier mention ears etc., which could result in a adhesion slot imo.

Will plugins have access to settings sent to CuraEngine, or do these have to be explicitly passed via the slot? If the latter, that would make it a lot harder to be able to have plugins that have settings in Cura.

We want to keep the data transfer as generic as possible between plugin and engine, due to various reasons, such as supporting different languages.

But each engine plugin will most likely be shipped with a Cura Front-end plugin, we will work on have some system in place to register specific settings in the front end for a specific engine plugin

[nallath]

But each engine plugin will be shipped with a Cura Front-end plugin, we will work on have some system in place to register specific settings in the front end for a specific engine plugin

Note that I do expect that the frontend plugin that ships with those backend plugins to be pretty simple. As a basis, all they would have to do is ensure the engine plugin runs and is registered. It's also the "natural" place where extra setup can be done (settings, etc).

[fieldOfView]

this will serve our Print Process & Material engineers, try out the plugin systems, modifying GCode in Python

But don't we already have ample Python-based postprocessing opportunities in Cura? (commonly via the OutputDeviceManager.writeStarted signal)

[jellespijker]

But don't we already have ample Python-based postprocessing opportunities in Cura? (commonly via the OutputDeviceManager.writeStarted signal)

Sure but we also need some test slots to design and implement the CuraEnginePlugin architecture and this is a very simple slot to test with.

Send string to plugin, get modified string back.

[jellespijker]

Checkout the discussion on Cura repo, regarding engine plugins https://github.com/Ultimaker/Cura/discussions/15629