Feature: Gmic and OpenFX integration

[blurymind]

Gmic is a popular opensource image effect system that has hundreds of filters (new ones coming out every week). It is extremely powerful and versatile. Both krita and gimp now have plugins to use it.

G'MIC is a full-featured open-source framework for image processing. It provides several different user interfaces to convert/manipulate/filter/visualize generic image datasets, ranging from 1d scalar signals to 3d+t sequences of multi-spectral volumetric images, thus including 2d color images.

These interfaces are:

1 - A command-line interface gmic, to use the G'MIC features from a shell. In this setting, G'MIC may be seen as a friendly companion to the ImageMagick or GraphicsMagick software suites....http://gmic.eu/

More interestingly - non linear video editors such as blender, flowblade and even a compositing package such as natron is starting to gain access to it! Currently it is a planned feature for natron - so it can operate on image sequences - not just individual images. https://github.com/MrKepzie/Natron

It's license should be studied here: https://github.com/dtschump/gmic-minimal/blob/master/COPYING It's github is here: https://github.com/dtschump/gmic-minimal

Basically, granted the license is compatible, it can be implemented as a node for the FX schematic in opentoonz. An easier approach to this would be to have it just as a layer adjustment filter - the way it is in krita and gimp.There are already several QT gmic addon implementations - not just krita's.

Relevant links: g+ news (follow for new filters news)- https://plus.google.com/+GMIC_software website - http://gmic.eu/

The colorize filter - Gmic has a filter that implements lazybrush - the coloring technology that tvpaint is currently using. Here is a demonstration of it here: https://www.youtube.com/watch?v=YigbVY9s6gU

The point here is that the first who got this feature as opensource was gmic devs- because gmic is a very good library for image processing effects!

Another feature request can be made on lazybrush alone :) But this is such a huge design task. having lazybrush technology in opentoonz would mean creating a new type of a level. Filed it here - https://github.com/opentoonz/opentoonz_docs/issues/195

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/34955289-feature-gmic-and-openfx-integration-access-to-hundreds-of-filters-fx-stack-or-as-a-level-color-adjustment-tool?utm_campaign=plugin&utm_content=tracker%2F33713530&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F33713530&utm_medium=issues&utm_source=github).

[yamisama]

I would suggest to do this indirectly by adding support for OpenFX plugins instead. OpenFX is an open standard initiated by The Foundry to standardize image effect plugins across post production software.

Pretty much every major compositing/editing/color grading software on the market supports this standard (with the notable exception of those from Adobe and Avid). There is a large selection of commercial and open source plugins available.

Since Natron is based completely on OpenFX plugins, the OFX plugin developed by the Natron team for their G'MIC integration could then be used in OpenToonz without any modifications. This has the added benefit that the integration would not be tied to OpenToonz releases and could be maintained independently, meaning that when new effects are added to G'MIC, the plugin could be updated and tested independently. To my understanding, OFX would even allow for open source plugins with incompatible license, as long as they are not deployed with the host software.

It should be noted, though, that the "Toonz Raster" level type in OpenToonz uses a rather smart custom way to represent color-mapped raster images (each pixel has a stroke coverage percentage value and two palette indices, one for fill color and one for line color, instead of the usual RGBA values). This type of raster data won't work with regular image processing effects natively. Images will have to be converted to RGBA in order to be processed via OFX or G'MIC, but I assume this is done anyway at some point in the node graph processing pipeline.

[blurymind]

@yamisama that is actually a good idea. We should dig more into it! :)

[yamisama]

I did some more research. The OpenToonz example plugins in the SDK do all processing in RGBA. It looks like plugins don't have to process in color indexed mode at all, which means the standard OFX processing pipeline should work perfectly fine.

I also found out that Natron and Tuttle OFX define reader/writer extensions for the OFX standard that allow for OFX-based file format plugins. If OpenToonz also supported these extensions, that would not only give us access to a lot of additional image and video file formats they are providing plugins for (most notably OpenEXR, DPX, and everything supported by OpenImageIO), but also Natron's FFmpeg plugin.

Having access to that plugin would mean that OpenToonz could get rid of the ancient 32-bit-only QuickTime dependencies with all their associated security risks and gamma bugs, while providing consistent output across platforms (right now, video rendering is only available on Windows as far as I'm aware). We would be able to render directly to a wide variety of professional video formats like DNxHD and ProRes.

Last but not least, the Natron team seems to also have an OFX plugin that integrates Disney Animation's SeExpr, which would also provide significant value to users since it allows them to build their own procedural effects with an easy to use scripting language, something that would come in very handy for special effects (clouds, fire, fog, etc.) in cartoon animation.

On the commercial side, we'd gain support for the famous Genarts Sapphire, Monsters, Boris Continuum Complete, Frischluft, and many more.

[adamearle]

I love your thinking and if you are waiting for someones approval than you sure as hell have mine.

I say go for the integration god know toons is in desperate need of an update. across multiple areas.

For example how many timelines does one application really need? This is one of the first things that should be soughted as well.

Adam Earle

On Sat, Jun 11, 2016 at 4:09 AM, Peter notifications@github.com wrote:

I did some more research. The OpenToonz example plugins in the SDK do all processing in RGBA. It looks like plugins don't have to process in color indexed mode at all, which means the standard OFX processing pipeline should work perfectly fine.

I also found out that Natron and Tuttle OFX define reader/writer extensions for the OFX standard that allow for OFX-based file format plugins. If OpenToonz also supported these extensions, that would not only give us access to a lot of additional image and video file formats they are providing plugins for (most notably OpenEXR http://www.openexr.org, DPX, and everything supported by OpenImageIO), but also for Natron's FFmpeg plugin https://github.com/MrKepzie/openfx-io/tree/master/FFmpeg.

Having access to that plugin would mean that OpenToonz could get rid of the ancient 32-bit-only QuickTime dependencies with all their associated security risks and gamma bugs, while providing consistent output across platforms (right now, video rendering is only available on Windows as far as I'm aware). We would be able to render directly to a wide variety of professional video formats like DNxHD and ProRes.

Last but not least, the Natron team seems to also have an OFX plugin that integrates Disney Animation's SeExpr http://www.disneyanimation.com/technology/seexpr.html, which would also provide significant value to users since it allows them to build their own procedural effects with an easy to use scripting language, something that would come in very handy for special effects (clouds, fire, fog, etc.) in cartoon animation.

On the commercial side, we'd gain support for the famous Genarts Sapphire http://www.genarts.com/sapphire, Monsters http://www.genarts.com/monstersgt, Boris Continuum Complete https://www.borisfx.com, Frischluft http://www.frischluft.com/frischluft/index.php, and many more.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/opentoonz/opentoonz_docs/issues/147, or mute the thread https://github.com/notifications/unsubscribe/APkuZK0xj3gHfumnX5kzv92UpkdCCwXBks5qKahGgaJpZM4IwaUc .

[jpturcotte]

Is anybody interested in looking into OpenFX plugin implementation? Because I am ready to put a bounty on it. @morevnaproject?

[morevnaproject]

@jpturcotte Thank you for reaching out! Please give me some time to examine the details of this request (@blackwarthog, please take a look as well).

Also, how many people are interested in this bounty?

[blurymind]

I am mostly interested in the lazybrush coloring feature that gmic provides. If you get the color marker positions to be copied between frames , it will make it very useful for coloring frames.

This request is more in the direction of using it for compositing though

On 3 Apr 2017 10:03, "Konstantin Dmitriev" notifications@github.com wrote:

@jpturcotte https://github.com/jpturcotte Thank you for reaching out! Please give me some time to examine the details of this request ( @blackwarthog https://github.com/blackwarthog, please take a look as well).

Also, how many people are interested in this bounty?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/opentoonz/opentoonz_docs/issues/147, or mute the thread https://github.com/notifications/unsubscribe-auth/AGMbVQyEUxgG4n9BlW4PRNae3alKNcNxks5rsLXPgaJpZM4IwaUc .

[jpturcotte]

@morevnaproject & @blackwarthog: Thank you for your interest. Please take your time, and tell us what you think. I do not know about developers, but maybe @shun-iwasawa and @ideasman42 could give us their quick impressions about feasibility and pertinence, if they are available. @blurymind: Yes, I understand. However, I believe OpenFX implementation could kill the most birds with the least amount of stones. It has already been requested on reddit (https://www.reddit.com/r/OpenToonz/comments/60a99g/does_opentoonz_have_support_for_ofx_effects_like/), and it would bring us in line with other FOSS and commercial efforts. Also, it would make us known in many circles, which could benefit development efforts.

[mcjadski]

I think this is a great idea, if it can be done.

[ghost]

Based on the discussion in opentoonz/opentoonz#1207 feature requests are going to be closed here on GitHub unless there is a developer actively working on the feature. Feature requests can be discussed at the OpenToonz Google Group: https://groups.google.com/forum/#!forum/opentoonz_en

[jpturcotte]

Posted to Google: https://groups.google.com/forum/#!category-topic/opentoonz_en/feature-requests/FnyZJ_E5e_E. @morevnaproject & @blackwarthog, would you still be open to look into this?

[yamisama]

@jpturcotte You might also be able to find developer interest in the bounty on the Natron/ButtleOFX teams, possibly also the OpenFX mailing list. These people are obviously very familiar with OFX.

The prospect of having fantastic plugins, such as the soap bubble effect that Iwasawa-san has kindly developed and the Dwango plugins, work in their software as well in the future (provided these effects would be migrated to OFX as well of course) might also garner some interest.

[morevnaproject]

@jpturcotte We are still open to look into this.

[jpturcotte]

@yamisama & @morevnaproject Great! Let us coordinate our efforts in the Google Group. There are already some valid concerns there.

[ghost]

Reopened since a Bounty looks to be posted sometime soon for this.

@morevnaproject @blackwarthog I think a bounty or fix for some of the basic vector level issues would be more pressing than this. Are you open to a bounty on those issues?

[mcjadski]

I'm inclined to agree with @turtletooth here. Unless you're confident that this would bring in more developers, I think this should be put off until the critical, program-ruining issues are fixed.

[RodneyBaker]

Follow up.

Was a bounty every opened on this? Was anyone taking this under wing?

[morevnaproject]

@turtletooth Sorry, I have missed your message. It worth to open a bounty for whatever anyone would like to get fixed. At the moment we are a bit busy with fixing other things, but as soon as we will return to "task choosing", we will pay attention to already funded bounties in the first place.

@RodneyBaker At the moment I am investigating other things, so I still not ready to give any price estimation yet.

[RodneyBaker]

@morevnaproject Thanks for the update. This is a large project to undertake so it's easy to see why there would be delays.

Recommend keeping this open as it does present significant gains in alignment with other open source software. This doesn't mean it has to be of any higher priority (feature wise) but only that it is currently on the development radar.

Added: I do note the recommendation from @yamisama who suggests why OpenFX development should likely take priority over Gmic development (because Gmic has been demonstrated to work through the path of OpenFX developement and integration.

[jpturcotte]

I just posted an update in the forums: https://groups.google.com/d/msg/opentoonz_en/FnyZJ_E5e_E/Ji2vM63SCQAJ.

[RodneyBaker]

I'm pretty sure this one is slipping to 2019 development cycle. @blurymind Do you see any more immediate path forward?

Sadly, no bounty has been posted for development of this yet. Closing for now with expectation it will reopen.

[RodneyBaker]

Reopening this FR as feature requests are now allowed in OT github.

[RodneyBaker]

Transferring this request to Opentoonz-docs repo for general roadmapping purposes...