Status of OpenMeshEffects

[EitanSomething]

What is the status of the project? Is it still being worked on?

[eliemichel]

Hi @EitanSomething, yes it is. I'm not rushing cause I have other projects as well, but I still need this somehow. I'm hopping to release a slithly more mature version soon, with at least the possibility to save files, which as you'll likely admit, a bare minimum to have.

The other important need at this stage is to support more point/vertex/primitive attributes, I'm thinking at least UVs and normals. As of now, the Open Mesh Effect standard does not provide a mechanism for it, so I'll have to alter it a bit.

What are you the most interested in regarding the project?

[EitanSomething]

I’m most interested in being able to use custom modifiers that people created in Houdini to be used in blender.

Will there be an OpenMeshEffects plug-in in Houdini or it be a separate application?

[eliemichel]

The Houdini's HDAs would be loaded using a dedicated Open Mesh Effect plugin: https://github.com/eliemichel/MfxHoudini The usage will be to get MfxHoudini.ofx (the core plugin), drop the HDA next to it, then use this Open Mesh Effect branch of Blender to load MfxHoudini.ofx as a modifier. Proof of concept works, I'd like to add the support for UVs and other extra attributes before releasing. For which kind of HDA would you be interested? Effects acting on geometry only, or more advanced thing reading/generating extra attributes?

[EitanSomething]

Can you created a https://blender.chat/ account. It would make it easier to communicate. Can you provide test files.

[EitanSomething]

Can you update to master

[eliemichel]

Yeah I am planning on syncing with 2.82. I try to sync with releases rather than with master, I feel it is a good trade-off between being a bit late and having to sync too often.

So you know, the current status is that I am focusing on reducing the amont of buffer copy when using the API. I'd like to minimize as much as possible the data allocated by plug-ins, by making them read data through a generic enough attribute layout mechanism, maybe inspired by OpenGL's Vertex Array Attributes.

The major problem for now is that when integrating OME in Blender, I have to 1. copy blender internal mesh representation to OME representation 2. call the plugin, that might - as in the case of the Houdini - reallocate data another time to match Houdini API's layout and 3. copy back from OME to Blender. With the ongoing work, steps 1. and 3. would be removed.

[EitanSomething]

I’m currently updating to master . Can you use clang format because every time I save open mesh effect files it reformats

[eliemichel]

(I am @Elie on blender.chat) Ok yeah I'm not using the clang format yet, maybe deactivate your autoformat for now on this branch, I'd avoid mixing up formatting stuff with "actual" code changes.

[EitanSomething]

I can just reformat every once in a while

[eliemichel]

Also, I have some ongoing uncommitted work about what I was saying in the previous message, which is why I haven't updated yet. Ok for formatting, since I'm no longer alone I guess I'll move my ongoing stuff to some branch.

I haven't checked is all examples are still working after the recent changes brung to the API, that could be something worth checking (including MfxVcg), and ideally automating those changes.

[EitanSomething]

Starting to build blender. I had to reinstall the libraries.

[EitanSomething]

Can you send me test files

[eliemichel]

Test plugins are in the repo here: https://github.com/eliemichel/OpenMeshEffectForBlender/tree/openmesheffect/intern/openmesheffect/plugins

Building the branch should build openmesheffect_color_to_uv.ofx, openmesheffect_identity_plugin.ofx, etc. You can just test with a cube where you add an Open Mesh Effect modifier basically ;)

[EitanSomething]

Found them. Can you commit the Sync with master

[eliemichel]

Mmh sorry if I make you work again but would you mind if I push what I was doing first and you sync from that? I was working this last hour on making it be stable enough for a commit and it is ready for push now.

[EitanSomething]

okay

[eliemichel]

Thanks, Done. You can go ahead!

[EitanSomething]

Can I get commit access. There are a bunch of small code style changes that I will be making and it would be easier if I could commit them myself.

[EitanSomething]

I sent in the pull request

[EitanSomething]

Can you accept the pull request

[eliemichel]

Problems are 1. you merged with master rather than 2.82 release as I suggested above; and 2. after putting some though in it, unless it causes major reading issue, I'd like to stick with the current code style for now. Either ways it makes it really hard to review that there is a mix of merging and actual changes, so I'll think I'll handle the merge.

Was there something you really need in master that isn't in stable?

[EitanSomething]

If we stay with master it will be easier when we submit the code because when we move from 2.82 to master it will likely cause things to break and it will be easier to fix them if we catch them more often. I will also be able to integrate it into the asset manager when that gets committed.

[EitanSomething]

all the mesh data should be put into a struct instead of the multiple variables like it is now

[eliemichel]

all the mesh data should be put into a struct instead of the multiple variables like it is now

Can you be more specific?

[EitanSomething]

A struct similar to blenders Mesh struct. It would prevent mfxModifier.c from getting messy. I submitted a pull request Syncing with 2.82.

[eliemichel]

I admit mxfModifier. is messy, it is actually kind of the main where I put stuff before it eventually gets cleaned up into separate file. I try to contain messiness in that if you will. Actually I am working on a way to use directly Blender's Mesh structure, but out of curiosity what variables of mxfModifier.c would you put in this struct?

I was thinking also a good way to help would be to try to write a simple plugin to check that the instruction guide is up to date (which it is not totally for sure).

[EitanSomething]

We should add MEdge, MVert, and MFace into the struct.

[eliemichel]

There is already a notion of vertice and face. Adding edges is being discussed for cases in which one needs to attach custom attributes to them. But I am still not sure which struct you are talking about exactly.

[EleotleCram]

Just discovered this! Very interesting. I was reading this tutorial because I had a similar idea: Exposing non-destructive modeling/blender modifiers to plugin architecture, turns out you already had drawn the same conclusions :)

Will try to build this fork later today...

[eliemichel]

Cool, I've just ported it to Blender 2.83 LTS, will release a binary soon I think!

[EleotleCram]

Building from source worked without major issues. Only had to change max into fmax in file intern/openmesheffect/blender/intern/mfxCallbacks.c to successfully compile on Ubuntu 18.04.

Will try to write an ofx plugin I've been thinking about now -to see where that API is at- next...

[eliemichel]

Thanks for the feedback! I'll fix this max.

For writing plug-ins, the most up to date example so far is the mfx_color_to_uv. It uses the plugin_support helper functions that I started working on to make attribute manipulation easier. There is still a lot that can be improved on this helper now that I am no longer expecting too much changes to the OpenFX API itself.

I am particularily interested in knowing what you find hard/painful to do with this API (and also what you find nice about it so it doesn't get dropped in a future change!)

[gerroon]

Any ideas if this branch will get into the master at all? Were you able to talk to Blender Devs about it to see if they have any interest in integrating this?

[EleotleCram]

I think it would be sort of important to at least register the ModifierType in the main Blender branch (even if it is #ifdef-ed out). (Getting a single line entry in DNA_modifier_types.h would suffice, I think)

As I understand it, basically OpenMeshEffectForBlender (blender-mfx in short?) now is forking the blender file format because of the change to the DNA and the claim of ModifierType=57 to be the OpenMeshEffect modifier.

[eliemichel]

@EleotleCram This is a very relevant point, I'll demand this.

[EleotleCram]

@eliemichel Should there ever be a problem in this regard, people can use this tool to update their .blend files to a new (free) id in the modifier types enum.

[eliemichel]

Nice, thanks for letting me know, and for making this tool :)

[Kleptine]

Is there a roadmap / status for this project? We're very interested in using it, but it's a bit unclear how close it is to being stable.

[eliemichel]

Hi @Kleptine

• The OpenMeshEffect API itself is settling, you can find its documentation here https://openmesheffect.org
• The implementation in Blender is usable. Remaining crashes are mostly caused by plugins, the host is not doing all the safeguarding it could to prevent faulty plugins to crash the whole program.
• There are too few available plugins so far so one point on the roadmap is to produce more example and resources/tutorials to help developpers write their own plugins.
• Then another target will be to support at least one other host (I am thinking of a game engine) in order to show how Open Mesh Effect can be a way to interexchange scenes containing non-destructive effects between softwares. The interest of a game engine is that it may also be used at runtime (which is a notable difference with the Houdini Engine since the latter requires a licence).

I am really interested in knowing your use case, I could even bias the roadmap towards it!

(PS: I don't usually take so much time to reply but, you know, summer break ;) )

[Kleptine]

Our use case is for integrating Houdini with Blender. We'd like to use Houdini for automated processing tasks like generating LODs, generating UV maps, etc. So we probably wouldn't use too many other plugins, unless they were also helpful to us in a gamedev context.

Biggest thing for us is just stability. End of the day, if it keeps crashing, our 3d artist can't do much about it. So half of this is me trying to convince them that it's worth using a separate build of blender! Other stuff like having better properties from HDAs exposed, other than floats, is pretty critical, but not totally show-stopping.

I'd be curious to see integration in Unity. Ideally all of this would run on import. Unfortunately, you might have a tough time, as there's no easy way to customize the import behavior of meshes. Personally I'm not so interested in Game Engine integration -- but only because most of the team doesn't have Houdini Engine installed, and I don't really want to make it a general dependency of running the game.

Blender is a much better place for us, because we can apply all this houdini stuff as a modifier, and it spits out an fbx like normal. Plus, then we can run much more expensive Houdini operations, because we know that it only needs to be run when the Blend file is modified. (If you add it to the engine, then it runs every time people are editing the game).

[eliemichel]

Alright then don't hesitate to open as many feature request/bug reports on the Houdini-related plugin! https://github.com/eliemichel/MfxHoudini For instance other-than-float properties are supported on this branch of Blender's side, it's the houdini bridge that is missing it (it's not a big change, it just wasn't in my priorities so far).

I value this stability concern, from your experience how crashy has it been? Did you figure out some patterns of action that make it crash?

(NB: Nobody can make the Houdini Engine a runtime dependency of a game or they'll have to ask all their players to buy a Houdini Licence!)

[Kleptine]

Will do! We're still evaluating, and this stuff is something we'll need a little later down the pipeline.

Stability-wise, it's not been terrible. I used it for a few hours, and it crashed when I was tweaking one of the HDA parameters. But Blender isn't the most stable thing to begin with, so it's hard to tell exactly what happened. Unfortunately couldn't reproduce (it might have something to do with setting a parameter outside the valid range).

Yeah, I don't mean a runtime dependency, but I don't even want to add it as an Editor-time dependency. Because then every developer on the team who touches the game has to have it installed. (Which is annoying and expensive)

[eliemichel]

What could help when it is hard to reproduce would be to run Blender from a terminal (Win+R, "cmd", then type the full path to blender.exe within double quotes ") so that when Blender crashes, the error log remains and can be reported here!

Also, just to be clear: Using MfxHoudini doesn't free you from needing a Houdini licence. The goal of Open Mesh Effect is to go beyond this link with Houdini and have a set of nodes (called "plugins" as they are loaded at runtime) that could get used in several softwares. For now there is Blender, but I'll develop hosts in other softwares, in particular in game engines. If you are a developper yourself, you can already write your own such plugins in C++ and use them as modifiers in Blender.

[tkarabela]

Hi @eliemichel, I recently found OpenMeshEffects while researching how to write custom modifiers for Blender. The idea of OpenFX-like standard really appeals to me and it's basically what I was looking for, so I've been playing around with it and I'd love to contribute to the project to help it get to production-ready status :)

My use-cases/goals are:

• using VTK in Blender - there are other ways (notably BVTKNodes), but they are more geared towards importing scientific data than general 3D work, non-destructive modeling and such
• experimenting with custom mesh effects/deformations in Blender - before OpenMeshEffects, there didn't seem to be a nice dynamic way to do this

After reading through the Blender host and MfxVCG plugin code, I've started to write MfxVTK using the MfxEffect classes from MfxVCG (I hope it's okay to include CppPluginSupport in my project? According to CppPluginSupport/CMakeLists.txt, it has the Apache 2 license.). Thanks to good documentation and the handy C++ wrapper, I was able to make it work, and now I'm starting to hit edge cases (like how to input/output edges ^^ or how to work with attributes). I've put Windows and Linux builds of my plugin on GitHub: https://github.com/tkarabela/MfxVTK/releases

Limitations I've encountered so far:

• (#27) working with something else than polygonal mesh (point cloud, edges)
  • the specification doesn't clearly say how to represent these (from my understanding: point cloud = points without vertices or faces; edges = 2-vertex faces)
  • the implementation has problems with edges (I've been looking into how to fix this)
• (#7, #21) working with attributes
  • listing all attributes of OfxMeshHandle, more customary attributes like kOfxMeshAttribVertexUV
  • plugin describing its input attributes, host providing UI to select them (eg. vertex weights, like builtin Blender modifiers)
• working with additional mesh inputs (like builtin Blender modifiers)
• (#20, kind of) using it as a deformation modifier for complex mesh, like a rigged character - I haven't tested this much yet, but currently it "forgets" some things like vertex weights
• UI niceties like min/max value for parameters, enum parameters (#18) and custom tooltips

[eliemichel]

Thanks a lot for this insightful feedback! I am glad to see that the project is readable enough for newcomers to find their way in writing new plugins. :) I'll try building yours.

Totally ok to import CppPluginSupport, it's meant for this very use case!

You can open an issue for your point "working with additional mesh input". This is an issue of the Blender integration, as the standard itself supports it already.

About deformers, I think that the Open Mesh Effect standard should get added a way for a plugin to tell that it only moves points, without touching the connectivity. This would enable a lot of speed ups in host implementations, and is by the way already something that Blender modifiers must tell when being defined. For now the Open Mesh Effect modifier always tells that it might be changing topology, turning such optimizations off.

You last point is also more about the Blender implementation, feel free again to open an issue to log those ideas.

[SetoKaiba]

Why not merge back to master as it doesn't violate the GPL license now? Or please at least catch up with the LTS verison. The latest LTS version is 2.93.1.

[tkarabela]

Why not merge back to master as it doesn't violate the GPL license now?

Some of the challenges are mentioned in this thread, both technical and political. Since then Blender also got Geometry Nodes in 2.92 which overlap with some use cases of OpenMfx.

Adopting OpenMfx would be a nontrivial commitment for Blender, and the current implementation (as well as the larger OpenMfx ecosystem) is not yet at a point where it would make sense for them, IMHO. This would imply that a version of OpenMfx would be effectively frozen in an LTS release for two years. This may do more harm than good if OpenMfx is not mature enough by that point.

[eliemichel]

@SetoKaiba The project is in sync with the last LTS already (it is currently based on 2.93). Regarding merging in Blender's master, I don't have the power to do this myself, and @tkarabela explains well the reasons why I am not particularily fighting for it!

[SetoKaiba]

@eliemichel I mean the latest LTS. The latest LTS is 2.93.1. It has some minor fixes. So I'm trying to compile one myself. Thank you for fixing the compile of test. I'll try to use worker_compile.py to compile one myself.

[eliemichel]

Oh ok I did not follow well this minor update. Generally speaking there is usually no conflict when rebasing onto an updated version of upstream Blender, unless there is a new modifier introduced in that version.