Modularisation of mgear

[miquelcampos]

From @mottosso on November 14, 2017 8:31

Hi Miquel,

We talked briefly about splitting mgear up into modules, such that a user could install only the parts of interest and to improve maintainability of smaller parts, as opposed to one large library.

Here's how I see that working.

• Core
• Data
• Extensions

             _______ 
            |       |
     .------| mgear |-------.         core
     |      |_______|       |
 ____|____  ____|____  _____|_____
|         ||         ||           |
| shifter || rigbits || simpleRig |   extensions
|_________||_________||___________|
 ____|___
|        |
|  data  |                            data
|________|

Data

I'll start with this, as it's best suited for splitting apart. The Components, these are the data to an otherwise data-driven core framework. The data determines what can be built and details of how to build it.

Different data in, different rigs out. Simple.

    data
 ____ | ____
\     |     /
 \    v    /
  \       /
   \     /
    |   |
    |_|_|
      |
      v
     rigs

If data were to be separate from the core, then there could be many different sets of data, by many different authors. This enables others to build their own library of rig components and facilitates discussion about the data itself, without getting distracted by the underlying core library.

For example, there could be a discussion about the various ways there are to build a component for IK/FK switching, and compare and contrast their differences. There could be many components out there, each with their own documentation and usage scenarios. A collection of components could have its own website, like how Material Design and Bootstrap is data to CSS, documenting each individual component and thinking about the design of data, separate from the underlying framework.

Core

The core library is what glues the data together. It's contains the mechanism for exposing the mechanisms to the user and how to actually read and make use of them.

In here I see every module under the mgear/ package. The packages in there are what I'd call "extensions".

Extensions

An extension builds upon the core library to form something more specific than mgear itself. Shifter is a good example of this. Shifter implements a data-driven approach to a rig framework and is, I think, the most flexible and powerful of the existing extensions.

The key point here is that mgear is fully capable of existing without these, but these are not able to exist without mgear. The extensions can be developed and maintained separately from mgear, and vice versa. Their own issue tracker, their own release cycles etc. Meaning the cognitive load when trying to understand either mgear or any of the extensions goes down by the amount of code each contain.

Say for example that, as a user, I was only interested in Shifter. At the moment, because it is bundled alongside mgear and the other frameworks, I can't tell whether I also need to learn and understand these other extensions. I can't tell whether making a change to Shifter would break simpleRig and I also can't tell which parts of mgear is actually used in Shifter, or if Shifter is used in mgear (i.e. a cyclic dependency).

Notes

Finally, these are just thoughts. I don't feel I know mgear well enough to say exactly this is the ideal approach. But I do think it lends itself well to being split up as there currently exists a few different frameworks embedded into one repository.

Something I'd like to see which I think would help adoption and maintainability is separating anything compiled from things that need no compilation. I found it really difficult at first to make a contribution to this project because I (1) didn't understand that I needed to build (instructions could help here) and (2) didn't have a suitable compiler installed to actually perform the build. Because of this I wasn't able to actually test what I contributed, and without automated tests we ended up with a contribution that broke mgear!

Here's what I'd like to see..

$ pip install mgear

This would grant a user with the basic framework for building tools with mgear.

$ pip install shifter

Now they'd be able to leverage a data-driven approach to building rigs.

$ pip install shifter-basics shifter-mocap

And now they'd have some data to get started with.

Copied from original issue: mgear-dev/mgear#94

[miquelcampos]

Hi Marcus,

I would like to add @gatgui to the conversation.

In short... yes! this is pretty close to what I have in mind. :)

I am not sure about the pip install. I meant this is a new idea that I have to digest (but I like it ). Provably the ideal will be to support several types of distribution. i.e: one simple for casual users, animators, freelance, etc ... and another more oriented to a proper pipeline. The simple can be prepacked with all solvers and C++ plugins compiled and distributed in gumroad or similar. While we keep open the doors in git hub and other package dist using pip. (at less this is the first idea that comes to my mind)

Regarding this topic and the general refactor for PEP8 that is needed. I have several concerns that I need to think about:

1. I want to keep compatibility for version 2.x. So functions refactor and module split I think should be targeting version 3.0 (for the unittest I think should be implemented ASAP)
2. Is my current roadmap for 2.3 and 2.4 properly set? Maybe I should review the goals to move faster towards 3.0
3. What will be a realistic time frame to aim?

what do you think @mottosso @gatgui ? Any advice here?

Thanks, Miquel

[miquelcampos]

I have re-organize the projects and the date for the next milestone.

Now I am aiming for 2.3 version Due by March 31, 2018 (I know this is hard to preview, but I like to work with a deadline in my head. So the target is clear. I will adjust this based on the progress each months) This is 3 month more than the previous deadline. Lets see if is enough :P

I have closed the 2.4 project and move all the goals to version 3.0 . The main goal was the "Stepper" tool. But I think will be better to have the proper separation before adding more modules.

PEP8 refactoring: I will use pep8 for the new code, but I will wait until 3.0 for full refactor. If someone is importing mGear modules for another tool will not break until next release. Also I will try to perform other codes clean up, if doesn't affect functions or attributes names. i.e: line width, etc..

For the test suite I have created a new issue https://github.com/miquelcampos/mgear/issues/95

Also I will add a few new shifter components to be included in 2.3, and yep! after that I will try to stay strong and don't add any other new feature or enhancement to the list.

EDIT: Actually one more tools should be updated ASAP is the mocap tool. Only for this, I will add new folder/module called "animbits". This will be also separated tin the future.... Now is time to stay strong?

[miquelcampos]

From @mottosso on November 15, 2017 6:45

PEP8 refactoring: I will use pep8 for the new code, but I will wait until 3.0 for full refactor. If someone is importing mGear modules for another tool will not break until next release.

I might be able to help here, cosmetic PEP8 changes will not (or should not, at least!) have any effect on the functionality of the code.

I'll put together a PR so we can chat about specifics, and which rules of PEP8 to exclude (such as mixedCase variable names).

[miquelcampos]

Yes, cosmetics changes is what I meant here:

Also I will try to perform other codes clean up, if doesn't affect functions or attributes names. i.e: line width, etc..

[miquelcampos]

From @liorbenhorin on November 15, 2017 7:3

Hi Guys I just want to say that i'm reading this and it makes me very happy to see mgear future! I can't wait to contribute the second I have some time to breath :)

[miquelcampos]

Hi @liorbenhorin Glad to hear :) Thank!

btw: I am trying to expose all the ideas that I have for mGear here: https://github.com/miquelcampos/mgear/projects/5

Some are very vague exposed, but hopefully will start some interesting conversations.

[miquelcampos]

From @mottosso on December 2, 2017 17:13

Hi @miquelcampos, I've put together an example organisation here on GitHub with some ideas around how I imagine this to work. None of it is functional; it's purely visual for now, to discuss some ideas.

https://github.com/mgear-dev

Overview

Some things that are necessary for this to work:

• Components are separated from mgear.maya.shifter.components into a new project, one I've temporarily called "pandora". This then also contains everything that needs compilation, which makes mgear into a...
• Pure-python project, the core library is a plain Python package with no external dependencies. All surrounding repositories then depend on mgear
• In order for there to be a working combination of projects, e.g. v1.2 of shifter alongside v 2.6 or mgear, there is dist which is a collection of projects that are known to work together. I figure this is where you could potentially bundle mgear into something similar to how it exists today. Something convenient for the end-user to install, but more difficult to contribute to.
• In order for components to be separated, I've added a mock shifter.plugins module that I expect to handle discovery and caching of external components, such as Pandora.
• I've added some basic logos to mgear and shifter. They're all found on Google and isn't something we should use long-term, it's just a mock up.

The bottom line is that this method of organising the project means each part is easier to understand, have less dependencies, can be mixed and matched freely depending on what the end-user wants (for example, I'm typically only interested in Shifter), tests are smaller and more targeted, documentation targeted and more clear, etc. etc.

I've added README's to each project, have a look and let me know your thoughts.

It's a large departure from how mgear is organised right now, so I don't expect this exact layout to remain; it's just for discussing the topic!

[miquelcampos]

Hi @mottosso Thank you for moving this forward!

• Pandora is a catchy name, but... I would like to keep the components with more descriptive name: i.e "Shifter_base_components". This will allow to create other components packets. ie: "shifter_xxxProject_components" or "Shifter_vehicle_components".
• Also the C++ solvers should be independent in another package. Not all the shifter components need it, an also other tools (i.e: rigbits) are using it
• I was thinking to call the core "mgear_core" and keep the dist with mgear name for distribution. But I think maybe your proposal makes more sense.
• The plugin idea for shifter is great! Now I am using a env variable to add components on project configuration, but also a more smart way to find components is really good idea
• About the logos, I am thinking to contract a friend (professional designer) to do it
• rigbits is not a "modular rigging framework" is just a collection of tools and boilerplates for rigging. i.e: a function to create rivets

Can I have access to the organisation? I would like to edit it

[miquelcampos]

From @mottosso on December 3, 2017 9:0

Can I have access to the organisation?

Invite sent, it's all yours.

"Shifter_base_components"

That's absolutely fine, but, the only thing I'd suggest is that you don't call it something that make it seem like it's required or "default". If you can make it sound optional, then it'd encourage others to look at it for reference when building their own alternatives, where we could potentially be looking at an ecosystem of many different component collections.

[miquelcampos]

I see the point, you are right.

Maybe instead of use "base" something like "shifter_components_basic_pack" ?

[miquelcampos]

From @mottosso on December 3, 2017 12:57

Maybe instead of use "base" something like "shifter_components_basic_pack" ?

Yes, but remember to sell it. :) Get people excited about it, a reason to use it, to devote time into understanding and learn it. Give it a theme. shifter_components_basic_pack is probably a little dull for anything public.

• "Have you seen this new mgear extension, KungFu? It's really cool and runs great on my i7"

Now try this..

• "Have you seen this new mgear extension, shifter_basic.. I mean, component_shifter_basic_pack.. no, shifter_component_basic_pack? It's.. um.. really cool and runs on my i7"

;)

[miquelcampos]

Hahaha! you are a master!

Yep is see the point. But still, I think "Shifter" is the catchy name for the rigging system. And the components should be descriptive of the content. I don't want to mislead the people with too many fancy names, if it is meant to work only together and not solo. I this case Shifter + his components.

Unless the Pandora pack is a shifter component pack to rig only Na'vi characters ;)

For other more distinct tools, I agree should have a name that make clear distinction and easy to remember. :)

[miquelcampos]

From @RafaelVillar on December 4, 2017 0:57

Hello everyone, I subscribe and I have been listening in closely. I hope I could share my thoughts on naming. I agree with @mottosso on salesmanship of a product. There needs to be an allure to something so it can gain traction. But before one goes too far, I think simplicity is always key. Shifter is a great name, but by itself it doesnt describe what it does.

So while on the topic of naming, I would suggest something like the following.

                              |mGear_core|

components/blocks/modules - tools - presets

everything together: mGrear Framework

mGear_Core: The backbone of any extensions. Something useful to the more seasoned or technical artists if they didnt download or want the rest. Components: Its almost a universal name, concept. A component, a building block or module. Arms, legs, spines etc, come from here. Tools: So simple that I do not need to explain it. Presets: Sure a simple rig is a simple rig, but what is it at the core? Its a preset, a starting point for other people to start from.

That brings me to my other suggestion. Presets, within the mGear framework there should be a system in place for recording presets from a scene. Lets say you made a dragon... You should be able to record that dragon as a preset for later use if you should need it.

Just spit balling. And hello everyone :)

[miquelcampos]

Hi @RafaelVillar about the presets comment. It is what I call "guide templates". This concept was in Softimage Gear version, but is not yet implemented in mGear. This will come before we move to mGear 3.x structure more info: https://github.com/miquelcampos/mgear/issues/46

[miquelcampos]

TO DO:

• [x] update menu to build based on the modules loaded
• [x] update the readme and information in each new submodule
• [x] Documentation update to work with new structure
• [ ] add information in mgear_dist read me of how to clone and deploy

[miquelcampos]

here is a image for architecture description