Morphs creation (Copy/Move/Delete/Rename/Backup morphs tool done)

So, I try to help by making tools to create like morphs (or muscle if I can figure how it works).

But I had to answer few questions first :

What is the scope ?
When can the user create morphs ?
Where are tools on the panel, are they available all the time ?
When can we lunch ?

So what I think about that:

To be able to create good morphs that can be used everywhere, user must starts with the basic shape of one of the 4 big types available in MB-Lab : male, female, anime male, anime female. So, to start with something, I created a “Base female f_NO01” (file characters_config.json) that has no phenotype, no race, and is the base model that used Manuel.
I also gave to this model a basic albedo.png that looks like clay. Not very good, but well...
Then I gave it no morphs, no phenotype, nothing. So it’s the most basic model we can have.
I also created dedicated directory for this type (\data\presets\none_female_base) with no file.
For me it’s the model user must use to make his own morphs for all female models (realistic female here)

You can see the content in characters_config here: "f_no01": { "description": "Generate a base female character for creating morphs", "template_model": "MBLab_human_female", "name": "f_no01", "label": "Base female (F_NO01) (AGPL3)", ... "texture_sclera_mask" : "sclera_mask.png", "morphs_extra_file": "", "shared_morphs_file": "", "shared_morphs_extra_file": "", "bounding_boxes_file": "human_female_bbox.json", ... "vertexgroup_base_file": "human_female_vgroups_base.json", "vertexgroup_muscle_file": "human_female_vgroups_muscles.json" }

Problem is that some morphs could be very specific to a race or a phenotype, and could not work properly in all models for a reason. So I decided that user could use the model he wants, and create his own morphs while other morphs are already used. To assure that, I put the “morphs editor” in the Pre-Finalized State session (gui_status == "ACTIVE_SESSION") and put it between “random generator” and “measures”. So, for the user who chooses “F_NO01” he can create, and for the others he can use other morphs as well.

So, as MB_Lab is not designed to create bodies from scratch, but more to use what it is available, I tried to be as simple as possible, while let the user to have the most useful tool to create new morphs.

So now here are the problems I’m facing :

Let’s take the model f_ca01 in characters_config.json. Few lines are dedicated to the morphs. "name": "f_ca01", "morphs_extra_file": "", "shared_morphs_file": "human_female_morphs.json", "shared_morphs_extra_file": "human_female_morphs_extra.json",
shared_morphs_file & shared_morphs_extra_file are basic morphs shared by all phenotype from female Caucasian. (extra_files is for fantasy, basically). Here it’s simple, all females share the same files. But it could be something else.
f_ca01 allows to use morphs that are dedicated to the type (female Caucasian) and the files need to be in the proper format for its name: “f_ca01_whatever_optionalWhatever.json”
More important: under those files, a same morph is split in two, for minimum values and maximum. Each morph end with _min or _max, so the engine can construct the final morph, from 0 to 1 (0,5 is the model before morphing).
morphs_extra_file: I think that the intent was to allow user morphs shared buy a whole type?
So under the directory \data\morphs you have 2 types of files, the ones who are dedicated to a whole range of human (male, female, anime male, anime female) and the others dedicated to types, and it seems that we can add all files we want if they are properly named.

At first, the tool would have looked like this (on the panel):

Create base model → copy all vertex from actual model shown, whatever it is.
Load base model → Load an old model.
Load work-in-progress → Load a model for a morph in progress.
Reset models → reload work-in-progress model.
Morph type → a list where we can choose if the morph will be for one type in particular or for a whole range.
Body type → Male or female.
Type name → (for example) an02
Type name extra → none or a name like the pseudo of the author.
Morph name → A name for the morph.
Min or Max → the target morph is for min values or max.
Save work-in-progress → Save all actual model as a work-in-progress.
Quick save → No dialog, take the name, add _min or _max, and _000 (or _001, _002, …) and save the morph, aka the vertices that are their location changed.
Save final morph: Same thing, but we have to choose min and max files, and the file is properly saved in the dedicated directory.

But, first I wanted to have my own models, copies of the ones used by the engine, but after a while I started to think that it was not good. The engine is complex, and I should keep a coherence with it. The model the user is working on is linked to the engine, so let’s keep it that way. And the engine has already many useful tools like reset, loading, saving… So, if the user wants making morphs from base model, no problem, he can use the “reset character” button. If he wants to create while character already has morphs applied, he can use ‘import/export character”.

So now the tool looks like this :

Morph type → a list where we can choose if the morph will be for one type in particular or for a whole range.
Body type → Male or female.
Type name → (for example) an02
Type name extra → none or a name like the pseudo of the author.
Morph name → A name for the morph.
Min or Max → the target morph is for min values or max.
Load work-in-progress → Load a model for a morph in progress.
Save work in progress→ No dialog, take the name, add _min or _max, and _000 (or _001, _002, …) and save the morph, aka the vertices that are their location changed.
Save final morph: Same thing, but we have to choose min and max files, and the file is properly saved in the dedicated directory.

The workflow would look like this :

I choose for what type the morph is for. → Type name = as01
Morphs are already existing for the model, so I write my pseudo for save files → Type name extra = Teto
Morph is for this model → Morph type = dedicated
For female → Body type = female
I’m sculpting for min values → Min or Max = min.
I do my stuff → Save work-in-progress 4 or 5 times.
It’s done, same thing for Max values.
It’s done. Save the morph. The file name is automatic, I just have to choose the proper file for min and max values. The file is saved in the dedicated directory. If the file already exists, the morph is added in the file. If the name of the morph already exists, the new one replaces the old one.

I’m working on it now, and if you agree, I’d like to make a morph.py dedicated for this, so the changes in other .py files would be minimal. I’d like to talk about the work in progress here too.

So, as I'm starting to think about my new tool, I was thinking also about my combined morph tool, that I didn't tested yet, especially because I'll need it for the new tool anyway.

So I started to test it and I did 2 very fast morphs to see what's going on when we combine them and how the engine handles them.

So first, I did my 2 morphs, one overlapping the other, of course, and did stupid shape but that's the point : see how it works. I did a pike and a bump with the morph tool, and saved them under a special file, f_af01_morphs_temp.json, because I'll combine them and after that I won't need them in their original form (separated morphs). Morphs under Body. Bump maxi Pick maxi. Don't show the combinations and min values (it's the same, but inside the body) So, quit Blender, reopen it, and voilà, the morphs are available, and show the same thing as I sculpted before.

Now the juicy part. I open the combined morph tool. Choose to combine 2 morphs, use my pseudo for the name. Then, without touching the morphs, but just doing combinations min/max (so 4 combination in total), I finalize the file. Quit Blender, delete temp file, just keep teto file. Open Blender, create an African female. First good new : the combined morphs are shown as separated : After that I played with cursors and... nothing spectacular

So I don't think that the advantages of combined morphs are here. Tomorrow, I'll remake the combined file, but this time I'll sculpt combinations before saving, to see what's going on. I think that the interest of combined morph is here : to add extra details that are impossible to have when the morphs are separated. But I'll check that tomorrow. Below the 2 json files if you want to play with them. Careful : change af01 to af02 or move the file elsewhere if you don't want to use them both in the same time. Because the engine add the vectors of the same name, combined or not... Interesting behavior by the way.

morphs.zip

So I did other tests tonight, and I think I nailed the thing.

What I did basically. I put the file "temp" with my 2 simple morphs (that overlap each over), then I took "min" for bump and "max" for pike : 001 Then I sculpted the 2 a little to have this : 002 Saved the morphs file, quit Blender and reopen it after deleting "temp" file. Tried "Pick" and "Bump" in "Body", with min and max, aaaand what I sculpted shows again, as intended. Of course, if I have "Bump" in 0.5 (aka the neutral value) and Pike at max, the result is slightly different from the first test where morphs were separated : The first morph was way more sharp, but it's normal, as the engine doesn't have the first separated 2 morphs no more. So, for me, and maybe I'm wrong, but as far I investigated in order to make the tool, here how it works :

The combined morphs are combinations from simple morphs that overlap, and are in the same category.
For the most cases, it is used when morphs topic are too extreme to have good results when used separately. For example, mass (aka fat) and tone (aka muscle). If you use sliders Tone and Mass in their extremes, for morphs that are on the same body part, but one is for mass, other is for tone, the result would be ugly at worst, weird at best.
So the tool allows the user to make simple morphs, then combine them, OR sculpt directly each combination that look good, in order to have nice mix when using the addon.
You can combine up to 4 simple morphs, that means 16 different sculpts, from "min-min-min-min" to "max-max-max-max". When saved, and used by the engine, the morphs seem to be separated, but work together.
The user can sculpt the most silly morphs he wants, there's always a neutral point (0.5).

The complexity is growing... not a bad thing, just something to note for myself.

I'm tired, and the tool(s) is not finished yet (@animate1978 , I pushed the commit to save files, just in case, but please wait until it's done), but here is a teaser :

What's this ? If the addon is a success, and morphs grow, there will be a time where you'll have dozens of morph files dedicated for one model available, and sometimes user will have to manage them. Make compilations, delete some morphs, move others, and so on. So the tool is made for these operations, copy, move, delete, rename and even make backups (with date). You can do all of that from a source to a destination, create new files if needed...

My only issue for now is :

You have to create a character even if you work on another morphs files than the model uses. That's because how Python in Blender is done.
Because of that, I watch and load every file in morph directory, so that increases the beginning before user can play. Not that much, but let say 2~3s. Could be annoying if other tools are added after that and do the same thing to starting time.

Quick update : This evening I did bugfixes (there were a lot!) and improvements in the code, simplification and so on. Now the end is near, the tools should be finished tomorrow (should...).

Anyway, @animate1978 , here is something that is relevant to be in documentation I think. Where exactly ? I don't know. In "known issues" maybe ? (below the separation)

About the use of drop-down lists:

There's a limitation in Blender, due to the way it works about drop-down lists used in GUI.
Let's say that we have 2 drop-down lists, the 2nd has its values driven by the 1st :
The 1st can have 2 values, the 2nd has its values depending of the 1st:

Where is the problem ?

Imagine that at first you choice "Ethnic group" -> "m_ca01_morphs". The index of the 2nd list is 18
Now you change your mind, and decide to choice something in "Gender". You'll have this (below) and warning messages about the 2nd drop-down list.
Don't panic, it's not bug, it's because of the way drop-down lists work : When you change the content of the component, it keeps in memory the index where it was before the update. So, if after the update, the list is shorter, the index will be out of the limit, and Blender will send lot of warning, but actually it doesn't crash. It just don't know what to show. So if you have this behavior, you just have to choose an item in the list. But don't wait, because if the content of the component is used later, and if it remains empty, it could crash the engine or stop to work properly.

This is great!

I did manage to get some warnings from Blender, doing what you described above. I was trying to do it purpose to see what would happen. There is a difference between Blender 2.82 and 2.83 (alpha builds)... there is a lot more information being printed to the terminal output in 2.83, not sure why and honestly I like that because it gives a little more feedback like during creation of a character, in 2.83 there term output prints this ...

... Morph Nose_NostrilSizeX_max duplicated while loading morphs from file Morph Eyebrows_Ridge_max duplicated while loading morphs from file Morph Eyebrows_Angle_max duplicated while loading morphs from file Morph Chin_Tone_max duplicated while loading morphs from file Morph Nose_NostrilDiam_min duplicated while loading morphs from file Morph Head_Flat_min duplicated while loading morphs from file Morph Chin_SizeZ_min duplicated while loading morphs from file Morph Nose_BasePosZ_max duplicated while loading morphs from file Morph Cheeks_ZygomPosZ_max duplicated while loading morphs from file Morph Nose_WingBump_min duplicated while loading morphs from file Morph Chin_SizeZ_max duplicated while loading morphs from file Morph Jaw_Angle2_max duplicated while loading morphs from file Morph Eyebrows_PosZ_min duplicated while loading morphs from file ...

Not sure if the duplication is intended. I think I have seen this before when creating the Anime characters. Anyways I will be working on the docs for these tools very soon since you seem to be close to completing these.

Nope, it's not that. I figured that weeks ago, don't worry. When the engine loads morphs, it loads many files where the morphs are. But if the same morph is in more than one file, it writes that. They are just warnings, the engines accepts them anyway, and replaces the old one by the new one. And typically, the anime model was clearly not finished, because many morphs are duplicated for a reason.

The Copy/Move/Delete/Rename/Backup morphs tool is done. I tested it, it looks functional and I didn't see bugs (I mean, after I fixed them ^^ )

@animate1978 and @everyone don't hesitate to test it (make backups before !) and tell me if there are bugs or things you don't like or should be improved.

I won't explain again how it works, the GUI didn't change much since my last explanations. But there's an addendum :

Tool limitations

When you copy/move morphs from a file to another, if the other file is a new file, it will not show up in drop-down list about input files. You have to quit Blender
When you rename, move or delete morphs, you have to reset/rescan first, then change input file and comeback to your file you're working on. It's not intended, but I didn't figure out how to change that yet. A workaround is to do "new general file" in Blender and create a new character.
If you rename a morph (one each time only), if you reset/rescan, it'll disappear from the items in the category. It's normal and due to the way Blender works. Short story, some variables must be created in the beginning of the session, and when the session is started, you can't change them anymore (or I didn't find how). All morphs' names are created at this moment and can't be changed after. So new ones created by the tool can't show up for this reason.
For the moment you can't create files for a whole gender. It's intended, as most of morphs inside are essentials for many objects in a model. Groups, bboxes, and so on.

Warnings

No undo with this tool. If you're not sure, make backups.
Reminder : There are 2 types of morphs : The simple, and the combined (combination of 2~4 simple ones). When you rename a combined morphs, be aware to rename it in the same way. Example : TorsoMass-Tone... wil becomes TorsoWhatever-YourChoice... Rename it TorsoWrongChoice... and engine will have bad behavior.
Also, don't forget that many morphs are essential for the engine, as they are used for other things like groups (used in muscles and measures), bboxes and so on. So use this tool wisely, and if you're not the creator of the base model, use it only when you do user-files management, fix duplicate names from different sources and so on. Don't touch any morphs if it's not necessary. There's a spelling mistake ? It doesn't matter.

animate1978 / MB-Lab