HLSL?

[Hogarth-MMD]

I don't imagine that Blender is going to be compatible with HLSL any time soon. But Blender materials do have SSS and ramp shaders. One of the biggest advantages of rendering with MMD instead of with Blender is the extreme user-friendliness of MME effects such as dAdultShader and HyperShader, as well as toon textures, which can be used to render the realistic reddening of shadows of human skin. If possible, I would like to be able to do all of my rendering of MMD models in Blender. Can someone (umm, Nathan for example :-) ) please explain how these effects work? Why don't they create a reddening of hair, clothes and eyes, for example? It seems like they can automatically detect skin and not skin.

[nathanvasil]

Different effects work differently. The code for all of them is available (MME compiles on load.) You can check it out and start tracing the code to see exactly how each effect works. That includes the default shader, which is a good place to start. If you have trouble understanding some part of a HLSL shader, I'd be happy to help. It can be confusing, but it should be easier since you have some coding under your belt.

As far as toons specifically, the default MMD shader handles them differently depending on the rendering mode. But basically, it's just a multiply layer on the ambient lighting. (MMD doesn't use the same style of rendering as Blender, though.) Skin gets redder not because MMD magically knows that it's skin, but because it has a red toon. If a different material had a red toon, it would get redder too. If it had a blue toon, it would get bluer instead. If it didn't have a toon, it would get blacker. If you put a red toon on a totally blue model, it would get blacker too (because 0,1,0 times 1,0,0 is 0,0,0.)

[Hogarth-MMD]

I did try opening dAdultShader and HyperShader files in a text editor, but I didn't understand how they work. That's why I am asking for help with it. It would be useful if we could re-create or approximate these effects in Blender.

[nathanvasil]

Well, you have help. What's the first thing you didn't understand?

My advice is, when you're starting out reading effects, ignore everything except the main pixel shader, which is probably going to be called "BufferShadow_PS"-- buffershadow means that it's in shadowing mode, PS means it's a pixel shader, which is responsible for picking colors for each pixel.

Most of the variable names should give a clue as to their role, but you can always search through the code for where the variable is declared, and possibly make test models to create controlled experiments. You can also do a sort of print debugging, just by typing in something like,

float4 debug = MaterialToon; return debug;

Then whatever color is in MaterialToon will get dumped to the pixel right then and there, and you can see what value that variable holds.

Intrinsic functions are documented on the web (if not particularly well.)

I don't know enough about Blender to code nodes or whatever in it yet. I could describe things, but the reality is that the best description is the code for each effect.

[Hogarth-MMD]

I suppose that the first part of the puzzle may be: How are these skin shaders defining skin and not skin? The issue of defining skin tone would enter into it in 2 aspects: 1) How does the shader know which materials are skin and which materials (or which pixels) are not skin. and 2) How does the shader control the rendered result to force the rendered result to be within the defined parameters for human skin rendering? Then the next question is how can we implement this using the Blender compositor and/or python scripting? I am not even sure if Blender is able to do this.

In this file of dAdultShader, I found these parameters listed, which seem to be Hue, Saturation, Value parameters for possible skin tone(?):

_dSkinSASCommon.fxsub

ifndef H_MIN

#define H_MIN 0.80 // 色相の下限(0～1)

endif

ifndef H_MAX

#define H_MAX 0.20 // 色相の上限(0～1)

endif

ifndef S_MIN

#define S_MIN 0.03 // 彩度の下限(0～1)

endif

ifndef S_MAX

#define S_MAX 0.40 // 彩度の上限(0～1)

endif

ifndef V_MIN

#define V_MIN 0.70 // 明度の下限(0～1)

endif

ifndef V_MAX

#define V_MAX 2.00 // 明度の上限(0～1)

endif

[nathanvasil]

Shaders don't know whether a material is skin, unless it knows because you only put it on materials you mean to be skin. It knows model material values, it knows textures, it can know a few other things if you want it to. Shaders do not "force renders into defined parameters," not whatsoever. The art of creating textures and materials is, in a way, the art of wrestling shaders into showing what you want.

Do an experiment. Give something metal a skin's textures and material settings. Like a sword or a gun or something. Load it up. Does your shader "know" whether it's skin or not? Give your skin the weapon's texture, the weapon's material settings. Does your shader know what is skin?

As fas as the file you mentioned, those do look like HSV values. If you want to know how and when they're used, you can do a search for the usage. I would take a look myself, but my version of DAS must be different than yours; so is the version I just downloaded to see if I could find what you're talking about. if you send me a copy of what you're working from, I could tell you more specifically.

[Hogarth-MMD]

Maybe a shader can detect if a pixel (not a material) is within defined parameters.

[nathanvasil]

Yes, there are ways to do that, but I don't think the shaders you're looking at do that.

[Hogarth-MMD]

dAdultShader https://bowlroll.net/file/65211

dAdultShader (older version) https://sta.sh/013x2sn1akuk

Shader Hyper http://www.mediafire.com/download/mg1t8ibxzhvsqbz/Shader+Hyper.zip

Shader Hyper has a folder in it which needs to be renamed to ascii text. MME does not like non-ascii file names or folder names. MainSh.fx is the primary human skin shader file in Shader Hyper.

[nathanvasil]

Well, what do you know. I looked at your dAdultShader (only.) Some of those do try to detect skin, it looks like. 肌自動認識版フルモード・ミラー設定例 and 肌自動認識版. The others do not. (I searched for #USE_AUTO_DETECT which is the preprocessor directive which tells the individual shaders to try to autodetect skin.)

It does this like it sounds, from HSV values, of material settings (ambient+diffuse) times tex values. If that lays within min and max H, S, and V, it treats it as skin. When it treats it as skin, it uses a particular tiling normal map. possibly different sphere maps, and possibly a particular tiling specular map.

I would be surprised if this works really well, but you be the judge. In my experience, you need to be careful (model-specific) about how you scale tiling maps. You also risk capturing materials you didn't intend. For instance, imagine something bronze: it could very easily get caught within those HSV ranges and look inappropriate because of it. But then that's probably why this autodetection feature was used on only two of the shaders in there.

This is a complicated collection of shaders, and you might be better off starting with something simpler to learn how it all works together.

[Hogarth-MMD]

As far as I know, Blender can already do everything that MME can do. There wouldn't be any benefit to bringing HLSL or MME into Blender, because Blender can already do it all. Blender is powerful software. The only exception to that, that I know of, is just the human skin shaders.

[Hogarth-MMD]

Here is a tidbit of information from the Blender User Manual about the scale value of SSS. Based on this information, 1 MMD unit = 8 centimeters, the SSS scale value for an imported MMD model should be 1/80.

https://docs.blender.org/manual/en/dev/render/blender_render/materials/properties/subsurface_scattering.html

Scale The scale of your object, in Blender units, across which you want the scattering effect to take place. Scale of 1.0 means 1 Blender unit equals 1 millimeter, scale of 0.001 means 1 Blender unit equals 1 meter. If you want to work out what scale value to use in your scene, just use the formula: (size in Blender units)/(real world size in millimeters)=scale.

[Hogarth-MMD]

@nathanvasil This is my personal favorite shader from the dAdultShader collection of shaders:
dAdultShaderNEvo.fx If you want to analyze only one shader, maybe you can analyze this one. Or if you want to pass the buck on this, then, of course, that is up to you.

[nathanvasil]

If there's something in particular you want to know, just ask, and I'll take a look.

[Hogarth-MMD]

Here is my latest python script offering. It interprets every material as being human skin and adds subsurface scattering to it.

import bpy

for o in bpy.context.scene.objects:
    if o.type == 'MESH':
        if hasattr(o, 'mmd_type'):
            if o.mmd_type != 'RIGID_BODY':
                if o.mmd_type != 'JOINT':
                    for m in o.material_slots:
                        m.material.subsurface_scattering.use = True
                        m.material.subsurface_scattering.scale = 1/80
                        m.material.subsurface_scattering.radius = 4.821, 1.694, 1.090
                        m.material.subsurface_scattering.color = 0.749, 0.571, 0.467
``

[Hogarth-MMD]

As far as HLSL and I are concerned, you can just assume that I don't understand any of it.

[Hogarth-MMD]

I've been mucking around trying to get a decent quality of render in Blender. But the quality of render than I can get in Blender is VERY inferior to the quality of render that I can get in MMD. In any rendering there is a tendency for highlights and shadows to cause too much loss of saturation of colors and I don't know how to solve that problem. In MMD it's easily solved.

[Hogarth-MMD]

Under the World tab of the Properties window is an option called Ambient Occlusion. Enabling Ambient Occlusion improves the quality of the render by a large amount.

[Hogarth-MMD]

Here is some information about the Blend modes which are used in Blender and in GIMP (including the exact mathematical formulas).

E = M + I

means, “ For each pixel in the upper (Mask)and lower (Image) layer, add each of the corresponding color components together to form the E resulting pixel's color. ” Pixel color components must always be between 0 and 255.

https://docs.gimp.org/en/gimp-concepts-layer-modes.

https://en.wikipedia.org/wiki/Blend_modes

http://www.paintshopprotutorials.co.uk/html/blend_modes_gimp.html

Multiply Screen Overlay Hard Light Soft Light Dodge and burn Divide Addition Subtract Difference Darken Only Lighten Only Hue Saturation Value Color

[Hogarth-MMD]

It would be good to have a complete listing of mathematical formulas which are used in Blender and in MikuMikuDance. If someone has some of this information, can you please post it or post a link to it?

[nathanvasil]

MMD formulas are just HLSL intrinsics. You'll see a lot of +, -, *, /, %. You'll see a lot of dot products of vectors being used, as well as max, min, floor, ceil, abs, sign. You can also use log, n^x, sine, whatever you want. While I'm sure there are some things you can't do, I've never run into anything. I believe that all of the modes you listed can be created with those tools. I understand there's even some NAN support in HLSL.

[Hogarth-MMD]

But what about the mathematical formulas which are used to render MMD specular, ambient, diffuse, texture, toon, sphere?

[Hogarth-MMD]

MMD ambient is a bit complicated. It can behave like a switch between material and texture. If the ambient color is set to 1.0, 1.0, 1.0 there is no rendering of diffuse or specular material values and all of the rendering comes from the textures.

[Hogarth-MMD]

@nathanvasil I have the information which you previously posted about the rendering of toon textures, thank you. Don't worry, you don't need to repeat yourself about that.

[nathanvasil]

But what about the mathematical formulas which are used to render MMD specular, ambient, diffuse, texture, toon, sphere?

Lots of different ways to do specular, but MMD's default is simple. Color += specularColor * pow(clamp(dot(HalfVector, NormalVector)), specularPower). That's Blinn specular. HalfVector is the vector laying between the pixel-to-eye vector and the pixel-to-light vector.

MMD handles ambient and diffuse in a strange way. Roughly speaking, color = ((((shadowBufferLookup-clamp(dot(LightVector, NormalVector))toon) (LightAmbientMaterialDiffuse)+MaterialAmbient))texColor. But that's only roughly speaking, and I'm sure my parentheses don't match up, and I probably screwed it up anyways. (Be careful when reading .fx files, MMD doesn't name its parameters the same as PMXE does; MaterialAmbient is diffuse, MaterialEmmisive is ambient, LightAmbient is scene light, MaterialDiffuse and LightDiffuse return black or white or something without any regard for material values.)

Sphere is also simple. It's a lookup based on the view-space normal vector x and y coordinates, which are used as a lookup value on a texture. For add spheres, the texture lookup is added to color; for mult spheres, the texture lookup is multipled into color. Regardless, alpha channel is multiplied, never added.

If the ambient color is set to 1.0, 1.0, 1.0 there is no rendering of diffuse or specular material values and all of the rendering comes from the textures.

Check again. MMD ambient IS weird, because it's doing some weird NPR stuff that's philosophically grounded in subtractive color rather than additive color. But make a material with a white ambient, a white diffuse, a white specular, and a black texture. You'll see the specular. In fact, all you'll see is the specular.

[Hogarth-MMD]

@nathanvasil wrote: That includes the default shader, which is a good place to start.

Please explain to me lucidly where and how I can find this default shader.

[Hogarth-MMD]

NPR Umm, Google seems to think that this is National Public Radio. LOL. Please define these initials.

[nathanvasil]

Here's my copy: http://www.mediafire.com/file/678xx9w9k7a8by3/base.fx . Easier than tracking down wherever I scored it from.

NPR is "non-photo-realistic," referring to cartoon-style rendering.

[Hogarth-MMD]

Hi @nathanvasil , Thank you for your helpful information. Do you have any idea what SKII is referring to? When I first read this, I thought that maybe HLSL has a built-in human skin shader function called SKII. But maybe it is just a simple assignment of an integer variable, and this theory shows how ignorant I am about HLSL.

Line 36: #define SKII1  1500
Line 37: #define SKII2  8000
Line 269:           comp=1-saturate(max(IN.ZCalcTex.z-tex2D(DefSampler,TransTexCoord).r , 0.0f)*SKII2*TransTexCoord.y-0.3f);
Line 271:           comp=1-saturate(max(IN.ZCalcTex.z-tex2D(DefSampler,TransTexCoord).r , 0.0f)*SKII1-0.3f);

[nathanvasil]

I haven't looked at those very carefully before. Those lines have to do with examining the shadow buffer, which is a depth map from the perspective of the light.

If an object is not in shadow, its depth is equal to the value from the depth map, so ZCalcTex.z - shadowbuffer.r == 0. If an object is in shadow, its depth is greater than the value from the depth map (in roughly world units, although there's precision loss.) Now, presumably, if something was around 1/8000th of a unit behind something else, it wouldn't get fully shadowed. This may be a fudge to deal with precision loss, but the presence of the transtex.y in the equation is weird-- it's basically saying, don't shadow them if they're getting a low V coordinate in the shadow buffer, when the actual orientation of the shadow buffer (light doesn't care about roll) is arbitrary. I might play around with it bit more.

Edit: Played with it some. It is interesting how their shadow buffer works, and I should read more on shadow buffer techniques; I've used them for some effects and may have done some things wrong. But to be clear, it has nothing to do with skin.

[Hogarth-MMD]

Is there any software which can take an MME .fx file and automatically translate it into a visual node setup, similar to the node setups of the Blender node editor? This could be extremely helpful to implementing MME effects in Blender.

[nathanvasil]

Not that I know of. The process is normally the reverse of that, generating shaders from node setups.

[Hogarth-MMD]

Theoretically it is possible to clone the MMD rendering engine into Blender, using the Blender node editor and Python scripting. In the very recent past, I had no idea that this is possible.

[Hogarth-MMD]

What is a "dirt shader"?

Commonly Ignored Feature #6: Dirty Vertex Colours http://adaptivesamples.com/2013/08/07/commonly-ignored-feature-6-dirty-vertex-colours/

"It’s a sort of quick, and fake ambient occlusion that you bake into the vertex colours of a mesh. It’s a little different from AO as it also gives some highlights on sharp edges."

Blender Manual Render » Blender Render Engine » Render Settings » Render Passes

https://docs.blender.org/manual/en/dev/render/blender_render/settings/passes.html "Compositing Ambient Occlusion

AO is a geometry-based dirt shader, making corners darker."