toon textures?

[Hogarth-MMD]

Hey there Powroupi, How are you doing? Are you making any progress with toon textures?

It is possible to convert an MMD toon texture for Blender. Blender uses spherical gradient textures for mapping onto reflection (or normal) co-ordinates. You can convert the vertical gradient of a toon texture to a spherical gradient using the polar co-ordinates filter in GIMP or Photoshop. After importing the texture in Blender, change the X and Y sizes of the texture to 0.5. (Use the multiply blend mode.)

A toon texture could theoretically be converted to a ramp shader by Python scripting. MMD toon textures affect the coloring of shadows.

The main reason why I still prefer MMD over Blender for rendering (apart from speed and simplicity) is the lack of Blender support for toon textures.

[powroupi]

Hi, @Hogarth-MMD, I'm good, thank you. :smile: Currently, I'm focusing on refining some UI panels and Operators, and the stuff of bones' local axes. :)

About toon textures, I think it is possible to use it and customize shading in material's node editor. But if blender can support it directly, it would be best. :smile: (for example, assigning a texture to a toon/ramp shader)

[Hogarth-MMD]

Hi @powroupi, I'm glad that you are doing good! 😄 Can you please be more specific about how this might be done using the node editor? Are you imagining that Cycles would be needed for this? I dislike Cycles because of the slow rendering speed. I'm not aware of anything in the node editor which can use the colors of individual image pixels as input(?).

[powroupi]

You can control the vector input of a Texture Node, just like controlling UV, but I'm not sure how MMD calculating the UV of a toon texture. (the V may be light vector dot normal vector...) :cry: Similar technique may also work in Cycles. :)

You might take a look at https://blendernpr.org/ or google search "blender toon" for further informations. :smile:

[nathanvasil]

I've never done Blender's nodes, but I know the math for the toons.

A lot of effects use the same version of the "default" MMD shader. This default shader used by effects is different from the actual default MMD shader in one particular circumstance, but it's almost impossible to tell unless you compare them side by side. One place where you can find this shader is on the Working Floor effect (where obviously you only want to manipulate positions for a mirror effect, but maintain the default look). The default shader uses some unusual terminology sometimes, and the structure's a little ugly. If you don't feel like tracing through the code, I can explain how toons are handled.... Well after writing a lot, doing it with math and pseudo-code seemed more clear than doing it with words.

In shadow mode 2, only a single pixel of the toon texture is used:

LN = lightVector dot surfaceNormal toonColor = toonTex(0,1) //get the lower left corner of the toon texture light = min (clamp(LN3), lightMap) lightColor = (texture (ambient+diffuse))+(specular Blinn-Phong(reflection))+(addSphere) shadowColor = (toonColor texture (ambient+diffuse))+(addSphere) pixelColor = clamp(shadowColor + ((lightColor - shadowColor) light))

In shadow mode 1, the left margin of the toon texture is used:

LN = lightVector dot surfaceNormal toonV = clamp((0.5 - LN) 0.5) toonColor = toonTex (0, toonV) //look along the left margin of the toon texture based on L dot N pixelColor = clamp((toonColor texture (ambient+diffuse))+(specular Blinn-Phong(reflection))+(addSphere))

Models without a toon texture go through a different rendering path, so you've basically got four rendering paths in the default shader, which is one of the reasons it's a little tricky to trace.

All output values are clamped to a range of 0-1, so if you output RGB 0.5, 2.0, 0.5, your output pixel is 0.5, 1.0, 0.5. (I gave you some bad information a long time ago when I told you that it was clamped in a way to maintain hue-- it's not, hue can suffer.)

[powroupi]

Thank you, @nathanvasil , I can understand your pseudo-code. :smile: I will try it in Blender's nodes when I have time. :)

[Hogarth-MMD]

Hi @powroupi, I recommend that you should add a toon custom property to every toon texture image of every material's toon texture, for every MMD model which is imported into Blender. The way that MMD_tools is programmed now, the toon texture custom property is in the material panel, which makes it a headache for anyone to program the correct display of toon textures. Any python function for toon textures first needs to be able to conveniently identify which texture images are toon textures.

[powroupi]

Sorry @Hogarth-MMD , I don't understand your requirement. Do you need a python function to identify toon textures, or automatically setup "mmd_material.toon_texture" just like the way of "Texture:" and "Sphere Texture" in MMD Texture panel? Or want to move MMD Texture panel to Texture tab of Properties window instead of Material tab?

[Hogarth-MMD]

Hi @powroupi, I'm thinking of a solution that involves converting a toon texture image to a texture ramp shader with a spherical blend procedural texture mapped onto reflection co-ordinates. So a "toon" custom property (in the textures panel) for every toon texture would be convenient.

[Hogarth-MMD]

Maybe nothing complicated is really needed with toon textures. Maybe we don't need the node editor. Maybe we don't need any spherical blend texture or mapping onto reflection or normal co-ordinates. Maybe all we need is just converting the toon texture image into a texture color ramp and setting the blend mode to multiply. But I'm pretty sure that converting to a color ramp and using the multiply blend mode are essential steps.

[Hogarth-MMD]

Here is the Blender manual page which explains color ramps: https://docs.blender.org/manual/de/dev/render/blender_render/materials/properties/ramps.html

So which of these 4 do we want?:

Shader
Energy
Normal
Result

We also need the toon color ramp to blend with the diffuse texture in the correct way. And we have 3 locations to choose from for a color ramp: material panel, texture panel, node editor. In the texture panel, I think that a mapping needs to be chosen(?), but which one?

[Hogarth-MMD]

Okay, I think I know how it should be done. The toon texture should be converted into a ramp shader for the diffuse texture with the multiply blend mode. Then the toon texture itself should be disabled. It won't have any mapping onto any co-ordinates at all other than the UV co-ordinates of the diffuse texture. In the unlikely case that a material has a toon texture, but no diffuse texture, the toon texture should be converted into a ramp shader for the diffuse color of the material (with the multiply blend mode). No node editor, no blend procedural texture, and no mapping onto any special co-ordinates! :-)

[powroupi]

Well, I'm satisfied with current result of "Shadeless" shading, not necessary to convert anything. If you need a "toon" custom property, you can modify mmd_tools/core/material.py and try anything you like... :smile: Change it to:

def update_toon_texture(self):
    mmd_mat = self.__material.mmd_material
    if mmd_mat.is_shared_toon_texture:
        shared_toon_folder = addon_preferences('shared_toon_folder', '')
        toon_path = os.path.join(shared_toon_folder, 'toon%02d.bmp'%(mmd_mat.shared_toon_texture+1))
        # self.create_toon_texture(bpy.path.resolve_ncase(path=toon_path))
        slot = self.create_toon_texture(bpy.path.resolve_ncase(path=toon_path))
        slot.texture['texture_type'] = 'SHARED_TOON'
    elif mmd_mat.toon_texture != '':
        # self.create_toon_texture(mmd_mat.toon_texture)
        slot = self.create_toon_texture(mmd_mat.toon_texture)
        slot.texture['texture_type'] = 'TOON'
    else:
        self.remove_toon_texture()

[Hogarth-MMD]

I ran a few tests with texture ramps and I got very disappointing results. :-( The color ramp completely overwrites the texture image instead of blending with it. And, strangely, only the top color of the texture color ramp would render. None of the lower colors would render, which makes no sense. :-(

[Hogarth-MMD]

The problem of only the top color of the color ramp being rendered was easily solved. I just needed to change the texture type to a procedural blend texture. So it seems that my earlier idea was better than my later idea.

[Hogarth-MMD]

import bpy

for o in bpy.context.scene.objects:
    if o.type == 'MESH':
        if o.mmd_type != 'RIGID_BODY':
            if o.data.materials is not None:
                for m in o.data.materials:
                    if m.texture_slots is not None:
                        for t in range(len(m.texture_slots)):
                            if m.texture_slots[t] is not None:
                                texture_name = m.texture_slots[t].texture.name
                                if t == 0:
                                    bpy.data.textures[texture_name]["mmd_texture_type"] = "DIFFUSE"
                                if t == 1:
                                    bpy.data.textures[texture_name]["mmd_texture_type"] = "TOON"
                                if t == 2:
                                    bpy.data.textures[texture_name]["mmd_texture_type"] = "SPHERE"

[Hogarth-MMD]

The above code is just iterating through the textures and assigning a custom property to each texture. When mmd_tools imports a model, the toon texture is always in the second texture slot.

[Hogarth-MMD]

" I'm satisfied with current result of "Shadeless" shading, not necessary to convert anything."

This statement does not seem to make any logical sense. "Shadeless" means that no lighting or shadows are calculated for a material. Toon textures influence the coloring of shadows. These are completely different from each other.

[powroupi]

@Hogarth-MMD Yes, you are right, sorry for my bad habits on programming. I think "Simple Shading" may make more sense. :smiley: If you have any suggestion to improve UI labels/discriptions, please let me know. You might make a wiki document for me (just like @nathanvasil did interface-notes), or pull requests if you know how to do. Thank you. :smile:

[powroupi]

@Hogarth-MMD ...Actually, I think "Shadeless" is fine because it mainly just enable "use_shadeless" of each material. And if "Shadeless" make no sense, "GLSL" may not make sense as well. So, I'd like to keep that as it is for now. :)

[Hogarth-MMD]

@powroupi When you say that "Shadeless" is fine, do you mean that "Shadeless" is the correct translation for the mmd_tools interface? Or do you mean that clicking on the "Shadeless" button of the MMD tools panel gives a completely adequate rendering of toon textures?

[powroupi]

@Hogarth-MMD I mean the "Shadeless" button, and I'm satisfied with the result of clicking on the "Shadeless" button (so I didn't mean shadeless, sorry, ...I guess there is a misunderstanding). :laughing:

EDIT: About -"Shadeless" is fine-, I mean the "Shadeless" button use label "Shadeless" is fine. :)

[Hogarth-MMD]

Quoting @nathanvasil : In shadow mode 1, the left margin of the toon texture is used:

LN = lightVector dot surfaceNormal toonV = clamp((0.5 - LN) 0.5) toonColor = toonTex (0, toonV) //look along the left margin of the toon texture based on L dot N pixelColor = clamp((toonColor texture (ambient+diffuse))+(specular Blinn-Phong(reflection))+(addSphere))

What about the influence of the lamp brightness? I am not seeing the lamp brightness in this formula?

[Hogarth-MMD]

Okay, this is shadow rendering, so I guess that there is no influence of lamp brightness and I asked a stupid question.

[Hogarth-MMD]

pixelColor = clamp((toonColor texture (ambient+diffuse))+(specular * Blinn-Phong(reflection))+(addSphere))

This is the complete MMD rendering formula for all 3 material attributes and all 3 textures, is it not? Except for lamp brightness and lamp color? How does lamp brightness and lamp color enter into it?

[nathanvasil]

Okay, this is shadow rendering, so I guess that there is no influence of lamp brightness and I asked a stupid question.

No, not a stupid question. But toon is multiplied into the shadowcolor. MMD then lerps between shadowcolor and color to determine a final color value. Shadowcolor doesn't care about lighting (notice that colored light affects both shadowed and lit areas), but the lerp between shadow color and color is determined by shadowing and LdotN.

pixelColor = clamp((toonColor texture (ambient+diffuse))+(specular * Blinn-Phong(reflection))+(addSphere))

No, not quite. There's a reason there are all those lines in the default shader :) Not too many though, I count fewer than 50.

But roughly speaking, to answer about light intensity, the entirety of light brightness factors into both the color in shadow and color in light. It shouldn't precisely be understood as the intensity of the directional light or of the ambient light. Sorry, it doesn't make a lot of sense.

The primary driver here is AmbientColor, which is defined as (MaterialAmbient (which is PMXE diffuse) * Light) + MaterialEmmisive (which is PMXE ambient.) Remember, default light is 154/255. (And I just learned today that light tops out at 254/255. Huh. Error by 1?) To this, the toon color is multiplied in on the basis of shadowing (multiplied only in shadows) and NdotL (multiplied in where NdotL is small); specular is added in, on the basis of shadowing (not added in shadows). Add spheres and mult spheres are applied without respect to shadowing.

[Hogarth-MMD]

"There's a reason there are all those lines in the default shader :) Not too many though, I count fewer than 50." Notepad++ counts 395 lines in the base.fx which you uploaded for me.

[nathanvasil]

Just read buffershadow_ps. Trace any undeclared variables.

[Hogarth-MMD]

Quoting from base.fx about the influence of light on render result:

Line 4: light = min (clamp(LN*3), lightMap)
Line 4: light = min (clamp(LN*3), lightMap)
Line 7: pixelColor = clamp(shadowColor + ((lightColor - shadowColor) * light))
Line 7: pixelColor = clamp(shadowColor + ((lightColor - shadowColor) * light))

(I don't particularly understand this stuff.)

[nathanvasil]

Those lines aren't in my copy of base.fx. Are you looking at the right file?

edit: but LN refers to dot product of light vector and normal vector, light is probably a temporary value used in a ps, if you're quoting from an effect file. min() returns the lowest of two values. lightmap probably is a read of the shadowbuffer. clamp() isn't a HLSL function but an OGL function; it clamps the values to a range of 0,1; the HLSL equivalent is saturate(). lightColor probably refers to the float3 light value defined by MMD.

Edit2: except maybe lightcolor refers to the pixel value if it were fully lit, since shadowcolor usually refers to the pixel color when it is fully unlit.

[Hogarth-MMD]

I apologize for my idiot error. Here are the search results for the word "light" in base.fx. So which of these lines has the answer to my question about the influence of light on rendering?:

Line 8: float4x4 LightWorldViewProjMatrix   : WORLDVIEWPROJECTION < string Object = "Light"; >;
Line 8: float4x4 LightWorldViewProjMatrix   : WORLDVIEWPROJECTION < string Object = "Light"; >;
Line 10: float3 LightDirection              : DIRECTION < string Object = "Light"; >;
Line 10: float3 LightDirection              : DIRECTION < string Object = "Light"; >;
Line 19: float3 LightAmbient                    : AMBIENT   < string Object = "Light"; >;
Line 19: float3 LightAmbient                    : AMBIENT   < string Object = "Light"; >;
Line 20: float3 LightSpecular               : SPECULAR  < string Object = "Light"; >;
Line 20: float3 LightSpecular               : SPECULAR  < string Object = "Light"; >;
Line 21: float3 LightDiffuse                    : DIFFUSE   < string Object = "Light"; >;       //always returns black, or white, I forget
Line 21: float3 LightDiffuse                    : DIFFUSE   < string Object = "Light"; >;       //always returns black, or white, I forget
Line 23: static float4 DiffuseColor  = MaterialDiffuse * float4(LightDiffuse, 1);
Line 24: static float3 AmbientColor  = MaterialAmbient  * LightAmbient + MaterialEmmisive;
Line 25: static float3 SpecularColor = MaterialSpecular * LightSpecular;
Line 119:   Out.Pos = mul (Pos, LightWorldViewProjMatrix);
Line 154:       Out.Color.rgb += max(0,dot( Out.Normal, -LightDirection )) * DiffuseColor.rgb;
Line 168:   float3 HalfVector = normalize( normalize(Out.Eye) + -LightDirection );
Line 186:         float LightNormal = dot( IN.Normal, -LightDirection );
Line 186:         float LightNormal = dot( IN.Normal, -LightDirection );
Line 187:         Color *= tex2D(ObjToonSampler, float2(0, 0.5 - LightNormal * 0.5) );
Line 210:     Out.ZCalcTex = mul( Pos, LightWorldViewProjMatrix );
Line 217:         Out.Color.rgb += max(0, dot( Out.Normal, -LightDirection )) * DiffuseColor.rgb;
Line 236:   float3 HalfVector = normalize( normalize(IN.Eye) + -LightDirection );
Line 274:           comp = min(saturate(dot(IN.Normal,-LightDirection)*Toon),comp);

[nathanvasil]

Line24 is the only real important issue involving light color. AmbientColor is both base color in light and base color in shadow.

Other stuff is peripheral (line 19 sets up line 24), red herring probably designed to waste your time (no such thing as LightDiffuse in line 23), or having to do with the direction of light rather than the intensity of light. Line 119 transforms a vertex for the perspective of the light in order to create a shadow buffer, but that shadow buffer is ultimately read only ever as 0 or 1. Line 210 is to setup the main PS to read the shadow buffer.

[Hogarth-MMD]

Thank you for your replies, @nathanvasil Does this make logical sense? Multiplying one color by another color causes darkening, so it seems to imply that light has a darkening effect!!!? :

Line 23: static float4 DiffuseColor = MaterialDiffuse float4(LightDiffuse, 1); Line 24: static float3 AmbientColor = MaterialAmbient LightAmbient + MaterialEmmisive; Line 25: static float3 SpecularColor = MaterialSpecular * LightSpecular;

[nathanvasil]

Sure, that's a way to look at it. But the other way to look at it is if there's no light, you're multiplying by zero, and if there's full light, you're multiplying by 1, so the more light, the less black you're multiplying in.

In general, you shouldn't really look at the base thing as being texture, that we're modifying with light. Instead think of the thing that we're seeing as the light itself, reflected off of surfaces. And you're multiplying materials and texture in, because those materials and textures don't reflect 100% of the light that hits them. This is actually one of the few ways in which 3D rendering is relatively physically accurate.

And you'll find, if you dig, that Blender rendering is the same, at least in this respect. You have a light value that gets modulated along its path to the camera, bouncing off surfaces and losing bits of its value each time-- getting multiplied by texture, by translucency, by material diffuse, etc.

[Hogarth-MMD]

Thank you for that very logical and enlightening answer. Your answer sheds light on this for me. Very helpful. (Puns intended. )

[Hogarth-MMD]

I have just conducted many tests attempting to implement toon textures in Blender. The results of mapping onto reflection or normal co-ordinates are total garbage. The ideas which I previously posted in this issue are total garbage and people can just ignore these ideas. I tried diffuse material ramp shader, texture ramp shader, linear gradient toon image with no ramp shader, spherical gradient toon image with no ramp shader. I tried powroupi's idea of using the GLSL lighting of mmd_tools. This does give an extremely crude and inadequate approximation of toon textures, but the orientation of the gradient is limited to being from top to bottom of the mesh. An object which is lighted by a GLSL light (=hemi lamp) does not cast any shadow. Results of shadeless are total garbage. A shadeless material cannot receive any shadow, but the purpose of toon textures is the coloring of shadows. Maybe toon textures can be implemented in Blender's node editor but I have not yet tested this possibility.

[powroupi]

You may try @vipper36's more_imitating branch which implemented in Blender's node editor. It shows the possibility of toon shading and toon edge. :smile:

EDIT: Note you cannot use multiple mmd_tools at the same time, so rename or remove the old one (or move to other folder) before using another one. :)

[Hogarth-MMD]

https://github.com/powroupi/blender_mmd_tools/network/members

Wow, many forks of mmd_tools! I did not know about this.

[Hogarth-MMD]

"try @vipper36's more_imitating branch"

@powroupi This branch is a "stale" branch. It has not been updated for 2 years. There is no current development happening with vipper36.

[powroupi]

@Hogarth-MMD yeah, it already sit there for 2 years. But the techniques of toon shading and toon edge are almost not changed for Blender internal renderer. :)

[Hogarth-MMD]

@powroupi This version of mmd_tools does not work, does it? I found a button to "create MMD scene". I found a menu entry for an "MMD render engine". I found some code for shaders. I don't see any special node editor setup. Are you able to tell me how to make it work?

[powroupi]

@Hogarth-MMD Hit Create MMD scene button, then import a MMD model, change render engine to Blender Renderer, take a look the material notes setup of the mesh object. Code rederence: core.material :)

[Hogarth-MMD]

I tried to render still images with the add-on of vipper36 but I cannot see any sign that he succeeded to implement toon textures or fake SSS. To me, it looks like these were unfinished tasks for him. I have a problem studying his node setups because some of them have 25 nodes all in exactly the same place and I don't know a way to unravel this spaghetti.

[PaulBRobinson]

@Hogarth-MMD you should be able to select all (a) while in the Node editor, then provided you have 'Manipulate Center Points' (alt + ,) enabled you can use the scale too to move them apart.

You can also use the built in add-on Node Wrangler which is amazing for helping with Node based tasks. It has an alignment tool (Shift + =).

I'm a bit rusty as I haven't had much time to use Blender recently, but hope that helps.

[Hogarth-MMD]

The first step to implement toon textures in Blender's node editor is just to manually create toon rendering on a simple test model. I and @powroupi and many other people have the skills with Blender's node editor to do this. The information provided by @nathanvasil has given to us excellent clues about how to do this.

[Hogarth-MMD]

Thanks for the suggestion about using Node Wrangler @powroupi .

[PaulBRobinson]

Good luck with the toon textures. I personally retexture because I prefer a look similar to the Project Diva games, and so just use Cycles and the Principled Shader, but it would be nice to see a toon style available too.

[Hogarth-MMD]

In the Blender node editor, there is no way to use an image pixel as input, is there? Selecting image pixels from their x or y or u or v co-ordinates?

[Hogarth-MMD]

Principled Node https://docs.blender.org/manual/en/dev/render/cycles/nodes/types/shaders/principled.html

Interesting @powroupi . Can you tell me how to have both the power of Cycles and the rendering speed of MMD? This would be a very nice piece of information. :smile:

[nathanvasil]

Can you tell me how to have both the power of Cycles and the rendering speed of MMD? This would be a very nice piece of information.

If you haven't already, check out the EeVee rendering engine for Blender. Currently in beta, but you can explore it. Using rendering techniques sort of in line with modern game engines.

https://www.youtube.com/watch?v=eAVjwXEjDdo

Note that there is some stuff that you can do in Cycles that you can't in Eevee. Ray tracing takes time. A good example of this is refraction, where you're just never going to be able to do per-pixel refraction in real time (because that would require rendering the entire scene for each pixel.) Or, you can't do "real" radiosity. But for most things, you can do tricks that get you close enough.

[Hogarth-MMD]

Okay, I am experiencing a nice, easy success with manually setting up a material node for toon rendering EXCEPT for ONE problem. My node editor toon material is not able to receive any shadow. I have no idea how to correct this. The material is being lit by the dot product of lightVector and surfaceNormal and apparently this does not take into account shadows being cast on the material.

LN = lightVector dot surfaceNormal toonV = clamp((0.5 - LN) 0.5) toonColor = toonTex (0, toonV) //look along the left margin of the toon texture based on L dot N pixelColor = clamp((toonColor texture (ambient+diffuse))+(specular Blinn-Phong(reflection))+(addSphere))