Channel can bake as vertex color and can be previewed using “Use Baked”

[morirain]

This is an unfinished commit, it needs testing or other changes.

Added a new channel option, when enabled, the "Bake All Channels" option will bake that channel into vertex color, you can preview the baked vertex color after turning on "Use Baked"

[ucupumar]

Thanks for the PR! After testing it, there are several things come to mind.

• I don't think the baked vertex color should have its own preview mode, maybe just add a toggle between the baked image and baked vertex color
• Since it's a toggle, I think there should be a specific node to store/preview baked vertex color inside ucupaint node. If you are confused about how the inside of ucupaint node works, you can ask me in ucupaint discord server, maybe I can explain it in more detail there.
• If the user didn't set the name of the target vertex color name, you should use a default name like 'Baked ' + channel_name and it automatically sets the name back to the channel attribute
• Include alpha option is not needed, since alpha should automatically included when the channel is using alpha.

[morirain]

After careful consideration, I believe that vertex color requires a separate preview mode, not only for the baked vertex color. Therefore, the vertex color preview mode should be independent. On the UI, I suggest having four buttons, representing R, G, B, and A. Users can combine or view one channel individually. This is due to the characteristics of vertex color, which make it very suitable for storing special data on the model in the workflow, rather than just for painting.

This is common in the NPR rendering of game engines. I will give you two examples.

The first example is that people will calculate the smoothed normals of the entire model and store them in the vertex color data, using the RGB channel. Then they use the smooth normal data to extrude the model, thereby make the anime outline of the model. These operations prevent the outline lines from breaking. Later, some people will also use the Alpha channel to control the width of the outline.

The second example is that some people use vertex color to blend different materials, which is common in making Landscape. These people need to be able to preview the four color channels independently.

Anyway, I would like to hear your opinion. Do you think these changes are necessary?

[ucupumar]

I see. you can also use custom textures for those purposes, it's not specific vertex color features.

I still don't think vertex color needs special preview mode, but individual 'R' G' B' in preview mode can still be really useful and it should work with any RGB data, whether it's from vertex color, textures, or any generated ones. To make things simpler, I think it's better to focus on vertex color baking to channel results first rather than the preview mode.

For your use cases, they're very similar to issue #8, which when implemented can provide combined channels into single textures or vertex colors. So, the solution for that issue can be implemented in another pull request.

[morirain]

@ucupumar check my new changes, I removed the individual vertex color preview mode, then added an option to switch between vertex color and texture, and now I can view them using the Channel Preview Mode.

[ucupumar]

Ok, thanks for the update! I just tested it quickly, I think the target vertex color name (bake_vcol_name) should be filled automatically if the user leaves it empty. For example 'Color' channel will automatically create a new vertex color named 'Baked Color' after baking and set it back as channel's 'bake_vcol_name'. I hope that's understandable.

[ucupumar]

Sorry for the late reply. It has been a busy week. I just tested it again, it's still not working for me. The target vertex color is still empty after baking. Can you show me a video demo or something?

[morirain]

The problem has been fixed, I forgot that it can sometimes be None.

[ucupumar]

I just tested it and okay, it works, but how does it work again? The target vertex color name is already filled from the start, I never knew something like this before

[morirain]

I'm immersed in my own work these days, I'm sorry that my reply wasn't very timely :).

Let me explain how it works. The key points is in the Property declaration. According to the documentation, I overloaded the getter function. When trying to access the property (from the UI or elsewhere), I check if it is empty and assign the default value.

get (function) – Function to be called when this value is ‘read’, This function must take 1 value (self) and return the value of the property.

bake_vcol_name : StringProperty(
            name='Target Vertex Color Name',
            description='Target Vertex Color Name',
            default='', get=get_channel_vcol_name, set=set_channel_vcol_name)

def get_channel_vcol_name(self):
    # When you open an existing blend project, the Property may be None instead of the default empty string, so I did a double check.
    name = self.get('bake_vcol_name', '')
    if name == '':
        # Access Property based on strings, This does not perform any getters and setters, preventing infinite loops.
        self['bake_vcol_name'] = 'Baked ' + self.name
    return self['bake_vcol_name']

[ucupumar]

Sorry for always replying late. 🙏 Oh I see, I never knew blender API had set get event. I thought it only had update event. I think this PR is almost done. One thing is the 'First force index' still looks weird to me. I'm thinking how about the baked vcol channels always use the first few indices according to the channel order?

[morirain]

Your idea is great, but I have some concerns. Of course, we can move the specified channel to the top level so it can be exported safely, but the user is not aware of this. If we add this feature, we need to include an extra note (probably weirder?). So I think we should keep the "First Force Index" option because it's simple. I have been using this feature recently and I use it to ensure that certain vertex colors are exported correctly.

[ucupumar]

Hmm, I see, but the problem is each channel has its own settings. How about moving the setting to bake all channels operator instead? Maybe it's only visible if there is any channel that enables 'bake as vertex color'.

[morirain]

That's a good solution. I'll update the code later.

[ucupumar]

Ok, thank you!

[morirain]

@ucupumar I've finished uploading the changes. 😁

[ucupumar]

Ok, I'll test it soon. Thank you!

[morirain]

I just realized that I should add a default option so that the user doesn't change the order of the vertex color. I also forgot about the UI display below version 3.20, I'll fix later.

[ucupumar]

Ok, I just tested it a bit, few things to note:

• Maybe add one more option to arrange first few vertex color indices to match all the baked vertex color order. So 'baked color' is followed by 'baked metallic', 'baked roughness', and so on, if all those default channels are using vcol bake.
• Baking to vertex color option should be only visible on Blender 2.92 or greater

Thanks for the great work! 🙏

[ucupumar]

Oh yeah, almost forgot. I don't think it makes much sense to bake normal channel to vertex color. I think you can make the normal channel have no option for bake as vertex color. One more thing, if the user is checking 'Bake as vertex color', wouldn't it be better if 'Use Vertex Color' is also enabled? So the bake result immediately uses vertex color instead of image.

[morirain]

Updated, check it out👌

[ucupumar]

Ok, thank you! I'll check it more thoroughly tomorrow 👍

[ucupumar]

Ok, I just tested the PR again. There are still a few things to note again:

• The channel's 'baked_vcol' should also be deleted when the channel is deleted, the code is in Root.py
• You should also put 'vcol_force_first_channel_idx' prop into BakeInfo.py, so the bake settings can still be remembered when you do a rebake again

I think that's probably all that is left, hopefully, thanks for the great work!

[morirain]

@ucupumar Maybe when the user deletes a channel, if the channel has vertex colors enabled, then the user should be asked if they want to delete the vertex colors as well. I think deleting things without prompting is a bad idea, what do you think?

[ucupumar]

Sorry, I've been busy today, I'll check ucupaint usual behavior tomorrow because that kind of issue is not only specific to vertex color, a baked image can also be lost if you delete the channel.

[ucupumar]

I just checked again, yeah you can add an option to prompt the user to also delete the vertex color since the default behavior will not delete it. I think the baked image is fine since it will detect if the image has more than one user before deleting it. Anyway, 'Use Vertex Color' in kinda redundant since it has the same function as 'Bake As Vertex Color', I think it's fine to remove 'Use Vertex Color', and to know which data is used, you can add '(Active)' beside the baked data name.

[morirain]

@ucupumar Done👌

[ucupumar]

Ok, already tested it, there are still some things that need to be cleaned up, but I think I'll do it myself after merging. Thank you for the good work!