Model position and sizing with `auto_scale` disabled

[cjmaxik]

Hello! I wonder how can I change model position and sizing with auto_scale parameter disabled. Auto scale requires lowering the SubViewport size to reduce GPU usage, and in turn this makes zoom ins too pixelated.

[MizunagiKB]

@cjmaxik Thank you for your feedback.

First, the behavior of the Auto Scale parameter in GDCubismUserModel is as follows:

When set to ON, it renders to fit within the SubViewport. When set to OFF, it renders the target Live2D model at its original size.

The scaling performed in the ON state is determined by GDCubismUserModel, but the final quality is determined by the resolution of the SubViewport and the size of the Sprite2D.

Therefore, if you want a beautiful image, the only option is to increase the resolution of the SubViewport. If you want to set the sprite size to 512x512, there will be a trade-off as follows:

If the discussion is not about the trade-off with the rendering quality, but the final rendering target is large (for example, 4K), and preparing a SubViewport of a size that matches it results in insufficient GPU memory, you may be able to draw directly without using a SubViewport.

The drawing quality itself does not change from the result of using the SubViewport, but you will not need to consider the GPU memory to be secured for individual direct drawing.

However, using this method may be affected by the drawing order of other objects and UIs drawn in Sprite2D (because the drawing closed in SubViewport is mixed with normal drawing).

I can’t say anything without trying it out, and I don’t have time to work on it right now, but if you could tell me a little more specifically about what you’re struggling with, I would appreciate it.

[cjmaxik]

I am using the viewport size of 1920x2000, and the target render is 1080p. It is the practical limit which still has an effect of dropping quality when zoomed in, but looks fine at 100% scale.

100% scale

Zoomed in

I noticed that when I disable auto_scale the performance gain is very noticeable, but, of course, the model is zoomed in significantly.

So I had a theory that I should be able to scale/transform the model inside the viewport to preserve the quality of the model. After all, VTube Studio does not require a significant amount of VRAM to show our model in full quality with zoom-ins.

[MizunagiKB]

Thank you for your explanation of the situation. I will first address the points you have raised and provide an explanation about SubViewport.

Regarding the issue where disabling Auto Scale seems to improve performance:

This is speculative, but I believe that by enlarging the character to be drawn, most of the drawing elements fall outside the area of the SubViewport, causing a skip in the drawing process. For example, if you only draw the face, there is no need to draw anything else, so it may seem like the performance has improved at first glance.

The process performed in Auto Scale is done in the following location:

https://github.com/MizunagiKB/gd_cubism/blob/45e3ed1d9c0d760b0d3e2b97c2bda416e6678ffc/src/private/internal_cubism_renderer_2d.cpp#L114-L119

Presumably, if the user resizes, the same process will run somewhere, so the computational load will hardly change.

Regarding the issue of consuming more memory than VTube Studio:

This is as you understand. GDCubism uses SubViewport not only for drawing but also for masking. And since SubViewport is treated as a node in GodotEngine, it cannot be reused, which further compresses the memory.

This has quite a large overhead, but using SubViewport makes it easier for both me and the user to process, so this implementation has been adopted.

Regarding the improvement proposal for GDCubism:

Looking at your usage, it is clear that you are trying to draw a Live2D model in a fairly large size.

Therefore, allowing scaling and positioning to be given from the outside will allow you to secure only the GPU memory for the area you actually want to draw. For example, if you only need to draw the upper body, the rest of the area is unnecessary, so you can make the rectangle smaller.

And this can also be expected to reduce the number of drawing elements, so there is a possibility that the load will be reduced.

Also, I did not assume that you would draw large, so manual adjustment seems useful from the point of view of quality adjustment.

The "direct drawing method" mentioned in a previous post can further reduce GPU memory consumption, but when I reread the code, it seemed difficult to implement immediately because a considerable amount of modification was necessary.

[cjmaxik]

Yeah, having a direct drawing method would be nice, so people can decide if they want quality or performance, and they can use dynamic Viewport size to mask out the unwanted model parts from rendering. Best of both worlds, actually.

Thank you for your thorough explanations, it is a pleasure speaking to you 🫡

[MizunagiKB]

While it would be ideal to implement direct drawing, our initial plan is to add the adjustment function that was first requested in the manual.

We are currently in the process of a provisional implementation, with plans to make the position and size of the Live2D model adjustable. This will be adjustable in real-time on the editor, so you can set the SubViewport size to the size you want to draw and adjust it while viewing the drawing results.

Once we merge into the 0.4 branch, we will contact you again.

[MizunagiKB]

@cjmaxik

We have released version 0.4.0. https://github.com/MizunagiKB/gd_cubism/releases/tag/v0.4.0

In the new version, the parameters adjust_scale and adjust_position can be used when building.

[cjmaxik]

Works exceptionally well, thank you for the quick turnaround! P.S. Compiled with Beta4 version of the Cubism SDK

[MizunagiKB]

@cjmaxik

The other day, we discussed the trade-off between rendering quality and GPU memory usage. A pull request has been made with hints for improving quality.

https://github.com/MizunagiKB/gd_cubism/pull/59

This involves providing hints to shader variables to achieve higher quality model rendering when applying mipmap textures.

My approach was to consider the rendered result as a texture and think about filtering it. However, if possible, providing a mipmap can enhance the quality at any size.

I plan to merge this, but I thought I should let you know as it seems related to our discussion the other day.

[cjmaxik]

You won't believe it, but I wanted to ask about that today, actually. Been working on the mipmap support for assets just the other day. Can't wait for the 0.5 release!

[MizunagiKB]

We have released version 0.5.1, which includes changes to shaders that now support Mipmaps. https://github.com/MizunagiKB/gd_cubism/releases/tag/v0.5.1

To enable this feature, you need to make the textures used in the Live2D model compatible with Mipmap.

Specifically, please enable Mipmap on the screen that appears when you re-import.

The following is an example of applying Mipmap.