MizunagiKB / gd_cubism

Unofficial Live2D Player for Godot Engine
https://mizunagikb.github.io/gd_cubism/
Other
111 stars 17 forks source link

Crash seems to be related to transparency. #52

Closed saierXP closed 3 months ago

saierXP commented 6 months ago

godot:4.1.1.stable gd_cubism: 0.3

When the target in the motion file is PartOpacity, it crashes when loading the model. If you turn on the viewport's transparent_bg, it seems that you can't have PartOpacity in the model or it will crash, also when transparent_bgis false, the motion file can have a few PartOpacity, but there's a chance that it will crash when it's cleared. By compiling 4.3 and the plugin, I can see that it crashes because of various memory issues.

Test models I can send to the maintainer's email.

MizunagiKB commented 6 months ago

Thank you for reporting the issue. I will look into the problem where setting PartOpacity on the Live2D model causes a crash.

You mentioned that you have a test model for verification. Would it be possible for you to attach it to this issue? If it’s difficult due to file size limitations or other reasons, I can also accept it via email. Please use the address listed in the SPDX-FileCopyrightText of the source code.

It refers to the item mentioned in the second line. https://github.com/MizunagiKB/gd_cubism/blob/main/src/gd_cubism.hpp

Thank you also for reporting about the operation in 4.3. GDCubism’s source code build itself is for 4.1, and 4.3 was completely unverified. To support 4.3, it will be necessary to align the dependent godot-cpp with 4.3, so it seems better to prepare a branch that assumes a build for 4.3 first.

saierXP commented 6 months ago

A download link has been sent by e-mail. Model file path is the same as the plugin example. This is my friend's model, I don't own it. He asked me for help, but I couldn't load it completely even after many attempts.

At first I kept one motion file referenced in l2d02.u.model3.json and deleted the rest. The model loads and unloads normally when the CurveCount in the corresponding *.motions.json is less than or equal to the number of Parameters. (For example, the number of Parameters in Mgirl01_xiufen.motion3.json is 68.)

Later on, I tried to keep a few PartOpacity, but the model can be loaded and cant't unloaded normally, and it can't be loaded if there are too many PartOpacity.

Even with 4.1 without debugging symbols, the captured crash message points to gd_cubism being in the process of clearing something, or allocating a container size.

MizunagiKB commented 5 months ago

I have received the data and confirmed its operation in debug mode. The issue seems to be occurring within the InternalCubismUserModel::motion_load function in internal_cubism_user_model.cpp.

This function further calls the Live2D Cubism Framework provided by Live2D, and there appears to be some issue with the memory operations being performed internally.

While a specific solution has not yet been found, the data you provided seems to be helpful for the investigation. I will continue to investigate and will contact you if there are any updates.

MizunagiKB commented 5 months ago

@saierXP

Update on Investigation Status

Upon final investigation of the crash site, it appears that a bug occurs when motion data is deployed within the CubismFramework and CubismCore provided by Live2D.

As a test, I wrote code to skip the loading of the motion. As a result, I was able to load the model you provided, and control the Parameters, Part Opacity, and reflect Physics in the Godot Editor.

Considering the possibility of compatibility issues between the Godot Engine and the CubismSDK, I tried loading it with the sample code provided by Live2D. However, a crash occurred in the same way, so there is a very high possibility that it is a problem on the SDK side.

I would like to report this as a bug in the SDK to Live2D, but I cannot provide the data you gave me. Therefore, I plan to identify the bug in the data structure and report it.

In order to narrow down the bug, I felt it would be beneficial to implement a feature to skip the loading of the motion.

MizunagiKB commented 5 months ago

In order to troubleshoot the issue, we have added a feature that allows you to toggle whether to load 'expression' and 'motion' during asset loading.

https://github.com/MizunagiKB/gd_cubism/tree/0.4

This does not solve the problem, but by turning off 'Load Motions', the Live2D model you provided can now be loaded.

screen_40a

I apologize for the inconvenience, but currently, I am unable to dedicate a substantial amount of time, so the investigation is not progressing.

MizunagiKB commented 5 months ago

The reason for the crash has been identified.

The crash occurred within the CubismMotion::Parse function  in https://github.com/Live2D/CubismNativeFramework/blob/develop/src/Motion/CubismMotion.cpp.

The reason for the crash was that the motion3.json file, which defines the motion of the Live2D model we received for verification, did not have the correct data structure.

To explain in detail, the file was set to allocate less memory than the actual motion information defined. Therefore, when the data reading begins, it tries to read memory beyond the allocated size, causing memory corruption.

The incorrect data generation is likely due to the data creation and output methods in the Live2D Cubism Editor.

Please contact Live2D Inc., providing information about the version of the Live2D Cubism Editor you are using and your operating environment.

MuYuJS commented 5 months ago

I'm the provider of the sample file,but i'm not the creater of those file. So i don't know the vision of Live2D Cubism Editor which created them. But it probably the vision between 2020(l2d02.u) to 2022.1(l2d34.u).

MizunagiKB commented 5 months ago

I am unable to determine what might occur due to differences between versions. Firstly, I would recommend you to ask the person who created the data to output it again. If that does not resolve the issue, I suggest contacting the developer.

The problem this time seems to originate from the data creation or the method of using the software, and I am unable to assist with that.

Lastly, I have registered a Python script that I created on gist. I am not sure if it will be useful, but it was created with reference to the CubismNativeFramework code for the purpose of checking the data this time. Please use it as a reference if it helps.

gist > https://gist.github.com/MizunagiKB/7fd109925e56db0ed88b77450cdea579

MizunagiKB commented 5 months ago

Here is the formal English translation for your GitHub Issue:

I found a post on the official forum operated by Live2D Inc., where they discuss the specifications of motion3.json. https://creatorsforum.live2d.com/t/topic/2193/2

From this post, you can also follow the link to the motion3.json specification document on GitHub. https://github.com/Live2D/CubismSpecs/blob/master/FileFormats/motion3.json.md

According to these two articles, it seems certain that the data causing the issue has incorrect values for the CurveCount, TotalSegmentCount, and TotalPointCount fields.

I'm not sure if this will work in your environment, but the tool linked in the previous post has a feature to attempt to repair motion.json files.

https://gist.github.com/MizunagiKB/7fd109925e56db0ed88b77450cdea579

By doing as follows, you can correct the erroneous data structure in src.motion3.json and output it to dst.motion3.json.

Please take a backup before running, as I haven't done much testing.

python fixed_motion3.py -i src.motion3.json -o dst.motion3.json
saierXP commented 5 months ago
python fixed_motion3.py -i src.motion3.json -o dst.motion3.json

The python script did fix the crash, and now it can play different motions!🎉 So this could be a data format incompatibility due to the live2d editor version?

MizunagiKB commented 5 months ago

Firstly, it's good to see that the Live2D model is now displaying correctly. This clearly indicates that there was an issue with the data.

The cause of the incorrect data, whether it's due to the environment where the data is being created or a problem with the software being used, is unclear.

It seems that the cause lies somewhere in the process before you receive the data.

I am unable to resolve this issue. It might be difficult, but please try to resolve it by using the official forum or similar platforms.

MizunagiKB commented 3 months ago

This issue appears to be resolved for now, so I will close it.