Losing Tracking

[chikaitoh123]

Hie.

Recently I have been using the this for my model. Unfortunately, my model keeps freezing at random.

To fix this, I have to manually change the OSC port on VMC, port on MC4UE Bones node, and port on Modify VMC4UE Morph to regain tracking. This only happens on UE4. When I use programs like VSeeFace, I never have this issue.

Is there something I'm missing or is there a fix?

Thank you.

[HAL9HARUKU]

There's a possibility that blocked by a firewall or security software. May be able to fix it by changing the settings.

[dpredie]

@HAL9HARUKU & @chikaitoh123 , the UE4 vanilla OSC plugin sometimes got disconnected, people usually run some check and remake the connection.

https://answers.unrealengine.com/questions/960576/unreal-engine-425-osc-plugin-disconnecting.html

maybe you can put this in the plugin?

[chikaitoh123]

Still no luck on my end. I still randomly disconnect on UE4 and UE5.

@dpredie Are you by any chance experiencing the same issue?

[chikaitoh123]

I used this blueprint. However after a while (it could be between 5 minutes ~ an hour), the model will randomly freeze. Then, I will have to close and relaunch the package build to get the tracking working again.

I tried launching the build on administrator mode as well. No luck yet.

[HAL9HARUKU]

Thanks for sharing the detailed information.

I was able to reproduce the problem on my computer. After about 40 minutes, the model froze. I will try to find the cause.

[chikaitoh123]

I tried running it only on Editor mode, the only way to fix it (while everything is still on 'play') is to change the port number and 'compile'. If its a package build, its to relaunch the package.

If you require someone to stress test it as well, I'll be more than happy to do so.

Thank you again for looking into this!

[chiisary]

dear @HAL9HARUKU I am agree with @chikaitoh123 when the track lost,I change the port,it can be fix but if i change it to the old port，it always lost so i think，the question is on the port ，which is take the data of track? thank u, I very like this plugins. I hope that it become better

[HAL9HARUKU]

I have prepared a mechanism to reconnect, so please try it.

[fofamit]

I've tested it with the reconnect blueprint and experience the same issue.

[NielsGer]

I'm also using the "fix" BP but still get the freezing error after an hour or so unfortunately

[toaxdjr]

Hey i also have that problem everything is working fine and it glitches a little and then the facetracking stops, when that happens the output log throws this (while debbuging the package in VS) [2022.04.10-00.32.06:003][853]LogStreaming: Display: 0.067 ms (0.027+0.039) ms for processing 158/350 objects in NotifyUnreachableObjects( Queued=0, Async=0). Removed 25/25 (207->182 tracked) packages and 133/133 (427->294 tracked) public exports.

[savagegunderson]

Hello, I am writing to add that I also experience this issue, the only clue to what the problem is that I've found in the message log is this: LogPluginManager: Mounting plugin VMC4UE LogClass: Warning: IntProperty FAnimNode_ModifyVMC4UEBones::Port is not initialized properly even though its struct probably has a custom default constructor. Module:VMC4UE File:Include/AnimNode_ModifyVMC4UEBones.h LogClass: Warning: IntProperty FAnimNode_ModifyVMC4UEMorph::Port is not initialized properly even though its struct probably has a custom default constructor. Module:VMC4UE File:Include/AnimNode_ModifyVMC4UEMorph.h LogClass: Display: IntProperty FVMC4UEBlendShapeState::Index is not initialized properly. Module:VMC4UE File:Include/VMC4UEBoneMapping.h LogClass: Display: FloatProperty FVMC4UEBlendShapeState::Weight is not initialized properly. Module:VMC4UE File:Include/VMC4UEBoneMapping.h LogAutomationTest: Warning: IntProperty FAnimNode_ModifyVMC4UEBones::Port is not initialized properly even though its struct probably has a custom default constructor. Module:VMC4UE File:Include/AnimNode_ModifyVMC4UEBones.h LogAutomationTest: Warning: IntProperty FAnimNode_ModifyVMC4UEMorph::Port is not initialized properly even though its struct probably has a custom default constructor. Module:VMC4UE File:Include/AnimNode_ModifyVMC4UEMorph.h

[toaxdjr]

//--ok so i found out what is happening at least to me ... 2 problems //-- the first problem was the face capture (blenshapes) would stop and would not come back //--in the function void FAnimNode_ModifyVMC4UEMorph::Evaluate_AnyThread(FPoseContext &Output) //--it would suddenly start going into this if (!VRMMapping.IsValid()) { UE_LOG(LogTemp, Log, TEXT("Not valid mapping")); return; } //--The issue was that the instance of VRMMapping would get emptied //--The solution was to instance FVMC4UEVRMMappingData on the class FAnimNode_ModifyVMC4UEMorph //--in the function void FAnimNode_ModifyVMC4UEMorph::BuildMapping() //--some of the code I added FVMC4UEVRMMappingData VRMMappingDataTemp = VRMMapping.Get()->VRMMapping; (this->VRMMappingData)->BlendShape = VRMMappingDataTemp.BlendShape; (this->VRMMappingData)->BoneMapping = VRMMappingDataTemp.BoneMapping;

//--and by using that instance everything works ... until i got "the freezing error after an hour or so" //--and figured it was like the last error and yes in OscReceiver->OSCReceiveEventDelegate //-- the invocationLits would randomly get emptied so ... //--the temporary fix make a ResetReceiverCallbacks function on UVMC4UEOSCManager class and call it on //-- UVMC4UEBlueprintFunctionLibrary::RefreshConnection void UVMC4UEOSCManager::ResetReceiverCallbacks() { if (Instance == nullptr) { Instance = NewObject< UVMC4UEOSCManager >(); check(Instance) Instance->AddToRoot();
}

for (auto& OscReceiver : Instance->OscReceivers) {
        for (auto& ISSMTM : Instance->StreamingSkeletalMeshTransformMap) {
            if (ISSMTM.Key == OscReceiver->GetPort() && !OscReceiver->OSCReceiveEventDelegate.IsBound()) {
                OscReceiver->OSCReceiveEventDelegate.AddDynamic(ISSMTM.Value, &UVMC4UEStreamingSkeletalMeshTransform::OnReceived);
            }
        }
}

} //--and so far I have had the packaged program running 4 hrs and it still tracks

[personalaccount12345666]

Toxadjr, can you make a fork for your fix?

[toaxdjr]

Toxadjr, can you make a fork for your fix?

Well, more than a fix is more like duct tape but I have forked it and updated it with the "fix" (when you "lose the tracking" it should come back after a minute at max) hope it works :D

[HAL9HARUKU]

Tweaked, Please confirm.

https://github.com/HAL9HARUKU/VMC4UE/releases/tag/v0.4.7

[toaxdjr]

Ok so update the VRMMapping issues are solved however the Event delegates still get emptied (v0.4.7)

*ran it again with a breakpoint on value change and got a message and a call stack from debugging Well... kinda found when it happens and how but as you can see in the screenshot [↓] FunctionName should be able to enter the loop and end correctly but no this returns false and then it goes into the else if on the second image and gets emptied ... ok so whatever is causing problems should be debuggable on the code snipped on the images above but I feel like else if ( FunctionIt->IsCompactable() ) is wrong ... it says it could not be executed however documentation says "True if the object is still valid and it's safe to execute the function call" <-- and this is on unreal engine code soooo ok so in Engine\Source\Runtime\Core\Public\UObject\ScriptDelegates.h Object.Get() in line 42 goes from working normally to returning a struct at NULL (aka nullptr)
where the normal response is ok so I'm pretty sure weak pointers are at fault here based on "The memory presented by TWeakObjectPtr can’t be prevented from garbage collecting." from this page (https://dawnarc.com/2018/07/ue4-tsharedptr-tweakobjectptr-and-tuniqueptr/) so when it gets destroyed anywhere else it will get "emptied" so I checked and the problem happens after reconnect is called wich kinda makes sense so you have to re-add the delegate again because the garbage collector will take it out gonna test 2 theories running a level without the reconnect BP and the other try to re-add the delegates after reconnect (like in the image) so after testing overnight seems like reconnect is not the only thing that deletes the weak pointers however the solution in the image works pretty well for me based on only having 1 delegate per port now I'm going to test with multiple

[k0123hj]

Any further informations? I tried to fix line as @toaxdjr mentioned, but still getting freezed. If anyone solved problem, please let me know.

[toaxdjr]

Any further informations? I tried to fix line as @toaxdjr mentioned, but still getting freezed. If anyone solved problem, please let me know.

well, I can see you didn't read the whole thread but here you go just add the code in the last image or just use the fork I made then place the reconnect actor blueprint in your world, and if it freezes it will reconnect after 1-minute max if you are unlucky :) but it will come back (btw as the solution is on an actor if you lose it on the editor just run the level for a min)

[k0123hj]

@toaxdjr oh yes, after I write that comment I did download fork that u made and replaced plugin flles,, and also place the reconnect actor in level, however it happened to freeze after 3 hours live, and then It didn't get reconnect in 2 minute :'( ...

in my case, I download files that you edited and replace the original files (keep the original files that you haven't touched),, am not sure do I miss something

[toaxdjr]

@toaxdjr oh yes, after I write that comment I did download fork that u made and replaced plugin flles,, and also place the reconnect actor in level, however it happened to freeze after 3 hours live, and then It didn't get reconnect in 2 minute :'( ...

in my case, I download files that you edited and replace the original files (keep the original files that you haven't touched),, am not sure do I miss something

Aight put a debug breakpoint inside of the function call when it gets called check that all the vars have the required values before exiting if it doesn't get called then is pebkac

[theboxlynx]

I'm having this issue on UE 5.1 and quite frequently. Sometimes it will trigger as quickly as 30 seconds after running, other times I can run for an hour or two and be fine. I did add a BP_RefreshConnection with a reconnect time of 1, 1.5, 10, 30, and 60, but all seemed to still permanently freeze at some point and never come back.

[Guizmo12]

Still happening in 5.1.1