Closed DatanIMU closed 1 year ago
PoseAI
commented
1 year ago We need some more information:
and then finally printing the recvMessage string to debug and then posting here could help us see what is causing the issue
DatanIMU
commented
1 year ago U4.26 and plug 1.4, I use example project in github, not test with marketplace. {'version': '1.3.1', 'userName': 'williams', 'sessionUUID': 'CC68047A-1656-46D6-993E-DE5ADEE72F21', 'deviceName': 'iPhone14,3'} {'HANDSHAKE': {'name': 'Unreal LiveLink', 'rig': 'UE4', 'mode': 'Room', 'context': 'Default', 'whoami': '', 'signature': '', 'mirror': 'YES', 'syncFPS': 0, 'cameraFPS': 60, 'handModelVersion': 1, 'packetFormat': 1}}
prehaps, it is caused by network situation. sometime UE works well, but sometime silk. and sometime parts of packages are OK, while the left parts can't be deserialized.
In fact, when I use iphone13 pm to work with poseCAM UE4.26, the movements in UE are not smooth. I guess maybe some packages are not deserialized.
I want to trace a log what UE plugin gets from the networks. and get exactly evidence what happened.
I guess maybe it is not a good way to send such a big package by UDP, package can not be deserialized, maybe just a normal UDP loss package.
when send package too fast or package too big or Rx side handle json too slow, all of them can cause udp loss.
Now, I decide to do three things:
DatanIMU
commented
1 year ago do you have a idea to trace what UE get from the internet? seems the package hold by const FString& recvMessage. I don't know how to put recvMessage in FLiveLinkLog::WarningOnce
PoseAI
commented
1 year ago Hi, we actually patched the code yesterday for versions 4.2x for something that is likely causing your issue: the use of a UE macro for converting the packets, which was leading to different behavior depending on compile/build and sometimes overwriting part of the packet. Can you check if you are using the updated version of PoseAIUdpSocketReceiver.h from yesterday? And if not, can you see if updating solves your issue? We corrected the UE marketplace plugin and the code itself on Github but haven't increment the plugin number on github yet (it will be 1.41).
On Sun, Oct 30, 2022 at 11:14 AM maemoKKK @.***> wrote:
U4.26 and plug 1.4, I use example project in github, not test with marketplace. {'version': '1.3.1', 'userName': 'williams', 'sessionUUID': 'CC68047A-1656-46D6-993E-DE5ADEE72F21', 'deviceName': 'iPhone14,3'} {'HANDSHAKE': {'name': 'Unreal LiveLink', 'rig': 'UE4', 'mode': 'Room', 'context': 'Default', 'whoami': '', 'signature': '', 'mirror': 'YES', 'syncFPS': 0, 'cameraFPS': 60, 'handModelVersion': 1, 'packetFormat': 1}} then continues send {'PF': 1, 'Rig': 'UE4', 'LeftHand': {'RotA': 'PauNXFKYihwft6IiPKrQcnHdKGeWpHKmLpXvvNOZM5pycfIiKfciqjLENXUuxoQkKBnbb/KULFb4p8KTN9UoxEPVItkpb7K8LbdOl9IYNRXCs7LXZpTaeN8pYxSBbq7faDQlZ36r', 'Point': 'VnhJ', 'IK': 'UJqUir'}, 'RightHand': {'RotA': '5GuBSCf326vPcKvy6dstT2jf1VzuTHcgw33pUGZ95vssR+hP0M0SSScGzG1UShbY4fsNPkeay/0tRUb7wn2qR2aV3yr2Oadayn0qQucFvp2/RYaE8DsEYuaA8cpUVRcm7YrtVFb+', 'Point': 'pMWC', 'IK': 'c9XCtP'}, 'Body': {'VisA': '10011', 'RotA': '40mgwCqQsoKkqdO0soKlqdO0yoPivPS5jC3Bob0Q40mgwCqQ40mgwCqQ1lt3sEutyXKetNZJ3ijh0lli0+mw2rkh2SmZ1tjk4Sj4z8bt1+hQ3JfNkaiA+Zoke5rS8WWiprrWyWKfZw+ed4nDo98ZUtdE7RsgWelg', 'ScaA': 'gLeogoAAAFAJAAf/f/', 'EveA': 'AAAf/AAAf/AAAf/AAAd+AAAf/AADdyAAAf/AABALAAAAA', 'VecA': 'eOc4f/T9gIY6WFufciZkc5sJ'}, 'ModelLatency': 7, 'Timestamp': 22191.875} {'PF': 1, 'Rig': 'UE4', 'LeftHand': {'RotA': 'OFr8WCKhhyv+r5HCOTo4bVHSKwbzoSJ8NDVhuoNiMJnSbZIbLXaDpwKYPGSvxJPnJckybKKVL+ZapDJsPrSqweOcIVh0bGLFMFamkzIBOsUvr7KpaeUHgx9KatU2dU9Yb1ThbR8w', 'Point': 'VnhJ', 'IK': 'UJqUir'}, 'RightHand': {'RotA': '5stXSfg52mu3cXwn65r4UUkk1B0UTcdAwR4TUWab6Rr6SbiSz302Skcgyt14Sxby5JrgP/fYyp1ORgcOwJ3LSBao4erIOxeYyP1HQ1cWvJ3cRfaS8dq+YxZt8voCVLcS7zqfVDbm', 'Point': 'pMWC', 'IK': 'c9XCtP'}, 'Body': {'VisA': '10011', 'RotA': '42m1v7qJs8KlqSO6s7KmqSO6y6PpvDS7i23Nog0B42m1v7qJ42m1v7qJ1guLsAukyqKotGZP3tj70Nl51Em+2fkw2Omh1tjq4Xjzz6b12DhV3FfOkXiA+cocd6rs8KWppXriwOIzZn+bd9nIo58VUZdt7UsOWymW', 'ScaA': 'gMengoAAAHAJAAf/f/', 'EveA': 'AAAf/AAAf/AAAf/AAAd+AAAf/AADdyAAAf/AABALAAAAA', 'VecA': 'efcOgJT0gKY3WmtRdZZvdBsC'}, 'ModelLatency': 7, 'Timestamp': 22191.893}
with timestamp plus 0.2ms continues.
prehaps, it is caused by network situation. sometime UE works well, but sometime silk. and sometime parts of packages are OK, while the left parts can't be deserialized.
In fact, when I use iphone13 pm to work with poseCAM UE4.26, the movements in UE are not smooth. I guess maybe some packages are not deserialized.
I want to trace a log what UE plugin gets from the networks. and get exactly evidence what happened.
I guess maybe it is not a good way to send such a big package by UDP, package can not be deserialized, maybe just a normal UDP loss package.
when send package too fast or package too big or Rx side handle json too slow, all of them can cause udp loss.
Now, I decide to do three things:
- get a trace what UE pose plugin get by udp, If you can guide me to program, please give a hand. I am not familiar with UE API.
- use network monitor to trace the whole internet
- cut pose AI package into three parts, and send by three udp package, body, l_hand, r_hand, and facial expression
— Reply to this email directly, view it on GitHub https://github.com/PoseAI/PoseCameraAPI/issues/20#issuecomment-1296220611, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOGO6DEGH6HVROUUVNPHYN3WFZKAXANCNFSM6AAAAAARRRD6BY . You are receiving this because you commented.Message ID: @.***>
-- Pose AI - 'Understanding bodies in motion'
NOTE: These emails are for discussion purposes only. Unless and until we agree and sign a written License Agreement, our email exchanges do not constitute a binding contract, or a commitment by Pose AI Ltd or its employees to provide any software or services. We accept no liability, obligation or responsibility toward the recipient.
DatanIMU
commented
1 year ago nice job， it works，
the live link in UE shows me "failed to deserialize json object" after check posecam plugin code, and modify like below in order to trace what sting the plugin get. but, it seems the code below is wrong,
I need help to print recvMessage or someting else to debug this issue?
void PoseAILiveLinkServer::ProcessNetworkPacket(const FString& recvMessage, const FPoseAIEndpoint& endpointRecv) { static const FGuid GUID_Error = FGuid(); if (cleaningUp) return; TSharedPtr jsonObject = MakeShareable(new FJsonObject);
TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(recvMessage);
if (!FJsonSerializer::Deserialize(Reader, jsonObject)) {
static const FName NAME_JsonError = "PoseAILiveLink_JsonError";
FLiveLinkSubjectKey failKey = FLiveLinkSubjectKey(GUID_Error, FName(endpointRecv.ToString()));
FLiveLinkLog::WarningOnce(NAME_JsonError, failKey, TEXT("PoseAI: failed to deserialize json object from %s, %s, %s"), endpointRecv.ToString(), Reader->GetErrorMessage(), **recvMessage);
return;
}