Closed 4thWallInt closed 1 year ago
Regarding the MediaPipe error, post the log file at here when an error occurs, which is usually in the saved/logs folder so that I can find out what the problem is.
Network, I don't think multiplayer animation has much to do with MediaPipe4U, it should be a server streaming animation data and client-side display problem, I believe UnrealEngine has the example of that. You can find UE document or market to get the sample.
Important note about the MediaPipeAminInstance (your animation blueprint). At the same time in the world outline you must make sure it has only one MediaPipeAminInstance , If you want to sync animation to many actors, you can use IK Reatarget to anothers. Because the landmarks data of mediapipe that made by MediaPipeHolisticComponent can be deal one time, multiple consumers (MediaPipeAminInstance ) will be get a low animation frame rate (Multiple MediaPipeAminInstance will be competition for data).
BTW: sorry for my bad English , this is a question of the structure of the program, which is difficult to explain in a short answer.
Hey Anders, this is Sameek , and I am working with Giovanni on this project. As you requested , here is the saved log file for the error which is occuring when I am switching the number of players to more than 1 in the editor, for multiplayer testing.
I dont understand what you meant on the second point, because all the other animations are working for the player at both client and server. Only the Mediapipe Solver nodes that we have setup in the animation Blueprints, are controlling all the players together instead of just the client player.
Also , we are not looking for syncing animations. Our use case is pretty simple, the mediapipe solvers should move only the local player , that is obvious right ? Because if I am waving my hand, I only want my player to wave its hand, and not the server player. And also if I am waving my hand, the other players shall also see my hand movement, currently if the opponent players are waving their hands, I am not able to view it.
From the logs, it can be seen that the component was reloaded or re-initialized for some reason. when mediapipe started, you can not uninitialize it or re-initialized it.
The following invocations should not appear after you start capture.
UMediaPipeHolisticComponent::CreateObserver()
UMediaPipeHolisticComponent::SetupObservers()
UMediaPipeHolisticComponent::InitializeComponent() <---------Here, why we need Initialize ? because uninitialize its owner Actor?
I think this is because you break the Actor for some reason, and moving the MediaPipe component to another Actor might solve this problem
What do you mean ? This is happening from your code only, we did not even have access to the source code, so we did not modify it.
I guess the actor is re-initialized, which causes the component attached by the actor to be re-nitialize. This is due to inappropriate usage, and it is possible that the code did not handle this error and caused the crash. But arguing about whose code it was generated did nothing to solve the problem.
The crash is causing as soon as I am calling the StartCamera Blueprint node from The MediaPipeHolistic component. If you dont want to solve the issues generated from your code, you can just say so that you dont want to provide support. But you are behaving as if this code is not written by you.
Hey I wanted to come in and just say that there is no need to argue over these things. On our side, we are just trying to do the best we can with the available information to resolve these issues. On Anders' side we completely understand that he is dealing with many requests and has a lot of work on his plate.
Let's keep the conversation civil please
From the logs, it can be seen that the component was reloaded or re-initialized for some reason. when mediapipe started, you can not uninitialize it or re-initialized it.
The following invocations should not appear after you start capture.
UMediaPipeHolisticComponent::CreateObserver() UMediaPipeHolisticComponent::SetupObservers() UMediaPipeHolisticComponent::InitializeComponent() <---------Here, why we need Initialize ? because uninitialize its owner Actor?
I think this is because you break the Actor for some reason, and moving the MediaPipe component to another Actor might solve this problem
I'm helping you, but you've been dissatisfied, it's not the attitude of discussing problems, I've told you how to try to solve it, but you don't want to hear it at all, you're just expressing dissatisfaction.
moving the MediaPipe component to another Actor might solve this problem !!!!!
It is still the same, it does not matter where I am moving the MediaPipe component, as soon as I am calling the StartCamera function, it crashes for more than 1 player. For just 1 player it is fine.
Is there this line in every crash log file?
Error: [mediapipe_api.dll] Invalid state: pipeline running
You can give me a crash log file, after you moved MediaPipe Component to another Actor .
MediaPipe4U_Demo.log Here it is.
Also please reproduce this at your end first. It is very easy to reproduce and also you will have better understanding at what is causing the issue
Anders, could you maybe try to reproduce this error on your end by just calling the StartCamera function while keeping the number of players to more than 1?
Okay, I will try to reproduce 2 players later.
Error: [mediapipe_api.dll] Invalid state: pipeline running
One thing I know from this line is that when you call StartCamera the MediapipeHolisticComponent has not finished initializing, or the level is streaming (If async load level).You call StartCamera is too early, it should be in begin play or later, I will add a check in the next release to prevent this call ...
I will try reproduce it.
Anyway, I'll replicate the multiplayer mode. And as a way to try it for now, inherit UMediaPipeHolisticComponent (C++, Blueprints can do the same), override its InitializeComponent function, add a feild as a flag (e.g. bInited = true) after the base UMediaPipeHolisticComponent ::InitializeComponent function call, and don't call camera when bInited not true:
YourComponent.h
UCLASS(ClassGroup="Default", meta=(BlueprintSpawnableComponent))
class UYourComponent : public UMediaPipeHolisticComponent
{
UPROPERTY(EditAnywhere, BlueprintReadWrite)
bool bInited = false;
virtual void InitializeComponent() override;
}
YourComponent.cpp
void UYourComponent ::InitializeComponent()
{
Super::InitializeComponent();
bInited = true;
}
if bInited is not true, dont call StartCamera
when i play PlayAsListenServer, every thing is OK. But, you can't use one camera to capture two windows at the same time, But even if you do that, it won't cause a crash
Maybe it's a buggy for a specific version, I'm packing a new version that can be downloaded later today.
BTW: In my case, I didn't crash regardless of whether the number of players was 2 or 3, but my machine was stuck over 3
Download a new version,it has been tested with multi player network mode.
No, you just tested it with the default project as it it. But you need to test it with third person player character, without possessing the default pawn, because clearly the multiplayer here means that we are controlling the two characters.
So to reproduce the issue -
I am still getting the error, attached are the screenshot and the log as well MediaPipe4U_Demo.log
OK~Wait for my testing
@sameekue5 One thing, let me know when this crash happened, from the logs it looks like stopping the game
Hey Anders, were you able on your end to reproduce the issue?
"So to reproduce the issue -
I've been fixing it for 3 hours, it's a bit of a troublesome problem, It's almost done. But I'm afraid I don't have time to test this problem, when I fix it is done, please test it, and then feedback here. because In China, it's early in the morning.
The crash happened as soon as I press the Play button, when it calls the StartCamera function in BP.
I have done without full testing, just hot fix :
https://1drv.ms/u/s!AkmROUeQfSBjzgEz6gu8FCr7ibot?e=x79QpF
BTW: I'm going to bed, it's too late and too late
The crash is not happening, but the video tracking is coming as black, and now the hand tracking is stopped as well. Kindly check
Which version worked well with the video tracking on your PC?
Unreal 5.0. It was working fine before with single player, but after your updates it stopped displaying the video
Hey Anders, just wondering if you are still going to be helping us tomorrow
Hi,
If plugin has BUG, post issue, I will fix it asap.
If it's just a technical discussion, I can't continue. I don't think discussing so much today are plug-in-related, and I don't have any plans to develop this piece of content.
BTW: I don't think a complex multiplayer game can be developed with Blueprints only, It's impossible to use Blueprints and wait for Unreal to do everything automatically, Epic can't do everything. Blueprints are just a designer tool. For my side, energy is limited. Sorry for that. Good luck !
Anders Xiao.
发件人: @.> 发送时间: 2023年2月12日 20:28 收件人: @.> 抄送: Anders @.>; State @.> 主题: Re: [endink/Mediapipe4u-plugin] Network replication / multiplayer issues (Issue #20)
Hey Anders, just wondering if you are still going to be helping us tomorrow
― Reply to this email directly, view it on GitHubhttps://github.com/endink/Mediapipe4u-plugin/issues/20#issuecomment-1427020447, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAB2UJEBTPGINGCRXPODD3TWXDJNFANCNFSM6AAAAAAUSQ7MWI. You are receiving this because you modified the open/close state.Message ID: @.***>
When I am testing this in the UE editor with two players (multiplayer mode) , with Play as Client or Play as Listen Server option, the program crashes, and it gives an error at the MediaPipeHolisticComponent cpp class, which we do not have access to. Also it doesn't seem to work with network replication at all, the pose solvers that we have set up in the animation blueprints, are moving all the players together in multiplayer mode, where actually it should just move the local player."