search

PoseAI / PoseCameraAPI

Tools to work with the Pose Camera app
Apache License 2.0
146 stars 38 forks source link

Unreal Engine 5 Demo -- app looks connected but camera not turning on #18

Closed kortina closed 1 year ago

kortina commented 1 year ago

I'm on Mac in Unreal Engine in the demo project.

I entered play mode, entered the ip address / port on my iphone, and in the Unreal console, I see this message:

LogTemp: Display: PoseAI: Sent handshake {"HANDSHAKE":{"name":"Unreal LiveLink","rig":"MetaHuman", "mode":"Desktop", "context":"Default", "whoami":"", "signature":"", "mirror":"YES", "syncFPS": 0, "cameraFPS": 60, "handModelVersion": 1, "packetFormat": 1}} to 192.168.1.16:56889

So it looks like communication has been established.

Note:

image

Any ideas what to try next to get this working? Excited to play with this -- the demo looks great.

System details:

  Model Name:   MacBook Pro
  Model Identifier: MacBookPro18,4
  Chip: Apple M1 Max
  Total Number of Cores:    10 (8 performance and 2 efficiency)
  Memory:   64 GB
  System Firmware Version:  7459.141.1
  OS Loader Version:    7459.141.1
  Serial Number (system):   YHX46TDYHP
  Hardware UUID:    81919AE7-C2F3-5CD2-84EA-A242F66B9CA6
  Provisioning UDID:    00006001-000C502C21F8801E
  Activation Lock Status:   Disabled
PoseAI commented 1 year ago

Hi. So you need to turn your phone sideways (make sure orientation lock is turned off on your phone first!). Basically room and desktop mode (from the handshake) expect your phone to be turned in "landscape" orientation. Portrait mode expects your phone to be upright in... portrait orientation!

The idea was that the developer may expect a certain view range depending on the gameplay, and thus requires the player to orient their phone to match.

On Mon, Oct 17, 2022 at 6:46 PM kortina @.***> wrote:

I'm on Mac in Unreal Engine in the demo project.

I entered play mode, entered the ip address / port on my iphone, and in the Unreal console, I see this message:

LogTemp: Display: PoseAI: Sent handshake {"HANDSHAKE":{"name":"Unreal LiveLink","rig":"MetaHuman", "mode":"Desktop", "context":"Default", "whoami":"", "signature":"", "mirror":"YES", "syncFPS": 0, "cameraFPS": 60, "handModelVersion": 1, "packetFormat": 1}} to 192.168.1.16:56889

So it looks like communication has been established.

Note:

  • in the PoseAI iphone app, I don't think camera permissions has been requested.
  • in the PoseAi iphone app, I see the message:

[image: image] https://user-images.githubusercontent.com/5924/196246676-8e5396cf-6489-44fa-8dc2-d4d681eae9a4.png

Any ideas what to try next to get this working? Excited to play with this -- the demo looks great.

System details:

Model Name: MacBook Pro Model Identifier: MacBookPro18,4 Chip: Apple M1 Max Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 64 GB System Firmware Version: 7459.141.1 OS Loader Version: 7459.141.1 Serial Number (system): YHX46TDYHP Hardware UUID: 81919AE7-C2F3-5CD2-84EA-A242F66B9CA6 Provisioning UDID: 00006001-000C502C21F8801E Activation Lock Status: Disabled

— Reply to this email directly, view it on GitHub https://github.com/PoseAI/PoseCameraAPI/issues/18, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOGO6DFWO5ZCWAOQS5HP3E3WDWGIHANCNFSM6AAAAAARHJT5SE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Pose AI - 'Understanding bodies in motion'

NOTE: These emails are for discussion purposes only. Unless and until we agree and sign a written License Agreement, our email exchanges do not constitute a binding contract, or a commitment by Pose AI Ltd or its employees to provide any software or services. We accept no liability, obligation or responsibility toward the recipient.

kortina commented 1 year ago

Ah, I had my phone on orientation lock! Works now. Is github issues the best place to ask some other questions about usage with unreal engine, or is there some sort of other forum where I should ask about that?

Specifically, I'd love to know:

lmk if there is another place to post questions like this (or if I should create separate issues for them so they index better).

Thanks for your help so far!

PoseAI commented 1 year ago

Hi can you check if you are in “desktop” mode? That would only animate the upper body. In Unreal this could be caused by setting the handshake, and potentially by using desktop animation blueprint (which blends in LiveLink only to the upper body).

We currently don’t provide any recording or freezing capability (our current focus is real time live animation rather than recording) but Unreal does have the Take Recorder which can be used to record animations as FBX and then you can select individual frames

On 17 Oct 2022, at 20:04, kortina @.***> wrote:

 Ah, I had my phone on orientation lock! Works now. Is github issues the best place to ask some other questions about usage with unreal engine, or is there some sort of other forum where I should ask about that?

Specifically, I'd love to know:

This seems to be working for the torso, head, and arms, but not the legs -- eg, I can't get the actor to sit down in the demo -- is this expected / do I need to do something else to control the rest of the body? Is there some way to "capture" a particular pose? -- eg, suppose I got the actor to sit down and I wanted to "freeze" that pose and apply it to another character rig, eg, in a Sequence? lmk if there is another place to post questions like this (or if I should create separate issues for them so they index better).

Thanks for your help so far!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

kortina commented 1 year ago

Ah, I was connected to the "Desktop" actor. Thanks!

Checking out Take Recorder now -- thanks for the tip!

PoseAI commented 1 year ago

No worries. Glad it is resolved