search

Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
141 stars 2 forks source link

Resonite SRAnipal doesn't compatible with latest SRAnipal client #1376

Open kazu0617 opened 9 months ago

kazu0617 commented 9 months ago

Describe the bug?

Resonite SRAnipal is 1.3.2.0, but SRAnipal latest version is 1.3.6.8. Due to Resonite SRAnipal version is so old, some VIVE Headset eye tracking / facial tracking doesn't work correctly.

To Reproduce

  1. Use any SRAnipal supported headset.
    • For Example: VIVE Cosmos with eye tracking, VIVE XR Elite with eye / face tracking or VIVE Pro Eye.
    • if you using SRAnipal old version under 1.3.6.6, uninstall it
  2. Install "VIVE Console for SteamVR" in Steam or OOBE.
  3. Launch Resonite. If you using VIVE Cosmos or VIVE XR Elite, launch with "ForceSRAnipal" args.
  4. When SRAnipal connects to Resonite, Crash it.

Expected behavior

Step 4 in "To Reproduce", works correctly.

Screenshots

No response

Resonite Version Number

Beta 2024.2.12.1430

What Platforms does this occur on?

Windows

What headset if any do you use?

VIVE XR Elite

Log Files

Nothing due to now shown in Log Files for that.

Additional Context

SRAnipal 1.3.6.8 can download from Vive Developers.

This workaround also works in NeosVR, so I wrote this article. but Resonite update anytime, so issue it.

Reporters

shiftyscales commented 9 months ago

Can you provide screenshot/video demonstrating the issue? I'm a bit confused.

Check in resonite, and not works correct with "Openess" in AvatarRawEyeData.

So does SRanipal 1.3.6.8 not work at all with Resonite? Or is it only that certain information isn't being passed properly? Is any eye data being registered/passed through?

kazu0617 commented 9 months ago

Can you provide screenshot/video demonstrating the issue? I'm a bit confused.

Check in resonite, and not works correct with "Openess" in AvatarRawEyeData. So does SRanipal 1.3.6.8 not work at all with Resonite? Or is it only that certain information isn't being passed properly? Is any eye data being registered/passed through?

Is this a video description from the installation process? Or is this a video from an already-installed state?

If it is the latter, I don't think we can get a very significant video because it crashes almost immediately after loading the SRAnipal DLL after startup. @shiftyscales

shiftyscales commented 9 months ago

Oh- it crashes outright? You didn't mention that in your initial description.

kazu0617 commented 9 months ago

Oh- it crashes outright? You didn't mention that in your initial description.

SRAnipal(Resonite): old <-> SRAnipal(Vive Client): old,like 1.3.1.0 <-> Headset(Vive Pro Eye) -> works. SRAnipal(Resonite): old <-> SRAnipal(Vive Client): current,like 1.3.6.11 <-> Headset(Vive Pro Eye) -> crush. SRAnipal(Resonite): current,like 1.3.6.8 <-> SRAnipal(Vive Client): current,like 1.3.6.11 <-> Headset(Vive Pro Eye) -> works.

SRAnipal(Resonite): old <-> SRAnipal(Vive Client): old <-> Headset(Vive XR Elite) -> not tested, but heard not works. SRAnipal(Resonite): old <-> SRAnipal(Vive Client): current,like 1.3.6.11 <-> Headset(Vive XR Elite) -> works, but occur some problem. SRAnipal(Resonite): current,like 1.3.6.8 <-> SRAnipal(Vive Client): current,like 1.3.6.11 <-> Headset(Vive XR Elite) -> works.

kazu0617 commented 7 months ago

@shiftyscales If you try it a few times, you may crash and sometimes not. I will investigate this issue a little more by myself.

PointerOffset commented 3 months ago

I recently acquired a Vive XR Elite + Full Face Tracking module so I've been trying to get this to work as well. I have no other SRanipal-associated hardware to test. I installed Vive Console through Steam which is currently packaged with SRanipal runtime version 1.3.6.12. I streamed SteamVR/Resonite to my XR Elite using the latest version of Vive Hub (2.0.0a Beta) over USB3. I ran Resonite with the -ForceSRanipal flag and ensured the runtime was up.

Here are some notes:

Attached are some relevant logs. One is with Resonite's current SRanipal SDK files, the other with DLLs pulled from the latest SDK.

If there's more specific testing or logging/data I can check, please let me know. I'll post updates if I find any more information.

Edit with additional info: With the updated SRanipal DLLs, I was able to validate my ValueStreams provided Eye Openness data and Eye Direction data. They did not provide Eye Position or Pupil Dilation data. The position data stayed at a locked value that I'm unsure of. Pupil Dilation stayed at 0.

PointerOffset commented 3 months ago

As far as the XR Elite goes, SRanipal just may not be working correctly with the hardware. Judging by this forum comment from a Vive employee: https://forum.htc.com/topic/18647-getting-pupil-diameter-from-vive-xr-elite-unity/?do=findComment&comment=58367

image

Unfortunately the wording in the thread is up to some interpretation but it does seem something is not working correctly for this particular headset. Considering HTC doesn't seem to be updating this runtime anymore, it's possible it may never work correctly.

I'm only speaking to the Vive XR Elite face/eye tracking through Vive Hub streaming. SRanipal should still probably get updated to the latest SDK version for anything else that supports it.

Vive currently recommends using their OpenXR extensions. That would have to wait for Sauce. They also mention pupil dilation is not available through OpenXR right now. However, Vive makes a module for VRCFT to use with Vive Hub/Vive Business streaming. It appears to make use of a "Vive Streaming SDK". I can't seem to find such an SDK on the Vive developer site, but the Resonite team could consider contacting HTC to see if this SDK is available for native integrations.

The VRCFT module is here for reference (closed source unfortunately): https://github.com/ViveSoftware/ViveStreamingFaceTrackingModule