Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
141 stars 2 forks source link

Initializing Vive SRanipal facial tracking with Vive Wireless is unstable #2396

Open dfgHiatus opened 5 months ago

dfgHiatus commented 5 months ago

Describe the bug?

Attempting to initialize SRanipal facial tracking with Vive Wireless is at best unstable. The culprit for this lies in the SRanipal initialization function, it returns an unhanded error code unique to wireless initialization. For reference, the SRanipal developer SDK outlines the following success/error codes:

//========= Copyright 2019, HTC Corporation. All rights reserved. ===========
namespace ViveSR
{
    /// <summary>
    /// error code of ViveSR
    /// </summary>
    public enum Error : int
    {
        RUNTIME_NOT_FOUND   = -3,
        NOT_INITIAL         = -2,
        FAILED              = -1,
        WORK                = 0,
        INVALID_INPUT       = 1,
        FILE_NOT_FOUND      = 2,
        DATA_NOT_FOUND      = 13,
        UNDEFINED           = 319,
        INITIAL_FAILED      = 1001,
        NOT_IMPLEMENTED     = 1003,
        NULL_POINTER        = 1004,
        OVER_MAX_LENGTH     = 1005,
        FILE_INVALID        = 1006,
        UNINSTALL_STEAM     = 1007,
        MEMCPY_FAIL         = 1008,
        NOT_MATCH           = 1009,
        NODE_NOT_EXIST      = 1010,
        UNKONW_MODULE       = 1011,
        MODULE_FULL         = 1012,
        UNKNOW_TYPE         = 1013,
        INVALID_MODULE      = 1014,
        INVALID_TYPE        = 1015,
        MEMORY_NOT_ENOUGH   = 1016,
        BUSY                = 1017,
        NOT_SUPPORTED       = 1018,
        INVALID_VALUE       = 1019,
        COMING_SOON         = 1020,
        INVALID_CHANGE      = 1021,
        TIMEOUT             = 1022,
        DEVICE_NOT_FOUND    = 1023,
        INVALID_DEVICE      = 1024,
        NOT_AUTHORIZED      = 1025,
        ALREADY             = 1026,
        INTERNAL            = 1027,
        CONNECTION_FAILED   = 1028,
        ALLOCATION_FAILED   = 1029,
        OPERATION_FAILED    = 1030,
        NOT_AVAILABLE       = 1031,
        CALLBACK_IN_PROGRESS= 1032,
        SERVICE_NOT_FOUND   = 1033,
        DISABLED_BY_USER    = 1034,
        EULA_NOT_ACCEPT     = 1035,
        RUNTIME_NO_RESPONSE = 1036,
        OPENCL_NOT_SUPPORT  = 1037,
        NOT_SUPPORT_EYE_TRACKING = 1038,
    };
}

And the undocumented error code is:

FOXIP_SO = 1051   // "Weird wireless issue"

EDIT: Curiously, the newer versions of the SRanipal documentation do acknowledge this, but not the older versions of the documentation (anything older than 1.3.6.8.). They describe the error code as:

 // 1036 - 1050 reserved
LIP_NOT_SUPPORT     = 1051, // VIVE Lip Sync is not supported by VIVE Wireless Adapter

This was discovered as part of my contributions to the VRCFaceTracking SRanipal module. Here is the source code for how we are handling this error:

private bool InitTracker(int anipalType, string name)
{
    ...
    var error = SRanipal_API.Initial(anipalType, IntPtr.Zero);

    handler:
    switch (error)
    {
        case Error.FOXIP_SO: // wireless issue
            Logger.LogInformation("Vive wireless detected. Forcing initialization...");
            while (error == Error.FOXIP_SO)
                error = SRanipal_API.Initial(anipalType, IntPtr.Zero);
            goto handler;
        case Error.WORK:
            Logger.LogInformation($"{name} successfully started!");
            return true;
        default:
            break;
    }

    ...
    return false;
}

TL;DR, Resonite needs to retry initializing SRanipal when this error is raised, until it succeeds or a set number of attempts are exceeded, rather than just giving up after the first attempt.

To Reproduce

1) Start Resonite using a Vive Wireless compatible headset. This can be the Vive, Vive Pro, Vive Pro Eye, or a Vive Pro 2 with a facial tracker installed (Please note the version of SRanipal does not contribute to the issue above) 2) Wait for Resonite to load 3) Observe lip tracking initialization is unstable (Using a Vive Pro Eye's eye tracking seems to exacerbate this issue, due to the eye tracking data requirement)

Expected behavior

Vive Wireless initializes face tracking consistently.

Screenshots

N/A

Resonite Version Number

2024.6.24.1317 - Latest

What Platforms does this occur on?

Windows

What headset if any do you use?

Vive, Vive Pro, Vive Pro Eye, Vive Pro 2

Log Files

N/A - Logs are not produced during the recreation of this issue

Additional Context

N/A

Reporters

dfgHiatus euphieeuphoria

shiftyscales commented 5 months ago

As you highlighted, use of SRanipal face tracking with the Vive Wireless adaptor is not officially supported by SRanipal, seeking input from @Frooxius on how / if we want to handle this.