CPU & GPU usage

[matcham]

Thanks so much for creating this tool, that's exactly what I've been looking for :-) (I'm not using VR chat, my goal is to receive Quest2 fingers data in Max over OSC). It kind of work, when running FingerOSC, I can see OSC message arriving on port 9000, but I also see that FingerOSC takes about 31% of CPU and more than 70 % of my GPU (nvidia rtx 2080) :-( I'm wondering if it would be possible to disable FingerOSC window rendering ? maybe it would spare some CPU/PGU ? because using that much power "just" to get finger tracking is a bit... too much :-s

[A3yuu]

Thank you for reporting I tried turning off rendering, but it didn't work. If you know the settings, it would be helpful if you could tell me. Due to the use of the SteamVR Plugin and Unity, especially Unity only uses constant calls and quaternion Euler transforms, so eliminating their dependence is a beautiful solution. I don't know any other way to get skeleton information.

[matcham]

Thanks for replying Sorry I can't help with Unity, I don't know much about it :-( Looks like you made the ALVR Quest --> SteamVR skeleton input bridge ? Out of curiosity : would it be possible to add the skeleton OSC send directly in ALVR ? it would be probably more efficient (and convenient :-)

Using FingerOSC : I can receive some data in Max, but I'm having hard time trying to decode the stream...

I receive something that looks like that :

print: /avatar/parameters/FingerFlag 0
print: /avatar/parameters/Finger0 0.27292
print: /avatar/parameters/Finger1 0.583896
print: /avatar/parameters/Finger2 0.436098
print: /avatar/parameters/Finger3 0.563946
print: /avatar/parameters/Finger4 0.611573
print: /avatar/parameters/Finger5 0.400248
print: /avatar/parameters/Finger6 0.605781
print: /avatar/parameters/Finger7 0.43422
print: /avatar/parameters/Finger8 0.612277
print: /avatar/parameters/Finger9 0.451443
print: /avatar/parameters/FingerFlag 1
print: /avatar/parameters/FingerFlag 0
print: /avatar/parameters/Finger0 0.590259
print: /avatar/parameters/Finger1 0.15501
print: /avatar/parameters/Finger2 0.635411
print: /avatar/parameters/Finger3 0.423307
print: /avatar/parameters/Finger4 0.617491
print: /avatar/parameters/Finger5 0.657338
print: /avatar/parameters/Finger6 0.7225
print: /avatar/parameters/Finger7 0.426231
print: /avatar/parameters/Finger8 0.572636
print: /avatar/parameters/Finger9 0.597717
print: /avatar/parameters/FingerFlag 2
print: /avatar/parameters/FingerFlag 0
print: /avatar/parameters/Finger0 0.034355
print: /avatar/parameters/Finger1 0.565307
print: /avatar/parameters/Finger2 0.521304
print: /avatar/parameters/Finger3 0.509824
print: /avatar/parameters/Finger4 0.595926
print: /avatar/parameters/Finger5 0.568805
print: /avatar/parameters/Finger6 0.678428
print: /avatar/parameters/Finger7 0.551483
print: /avatar/parameters/Finger8 0.647217
print: /avatar/parameters/Finger9 0.51984
print: /avatar/parameters/FingerFlag 3
print: /avatar/parameters/FingerFlag 0
print: /avatar/parameters/Finger0 0.668995
print: /avatar/parameters/Finger1 0.199142
print: /avatar/parameters/Finger2 0.651661
print: /avatar/parameters/Finger3 0.436467
print: /avatar/parameters/Finger4 0.675957
print: /avatar/parameters/Finger5 0.641935
print: /avatar/parameters/Finger6 0.703965
print: /avatar/parameters/Finger7 0.427441
print: /avatar/parameters/Finger8 0.652275
print: /avatar/parameters/Finger9 0.566474
print: /avatar/parameters/FingerFlag 4

But I can't find any documentation on how fingers orientations are encoded...

looking in your code, I've found :

[SerializeField]
    private SteamVR_Action_Skeleton SkeletonActionLeft;
    [SerializeField]
    private SteamVR_Action_Skeleton SkeletonActionRight;
    public float[] data = new float[40];
    public bool isTracked = false;

    int[] FingerNameStretched_SteamVR =
    {
        SteamVR_Skeleton_JointIndexes.thumbProximal,
        SteamVR_Skeleton_JointIndexes.thumbMiddle,
        SteamVR_Skeleton_JointIndexes.thumbDistal,
        SteamVR_Skeleton_JointIndexes.indexProximal,
        SteamVR_Skeleton_JointIndexes.indexMiddle,
        SteamVR_Skeleton_JointIndexes.indexDistal,
        SteamVR_Skeleton_JointIndexes.middleProximal,
        SteamVR_Skeleton_JointIndexes.middleMiddle,
        SteamVR_Skeleton_JointIndexes.middleDistal,
        SteamVR_Skeleton_JointIndexes.ringProximal,
        SteamVR_Skeleton_JointIndexes.ringMiddle,
        SteamVR_Skeleton_JointIndexes.ringDistal,
        SteamVR_Skeleton_JointIndexes.pinkyProximal,
        SteamVR_Skeleton_JointIndexes.pinkyMiddle,
        SteamVR_Skeleton_JointIndexes.pinkyDistal,
    };
    int[] FingerNameSpread_SteamVR =
    {
        SteamVR_Skeleton_JointIndexes.thumbMiddle,
        SteamVR_Skeleton_JointIndexes.indexProximal,
        SteamVR_Skeleton_JointIndexes.middleProximal,
        SteamVR_Skeleton_JointIndexes.ringProximal,
        SteamVR_Skeleton_JointIndexes.pinkyProximal,
    };

I managed to get an array of 20 values for each hand, it doesn't seem to match the order described in the code above. Do you send these data in a format specific for VRchat / Unity plugin, or is it raw output from steamVR skeleton input ?

I also noticed that OSC messages are sent at 90 fps, but if I filter repetitions, finger tracking seems to be at around 22.5 fps. Is that expected ?

[A3yuu]

今のALVRをビルドできないのと、こうすればLMなど他のデバイスでも使えるのでこうしています。依存が減ってメンテナンスが容易です。

VRCがの16データしか許さないため、20のデータを半分の10にして2回送ります。(左右合わせて4回) また、1フレームでアニメーションが再生されるため4フレーム使ってその4回を再生します。

[A3yuu]

https://github.com/A3yuu/FingerTruckerOSC/pull/2#issue-1257495334