[BUG] Incorrect packets over the VMC protocol crash Inochi

IcedQuinn commented 1 year ago

Validations

[X] I have checked for similar bug reports and could not find any.
[X] I have tested and confirmed that this is an issue in an official branded build.

Describe the bug

Sending an incorrect OSC bundle to the VMC endpoint will crash inochi session with an array out of bounds error. The software will not close correctly and will hang at the stack trace.

Reproduction

Add a VMC Tracker
Create an OSC device in Ossia Score
Send a float value to /VMC/Ext/Blend/Catte
Observe the crash

This should not strictly require Ossia since it looks like the issue is VMC expecting a Name+Float bundle, receiving only a Float, and failing to fail gracefully.v

System Architecture

x86_64

Operating System

Linux

Version

v0.8.0

Logs

Click to expand!

icedquinn@astaraline ~/D/inochi-session-linux-x86_64 [SIGKILL]> ./inochi-session [INFO] Inochi Session v0.8.0, args=[] [INFO] Lua support initialized. (Statically linked for now) [INFO] Scanning plugins at /home/icedquinn/.config/inochi-session/plugins... [INFO] Found zone Yass [ERR ] Could not start texture sharing, it will be disabled. Is the library missing? core.exception.ArrayIndexError@../../../.dub/packages/vmc-d-1.1.3/vmc-d/source/osc/message.d(140): index [1] is out of bounds for array of length 1

Additional Context

This issue is a fork of #41, a bug I discovered while experimenting with that issue.

LunaTheFoxgirl commented 1 year ago

I've added some robustness fixes to facetrack-d, as well as added support for specifying bone/blendshape names via the address path. https://github.com/Inochi2D/facetrack-d/commit/8aeb4c28136739da8f7b245d3d005b82e433ec34

grillo-delmal commented 1 year ago

Btw, this still crashes on 0.8.2, will look into it later.

import liblo

liblo.send(9999, liblo.Message("/VMC/Ext/Blend/Val", ('s', "tgl_normal")))

grillo-delmal commented 1 year ago

This should be fixed on the next release since https://github.com/Inochi2D/facetrack-d/pull/20 was merged

IcedQuinn commented 5 months ago

It looks like this may have regressed. I just loaded the latest release, added the VMC adaptor, pointed MeowFace at it, and the program immediately takes a dive. Which is somewhat worse than before since that didn't use to happen.

grillo-delmal commented 5 months ago

:eyes: you are right, it does consistently crash with MeowFace... which is weird, it doesn't crash when using Puppetstring (which also talks VMC)... I'll look into it right away with a debugger and see what is happening

grillo-delmal commented 5 months ago

It's a weird bug, but I'm both reproducing it in older versions of session (0.8.0, 0.8.3 and 0.8.4) and on the latest version by reverting facetrack-d, so I don't think that it has to do with the update.

I don't know if it's the same thing that you are experiencing, but I'm currently getting this message when checking a debug version:

core.exception.ArraySliceError@../vmc-d/source/osc/message.d(39): slice [0 .. 18446744073709551615] extends past source array of length 2024

Tomorrow I will check more thoroughly and see if I can find what is the problem with MeowFace ...

IcedQuinn commented 5 months ago

core.exception.ArraySliceError@../vmc-d/source/osc/message.d(39): slice [0 .. 18446744073709551615] extends past source array of length 2024

It's a segmentation fault for me; likely since its an official release build.

There might be a way to pin it down

Rig the VMC tracker up to be unit tested (however hard this is; i don't know the code and i don't speak D2)
Test case to feed it samples from the different trackers
Use git-bisect w/ the tests to run down the commit that broke Meow input

I've used it to find obscure breakages in other projects.

grillo-delmal commented 5 months ago

I tried to connect MeowFace to an actual generic osc listener and didn't get any data... so I started searching around what was wrong with this, and after looking around a bit... I noticed that MeowFace doesn't even send it's data as VMC xD. It actually sends it's data using the iFacialMocap protocol ;). Inochi Session is actually able to process that protocol, and it works without problems :+1:

If you still want to use MeowFace through VMC, you can use an app like the one here https://github.com/Suvidriel/FaceMocap2VMC

IcedQuinn commented 5 months ago

Edit: this is very odd. Listening with netcat -ul <port> doesn't turn up any data at all, like it wasn't receiving packets. Yet Session crashes the instant I push the button on Meow. :thinking:

Ultimately ended up running Puppetstring to get a working grid. Though it's still mysterious (is it sending size zero packets?) and the tracking systems shouldn't be able to drop the system.

LunaTheFoxgirl commented 5 months ago

Edit: this is very odd. Listening with netcat -ul <port> doesn't turn up any data at all, like it wasn't receiving packets. Yet Session crashes the instant I push the button on Meow. 🤔

Ultimately ended up running Puppetstring to get a working grid. Though it's still mysterious (is it sending size zero packets?) and the tracking systems shouldn't be able to drop the system.

This is more a side effect of the OSC library we've "inherited" in vmc-d. It probably should be rewritten. It expects that packets will always be well formed.

Inochi2D / inochi-session