search

regzo2 / PicoStreamingAssistantFTUDP

Demonstration of how to fetch face tracking data via the Pico Streaming Assistant.
25 stars 10 forks source link

Version 1.1 plugin fails to start (Occasionally shows that the port is occupied during initialization) #2

Open TheBug233 opened 1 year ago

TheBug233 commented 1 year ago

Maybe there are some bugs in the plugin initialization detection in 1.1? Now the plugin will only start properly if you open Streaming Assistant and start steamvr while the plugin is showing "initializing". If the streaming assistant is opened before starting the plug-in, It may show that the port is occupied, but only VRCFaceTracking is using this port after command query, We have confirmed that pico_et_ft_bt_bridge.exe has been deleted If the streaming assistant and steamvr are not started within a period of time after this message is displayed, 10060 error will be displayed.

regzo2 commented 1 year ago

I can't seem to replicate this on my own setup (I don't have the hardware or Streaming Assistant to verify if it works correctly, I am using an emulator app I wrote to make sure the data is sent correctly). The module listens on the port for any face tracking data during the initialization process for 15 seconds, all you should have to do is use the headset during initialization to successfully handshake with the Streaming Assistant. It's the same as before except now the module listens for any face tracking data on initialization as well to verify if the module needs to be active.

The way that VRCFT's module system works means I have to account for any other modules being loaded as well, so unfortunately I can't indefinitely halt the initialization to wait for the Streaming Assistant to send data.

I am actually going to have my hands on a Pico 4 Enterprise over the next several days pretty soon so I will make sure that the handshaking process works as intended.

frg2089 commented 1 year ago

In version 1.1, you should run Streaming Assistant and then VRCFT, making sure your Pico device is always on during this time. It will not send data when the screen is off.

If Streaming Assistant does not send data to VRCFT within 15 seconds, then the initialization will fail.

and 10060 is the connection timeout.

So.

  1. Make sure the pico_et_ft_bt_bridge.exe has been deleted.
  2. Make sure your Pico device's screen is on.
  3. run Streaming Assistant.
  4. run VRCFaceTracking.
  5. Done and Enjoy it.
altunidev commented 3 weeks ago

Hey there,

I've just gone and done some testing on my own hardware. It seems to act pretty consistently in the following way:

As noted above, I've attempted to delete the .exe file (renamed and moved to a completely different directory, in case I might need it later) and it seems to have pretty similar results regardless. I was just wondering if there was any knowledge here on what the discrepancy may be in terms of wireless vs wired connectivity, especially with the emulated version (as noted above by regzo2).

If any further testing could be done with this plugin, I'd be happy to help!

Further details:

RVRPlugin.ini config file:

[PICO]
...
FtMode=2      # <---- should be the same as on the non-business software
...

GUI reflecting this option image

rogermiranda1000 commented 3 weeks ago

@altunidev all my tests with Business Streaming were with wifi, and they work. What order are you following in launching the programs (VRCFT&BS)?

altunidev commented 3 weeks ago

I've been always starting with the Pico Business Streamer -> SteamVR -> VRChat -> VRCFT. I assumed that was the correct way to do things, right?

My next idea is to ask around and try ET/FT in Resonite (I heard it's natively integrated some time ago), just to double check that it may be my streamer app instead of this plugin. If it does end up being my streamer app, I might as well try reinstalling and seeing if that helps at all... All things for tomorrow and later this week though, I suppose.

rogermiranda1000 commented 3 weeks ago

@altunidev today I've got a bug report and I'd like to check something just in case... are you using a VPN? That could explain why the wifi doesn't work but usb does

altunidev commented 3 weeks ago

@rogermiranda1000 Not actively using a VPN, no. I have one installed, but I imagine that if it's not running, it shouldn't affect the wifi connectivity, right? Especially if all other features (normal streaming, audio, etc.) works as expected via wifi.

My next thought (at the moment) is to uninstall everything Pico related on my PC (as I originally tried connecting my Enterprise headset to the consumer variant of the Streamer app), and reinstall fresh. However, if my current setup exhibits a bug you'd like to test a little more, feel free to let me know and I'd be more than happy to try and help figure things out!

altunidev commented 3 weeks ago

Quick update -- I tried removing everything from my PC pico streamer related, reinstalling the Business Streamer, and trying again. Unfortunately, it seems the behaviour remains the same (only working over USB).

I could very likely try on a different PC (have a VR-capable laptop), although the same VPN software is installed on that one (also not used most the time). Could also try uninstalling the VPN and all network related things to the VPN, but that may be more effort than what it's worth.

Any other ideas or suggestions? Already, your willingness to help is greatly appreciated! @rogermiranda1000

rogermiranda1000 commented 3 weeks ago

@altunidev are you able to connect to BS using wifi? (Are you not able to connect, or are you able to connect but then only ft doesn't work?) Because I had some problems with usb too but then it's unrelated to the module.

altunidev commented 3 weeks ago

@rogermiranda1000 The streaming works as intended over both WiFi and USB for me (using v1.1.0.1 provided from this link: https://business.picoxr.com/us/doc/BusinessStreaming). I'm able to use SteamVR and have video, controllers, audio, etc. all through both WiFi and USB.

However, when attempting to do Eye and Face Tracking, I haven't seen any success with going through WiFi. It doesn't seem to have any issues with ET/FT over USB though

rogermiranda1000 commented 3 weeks ago

@altunidev could you restart your computer, and run this in the powershell as admin? netstat -aon | Select-String -Pattern '29765'

The facetracking data is sent using the port 29765, so if any other program is using it before opening Business Connect it won't be able to create the server. If you see nothing then it's fine, if you see something reported without opening BC/VRCFT then the problem is that another program is using the port.

altunidev commented 2 weeks ago

After doing a fresh reboot and running that command, I don't see any pattern with the number 29765 running.

I'll try opening my normal programs (Steam, Discord, etc.) and seeing if those take any pattern in that port number with the same command again...

altunidev commented 2 weeks ago

Quick update -- after opening all my normal apps, it still didn't populate. The only instance that I found that port was after I turned on wireless streaming on the Pico Business Streamer app, and connected with my headset.

(when wireless) UDP 0.0.0.0:29765 *:* 22640

Rebooted and tried again wired:

(when wired via USB) UDP 0.0.0.0:29765 *:* 3184

So the port seems to only be used by the appropriate streamer software after initializing a stream