SlimeVR / SlimeVR-Tracker-ESP

SlimeVR tracker firmware for ESP32/ESP8266 and different IMUs
Apache License 2.0
859 stars 296 forks source link

[Feature request] webcam based optical tracking and calibration (2 to 4 webcams) #347

Closed tails101 closed 1 month ago

tails101 commented 1 month ago

After seeing this tutorial i understood something.

https://www.youtube.com/watch?v=HQSeSZWSZ58

Psmove controller are just IMU trackers like the slimevr just with a light point on them.

I think a webcam based tracking should eliminate the drifting of slime VR and improve their calibration, or reduce the number of trackers needed. The only thing the slimeVR would need is:

A RGB led in each tracker.(for identifying and tracking) A transparent case (or a small window)

And mostly modification in the main server. To handle the optical + imu tracking and the webcams management on the PC side.

This is a small physical modification in my opinion of the trackers. But big changes in the software and quality of tracking.

What do you think of this idea?

kruemmelbande commented 1 month ago

that just sounds like psmove with extra steps. you can very much DIY a psmove controller by sticking a lamp on a slime tracker iirc, so I don't see why that's something slime would need to do

tails101 commented 1 month ago

that just sounds like psmove with extra steps. you can very much DIY a psmove controller by sticking a lamp on a slime tracker iirc, so I don't see why that's something slime would need to do

Yes and no. Psmove are not open source and those things are getting old... And maybe will become hard to find in good state one day... Their drivers are old already making them hard to install and pair on newer versions of windows. This is something slime VR won't suffer. And slime VR will be a all in one replacement solution. And it will not be sticking a lamp to a slime VR... Since a RGB led is a small surface component that can be integrated on the main circuit board.lets not forget the software component. The psmove here is a mess.

kruemmelbande commented 1 month ago

you can literally DIY psmove controllers though? I've seen people just stick a lamp on a slime tracker. idk which software and firmware they use but its possible

tails101 commented 1 month ago

you can literally DIY psmove controllers though? I've seen people just stick a lamp on a slime tracker. idk which software and firmware they use but its possible

I have literally never seen that...any tutorial,video or link? And used as body tracking? Sources are important.

kruemmelbande commented 1 month ago

the documentation for slimevr seems to not be complete, buy PSMoveServiceEx claims to support slime https://github.com/Timocop/PSMoveServiceEx/wiki/Compatible-Devices#-slimevr

tails101 commented 1 month ago

the documentation for slimevr seems to not be complete, buy PSMoveServiceEx claims to support slime https://github.com/Timocop/PSMoveServiceEx/wiki/Compatible-Devices#-slimevr

Since it say

TO DO

It is wrong...so no.

kruemmelbande commented 1 month ago

question is, is the documentation todo, or is the slimevr support todo.

tails101 commented 1 month ago

question is, is the documentation todo, or is the slimevr support todo.

No documentation, no tutorials ,no in use so no.

The other thing is. I would like it work with regular webcams or phones webcam and not just the Kinect or pseye... So no this is another big difference.

kruemmelbande commented 1 month ago

then ask the psmove service devs to add it? I mean, if slimevr were to support it that wouldn't happen overnight either. you'll have to wait either way, its just that psmoveservice already has it on the roadmap. Slimevr currently just doesn't so anything remotely like this, so it would be much harder to implement it from scratch, when you could just use psmoveservice.

tails101 commented 1 month ago

then ask the psmove service devs to add it? I mean, if slimevr were to support it that wouldn't happen overnight either. you'll have to wait either way, its just that psmoveservice already has it on the roadmap. Slimevr currently just doesn't so anything remotely like this, so it would be much harder to implement it from scratch, when you could just use psmoveservice.

Waiting isn't a problem,but The psmove service interface is a mess.and it has not been updated in 2 years so asking them is a lost cause in my opinion. Slime VR in the other hand have a much better interface and is a lot more active. And most of the hardware is slime VR based. Psmove service software is open source so slimevr server could include part of the software inside,no need to start from scratch on both side.

kruemmelbande commented 1 month ago

idk what to tell you, psmoveserviceex isn't abandoned and has had updates this year. slimevr doesn't do positional tracking, or camera based tracking, or outside in tracking or anything like that, while psmoveserviceex does all of the above and works with slime. I really don't see this being a thing that slime supports. I mean, psmoveservice just needs to implement the slime protocol and I guess do a firmware tweak to allow an led on the tracker. slimevr is open source, they can just do it.

tails101 commented 1 month ago

idk what to tell you, psmoveserviceex isn't abandoned and has had updates this year. slimevr doesn't do positional tracking, or camera based tracking, or outside in tracking or anything like that, while psmoveserviceex does all of the above and works with slime. I really don't see this being a thing that slime supports. I mean, psmoveservice just needs to implement the slime protocol and I guess do a firmware tweak to allow an led on the tracker. slimevr is open source, they can just do it.

https://github.com/psmoveservice/PSMoveService Strange i see archived,read only...

kruemmelbande commented 1 month ago

that's psmoveservice. I'm talking about psmoveserviceex.. https://github.com/Timocop/PSMoveServiceEx

tails101 commented 1 month ago

This doesn't change anything,slimevr still need an RGB led for identification. I guess this is the end of the conversation. I'll open a feature request on both side and see witch side will be faster.

Timocop commented 1 month ago

I think adding bulbs to SlimeVR and track them optically would defeat the whole point of it being SlimeVR.

It also seems you are confusing Legacy PSMoveService with PSMoveServiceEx. PSMoveServiceEx is still being updated to this day and already supported the owoTrack/SlimeVR protocol before SlimeVR even shipped officially (3 years ago). Visit the PSMoveServiceEx Discord server for additional details since the Wiki is still WIP (hence the tons of todo mentioned). People have been using SlimeVR trackers with bulbs for years now.

Eirenliel commented 1 month ago

Out of scope.

tails101 commented 1 month ago

I would like to have it re open... Slimevr is an open source project. That mean nothing should be out of scope and it should be open. It is a bad idea to gate keep (a bit like Wayland) Maybe for a slime vr2 Something the pico tracker beta showed us is A imu tracker will always drift what ever correction applied And it will always need an external view to correct itself. Optical tracking will make things cheaper too by reducing the minimum tracker number needed to 3 (or 2) instead of 5. And adding feet tracking. (Instead of 7 to 8)

kruemmelbande commented 1 month ago

if you want to implement it, do it yourself. I mean, its open source, just go do it. Open source projects do have a scope. if you don't like it, fork it. the slimevr devs don't owe you any development time, and they have better things to do, especially when optical tracking is completely unrelated to slime and psmoveserviceex literally does exactly what you want.

Slimevr isn't gatekeeping, you are completely free to do it yourself. The wayland example is dumb cuz with waylans everybody has to agree with the standart. With slime you build your own trackers and you rewrite the software yourself and that is. Go do that, but don't demand the devs so everything you want

tails101 commented 1 month ago

if you want to implement it, do it yourself. I mean, its open source, just go do it. Open source projects do have a scope. if you don't like it, fork it. the slimevr devs don't owe you any development time, and they have better things to do, especially when optical tracking is completely unrelated to slime and psmoveserviceex literally does exactly what you want.

Slimevr isn't gatekeeping, you are completely free to do it yourself. The wayland example is dumb cuz with waylans everybody has to agree with the standart. With slime you build your own trackers and you rewrite the software yourself and that is. Go do that, but don't demand the devs so everything you want

i would love to... but im an artist and not a programmer... maybe if you want a ux or ui concept. i would love to help... we are so rare in the opensource community. i do french translation as well if needed.

kruemmelbande commented 1 month ago

then this realistically doesnt have a change of happening. Its not on the roadmap, it wont be on the roadmap. If you want this feature, youll have to find somebody to do it, but the slime team probably isnt gonna do it. Again, psmoveserviceex does literally exactly what you want. Go ask them to support different kinds of cameras or whatever u need cuz this just has nothing to do with slimevr.

tails101 commented 1 month ago

Asked them on their discord and yea their software side is ready but to have slimevr trackers with adressable rgb led the hardware need some deep redesign.

kruemmelbande commented 1 month ago

slimes with rgb leds exist and people have been making them for psmove for a long time.

tails101 commented 1 month ago

slimes with rgb leds exist and people have been making them for psmove for a long time.

yea but not adressable. they use a second device to control the rgb or set up the value permanently with fixed voltage. wich is not the point i am asking about.

kruemmelbande commented 1 month ago

Well what your asking for wont happen until you either just do whatever psmove does, or you implement it yourself