Suggestions for Higher Resolution cameras that can do 60FPS

[HipsterSloth]

Reposting @CircuitLord's question from another location so that others can chime in here:

"Can you suggest a webcam to me that is similar to the ps eye with a higher resolution and is compatible with your PS Move Service?

I've been looking at your project for a while, and I'm going to try it myself, except I've noticed the controllers in game are a little jumpy, and their movement isn't as smooth as the real vive. I assume a higher resolution would help with this.

I plan to buy two of whatever webcam I decide to get, and possibly two more in the future. Oh yeah, and I don't really want to spend over 30 bucks per camera.

Any help is appreciated. Thanks for creating this awesome program, it will help tons of people!"

[HipsterSloth]

Reply from Cboulay:

"There are higher resolution webcams but they are all much slower (max 30 fps), and they're all around $100.

Hold off on new cameras until we have the filtering worked out. That should get rid of most of the jitter.

And no matter what cameras you get, no matter how much filtering we do, it's never going to be as good as the Vive. I suppose positionally you can get close if you have 6+ cameras all with their FOVs containing the sphere, but the IMU in the PSMove isn't as good as in the Vive wands so there's going to be some error there."

[CircuitLord]

First, thanks HipsterSloth!

Now my message back to Cboulay:

Thanks for the reply.

Okay. Yeah, I looked around some, but could only find 1080p cameras with 30 fps max.

Yes, I knew from the start this would never be a vive replacement, but I plan to get as close as possible to a vive for a lot less money.

[HipsterSloth]

This is actually why I had started work a while ago on PS4 camera support. It turns out it's really hard to find a cheap HD webcam that can do >=60hz. The logitech C920 used to be able to do that but then they started hardware locking the framerate to 30hz, or so I have been told. The PS4 camera appears to be the only one I can find that is HD and has fast refresh framerates.

The problem with the PS4 camera is that it has a custom AUX port even though it's a regular USB3 camera (see Issue #35) for details about that. It's also been a huge pain to try and get working in Windows (due to lack of needed functionality in libusb).

As for other cameras, we do want to generically support other webcamera through OpenCV's generic camera interface. However this requires two things:

1) The ability to filter out cameras that you actually want to use for PSMove tracking. Other HMD tracking cameras should be ignored. See issue #76.

2) The ability to calibrate lens properties of the camera, in particular the focal length. I made some progress on a tool for this (see issue #50), but in it's current implementation it was hard to get a reliable calibration. Still more work is needed to make that a user friendly experience.

So for now the best best is to stick use two-four ps3eye cameras. With multiple cameras you can triangulate the controller position, which helps with the jitter. And ps3eye cameras are super cheap from Amazon: https://www.amazon.com/PlayStation-Eye-3/dp/B000VTQ3LU

[CircuitLord]

I've heard about that PS4 camera problem, its really annoying that Sony made it irregular so that it has to be hacked to make it work with a pc. Anyone without a basic knowledge of circuits (not me), won't be able to make it work.

Yeah, I was just gonna buy 4 of the ps eyes since they are so cheap. Is 4 the max amount of cameras this software supports so far?

[HipsterSloth]

Yep 4 is the max. It's sort of an arbitrary limit that I can set higher (hardcoded limit at the moment), but in practice many machines tend to get bogged down trying to deal with more than 4 video feeds. I would also recommend buying just 2 first, and see how that works for you. Adding more camera past 2 can help with 360 tracking, but wont really help with tracking quality. Triangulation happens between all pairs of cameras that can see the controller, then we average the result together.

If you get two cameras working, then feel like adding more, check out issue #148. People in there talk about their 3+ camera setups.

[CircuitLord]

Oh okay. I can see how so many webcams might be too much to handle for a pc.

Okay, thanks for the tip.

[picklefart9]

Just positing this as a camera that might work in the future. These are currently so expensive that they defeat the purpose, but there are many cheaper variants with lower resolution. I imagine these will come down in price soon: https://www.aliexpress.com/item/Linux-Full-HD-1080P-OV2710-micro-usb-camera-board-for-android-60fps-with-170-Degree-Fisheye/2036014832.html

[HipsterSloth]

Wow! I'd call $38 a pretty good deal if it actually does HD resolutions at 60hz. I just ordered that. This might be a great alternative to the ps4 camera if this works.

[cboulay]

1080p is about 2200 diagonal pixels. If the lens fov is accurate, that's about 13 pixels per degree. PSEye is about 11 ppd. I expect there won't be much of an improvement in accuracy, but that is a major increase in FOV (75->170). That also means it'll be next to useless without a proper camera calibration :\

"High frame rate– deliver 60 fps in 1280X720 resolution,30 fps in 1920X1080 resolution." That's actually a decrease in accuracy at 60 fps (8.6 ppd), but again, a much larger workspace.

[CircuitLord]

I agree, that looks like a great camera. Can you let me know how it works out for you @HipsterSloth? I'd be interested in buying 2 if they work.

[CircuitLord]

Yeah, I'm more interested in the 170 FOV, that will get rid of a lot of the deadzone problems.

Do you think this camera would work with your software? If so, it might be a slightly worse alternative to the PS 4 Camera. https://www.aliexpress.com/store/product/720p-hd-MGPEG-YUY2-mini-CMOS-OV9712-Dual-Lens-USB-2-0-people-count-Camera-module/913995_32402153784.html?spm=2114.8147860.0.95.UvUup2

[rovarma]

That looks like a pretty interesting camera. Do note that, reading the datasheet, it can only do 30 FPS at 1080p, which makes sense given that it's still USB2. The 720p mode can do 60 FPS, but has the disadvantage that it's actually a crop, so you lose the data outside of the 720p range, which may or may not be a problem.

[rovarma]

Also, I'm skeptical about USB2 performance in 720p mode. Even in Bayer mode, it would still need 1280x720x60 = ~52 MB/s of bandwidth, which seems a tall order for USB2.

[CircuitLord]

Yeah, that camera does seem a bit shady. If the 720p works at 60 fps without too many comprimises, it might be a good option.

[HipsterSloth]

Good news (kind of)! The camera @picklefart9 suggested finally arrived. I was able to conduct some quick testing and it does look like it can do 1280x70@60fps! I was able to test this in a OBS, skype, and Yawcam. I didn't need to install any special drivers. It just worked out of the box.

However there is an important caveat. It achieves 60fps via MJPG compression. I don't know yet how much this compression will affect the tracking quality so some testing is needed. Also I can't yet figure out how to make OpenCV open a MJPG stream. It wants to default to YUYV, which definitely destroys the USB bandwidth (you only get 7fps in this mode).

[picklefart9]

Doh! That sounds so close yet so far. I imagine the compression adds latency too?

[HipsterSloth]

It doesn't seem to add any perceivable latency (just me waving my hand around the video feed seems to not lag by any amount I can notice), but I won't really know until I can do some tracking testing with it. We're already going to have to compensate for video feed latency on the ps3eye camera (it's on the order of milliseconds) so if the MJPG compression adds any additional latency amount we can compensate for that as well. It may actually be that any additional time spend compressing the image (which is done in hardware) is made up for by not having to transmit as many USB packets to deliver the video frame (MJPG claims about 1:20 compression, as compared to the 1:3 you get with the Bayer filter on the psmove). My larger concern is compression artifacts around the contour of the bulb affecting tracking quality. All that said, until I can actually get the video feed running through PSMoveService, any latency or tracking quality concerns are just speculation on my part.

[HipsterSloth]

A bit more progress. I was mistaken about OpenCV not being able to open an MJPG stream. Something much weirder was happening. After my OpenCV testing last night I couldn't get the camera to run at 60fps in any of the other web cam tools I was using (stuck running at 7fps). I was worried the camera broke so I tried it on my wife's laptop. It worked again for her so we tried playing with the camera settings. Turns out setting the exposure to anything but "Auto" completely murders the frame rate. No idea why this would be the case. You can set all of the other camera settings, most importantly the "gain", but not the exposure. Super weird. So when I prevented the OpenCV test app from setting the exposure it ran fine. The USB cable is super short so here is an awkward camera angle shot (PC is on the floor):

Now that this works in the OpenCV test app it's possible to do some tracking testing.

[picklefart9]

"Turns out setting the exposure to anything but "Auto" completely murders the frame rate. No idea why this would be the case." Mark that down as one of the 'issues' that would constantly get posted.

I do see some grain in the image that looks like it could impact tracking, but overall not bad. Good luck on your tracking tests!

[rebitti]

Hello @CircuitLord , forget high resolution 60 fps cameras are sooo expensive ! Look at Pointgrey website and you will see. PSEYE is the best cost/benefit.

[CircuitLord]

Hello @CircuitLord , forget high resolution 60 fps cameras are sooo expensive ! Look at Pointgrey website and you will see. PSEYE is the best cost/benefit.

Yes, I agree that the PS Eye is one of the best options out there. This topic is more about the people who have a little more cash to burn, and can get a slightly better setup than just PS Eyes.

[lamiska]

Did anybody try this camera?

https://goo.gl/l5J6wZ

https://goo.gl/dkVZJk

Somebody on internet claims it can do 60fps - https://www.reddit.com/r/Twitch/comments/2lfe8s/good_webcam_for_streaming/cluaxim/

[gb2111]

is 60fps really a requirement? because i am playing with riftcat when I set thread delay that test_console_client shows 30 fps I don't feel much difference. i know its not same fps as camera fps but i am still interested. thanks.

[Zangetsu38]

Hello, with rpcs3 team developer works on PSMove integration, us trying using generic camera witn off course 60hz possible, like C920, Brio (Brio is compatible 720p up to 90hz, and 1080p 60) etc. So i want know if that is juste possible for can emulate PS3 move game, with generic camera compared PSEye, i see here and other website say that is limite impossible, but in reallityn it is possible with uvc or other things ? And if any developer can join us for help with integration real PSMove and PSEye for start is welcome, for moment have only implemented Fake Move/Camera :)

[G1jsb3rt]

How about using a kinect?