search

PointCloudLibrary / pcl

Point Cloud Library (PCL)
https://pointclouds.org/
Other
10k stars 4.62k forks source link

Support RealSense cameras on Linux #1215

Open taketwo opened 9 years ago

taketwo commented 9 years ago

Intel will be releasing their new RealSense 3D camera soon. Unfortunately the list of officially supported OSes includes only one item, Windows 8. I already have a working PCL grabber that uses official SDK for Windows. It's more or less complete and will be integrated in the mainstream PCL soon.

A more interesting topic is support for RealSense cameras on Linux. @teknotus has been working on this for several months and has managed to get color, depth, and IR data from the camera on Linux. He also has a Qt-based app to view the streamed images.

In this issue I propose to discuss how we can integrate RealSense support into PCL for Linux.

taketwo commented 9 years ago

As far as I can see, the depthview app currently has no separation between Qt interface and low-level USB interfacing. @teknotus do you plan to extract low-level code in a separate library and provide a high-level API? If yes, then PCL may depend on the library and PCL grabber will be a thin wrapper. If no, then we will need to put the low-level stuff directly into the grabber. (This is not something new, we have other grabbers working with USB directly.)

teknotus commented 9 years ago

I always intended to have a library with several parts for low level, data processing, and GUI components as separate dependency trees. Trying to create a clean API before I even knew how to use it was too much to figure out at once though. All of the low level stuff is in one file that mostly has Qt specific stuff for communication with other parts of the app that could easily be turned into wrapper functions. There are still a few unknowns that complicate making a finished API, but if the data is easier to access then more people can try to figure out those issues. If there are any great existing API's that might be a good reference for me. I've been wanting to do some major refactoring anyways.

taketwo commented 9 years ago

libfreenect perhaps?

teknotus commented 9 years ago

FYI I started working on this, but it's not in a useful state yet. Hopefully in the next week.

taketwo commented 9 years ago

Great to hear! By the way did you have a chance to integrate your patch for the depth stream formats support into the kernel?

teknotus commented 9 years ago

I haven't submitted the kernel patch upstream partly because I wanted to have an explanation of what the difference is between the two depth formats is, but I haven't had time to finish my calibration tool yet. The other issue is that submitting patches to the kernel has to be done in a very specific way that almost requires doing it from the command line, and I have to do some work to setup those tools.

taketwo commented 9 years ago

Not that I ever submitted a kernel patch... but if I could assist in some other way, let me know.

teknotus commented 9 years ago

And a whole month goes by... I'm closer. I have about 100 lines of code for detecting every v4l2 camera, and gathering enough information to determine if it is a realsense camera, whether it's a color or depth camera, and what serial number it is. I have about another 200 lines that starts up two cameras and uses an epoll event loop to service them whenever a frame is ready. Some of the functionality that should be in the library is still in main, and it doesn't shut down the camera cleanly on exit yet so the camera needs to be reset (usb unplug replug) before it can be used again. I think that is too frustrating a situation to share to other programmers so I'm not releasing it yet, but it's been a month so I figured I should at least provide an update. There is a lot of other functionality in various pieces of code including depthview that I also want to add, but I'll probably release it as soon as the current features are stable.

VictorLamoine commented 9 years ago

@taketwo I believe this is fixed thanks to your work in #1230 and #1336.

taketwo commented 9 years ago

Oh no, RealSense is a different family of cameras, so the issue is still open :(

teknotus commented 9 years ago

I released some code about a week ago that might make things easier. I put an MIT license on it, but can cross license it the same as PCL if that helps. Much simpler code than the depthview app for getting to the data. Definitely not formulated as an easy to use library yet help would be appreciated. Still need a temperature calibration tool as the depth numbers change with temperature.

https://github.com/teknotus/depthview/tree/rstools

teknotus commented 8 years ago

Intel just released an apache licensed library for the two released cameras, and a third soon to be released camera for Linux, OSX, and Windows.

https://github.com/IntelRealSense/librealsense

bloyl commented 8 years ago

@taketwo

Sorry to jump on an old thread but I'm unclear on this issue.

Does the PCL grabber shipping with PCL 1.7 support the opensource library ( https://github.com/IntelRealSense/librealsense) or is it still dependent on the official windows SDK and thus windows only?

Thanks

taketwo commented 8 years ago

PCL has support only for the official windows SDK. And it is only present in master, not in any of the released versions.

stale[bot] commented 4 years ago

Marking this as stale due to 30 days of inactivity. It will be closed in 7 days if no further activity occurs.

SergioRAgostinho commented 4 years ago

@kunaltyagi I'm not sure we can close this just yet because the current librealsense 2.0 support we have in our source tree has some issues with linux which have not been weeded out. We can always close this one and open a more up-to-date detailing the current issues.

stale[bot] commented 4 years ago

Marking this as stale due to 30 days of inactivity. It will be closed in 7 days if no further activity occurs.