Can we get both RGB and Depth? or is it one or the other?

[tiancovici]

So when the library is installed, and I have /dev/video0, can I only get depth data or can I combine this library with another's for rgb?

[TimboInSpace]

From my experience (in opencv only), it's one or the other if you're trying to access it from one software. For example, my opencv installation could not open the depth stream by default because it fails to detect the stream type and defaults to opening the rgb stream. Since modifying the default behaviour of opencv to choose y10b format, it always chooses the depth stream. i.e. from my experience, one installation of opencv cannot handle both streams. Maybe you would have luck opening the y10b formatted stream in one application, and rgb in another? I have doubts though.

[tiancovici]

TimboInSpace, 2 questions

I ended up installing librekinect and was able to view it via camorama.

1. Every time I reboot the pi, do I have to make && make load the librekinect for the kinect to be mounted?
2. Do I just instantiate videocapture in opencv to slice images from the /dev/video0? or is there another step I need in the process.

[TimboInSpace]

1. Certainly not. You should only need to do that once. If you install a new kernel, you'll have to re- modprobe it, but thats all.
2. Yep that's exactly how it works. Once configured, you just open it like any other camera in the same procedure as the opencv tutorials. You may need an additional cv::waitkey(30ish) if it's running choppy. For /dev/video0, here's some snippets from my code:

main { cv::VideoCapture cap(0); if (!cap.isOpened()) { std::cout << "Unable to open camera..."; } cap.set(CV_CAP_PROP_FRAME_WIDTH, 640); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 488); cv::Mat image; cap.read(image); while (forever) { cap.read(image); cv::waitKey(30); // do stuff with image } }

[tiancovici]

How do I modprobe it?

Failing to save a frame from the capture. I confirmed opencv is working, but when ever I try to run a program, it says OpenCV Error: Unspecified error (could not find a writer for the specified extension) in imwrite, file /home/pi/opencv/modules/imgcodecs/src/lodsave.cpp terminate called after throwing an instance of 'cv::Exception' what(); /home/pi/opencv/modules/imgcodecs/src/lodsave.cpp: error: (-2) could not find a writer for the specified extension in function imwrite

all I added to you code was the following

include opencv2/opencv.hpp // note < > brackets were removed because they hide text

include string

include iostream

using namespace std;

string curFilePath; int main(int argc, char **argv) { cv::VideoCapture cap(0); if (!cap.isOpened()) { std::cout << "Unable to open camera..."; } cap.set(CV_CAP_PROP_FRAME_WIDTH, 640); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 488); cv::Mat image, frame; cap.read(image); int i; while (i < 10 ) { cap.read(image); cap >> frame; cv::waitKey(30); //Save Image curFilePath = String("./depth") + i + string(".png"); cv:imwrite(curFilePath, frame); i++ } }

[TimboInSpace]

Since this isnt an issue with librekinect, it would be good to move this discussion to the OpenCV forums. For now, it looks like you may have neglected to specify the image compression parameters (other than implicitly by the file extension). From the OpenCV docs http://docs.opencv.org/modules/highgui/doc/reading_and_writing_images_and_video.html#imwrite :

vector compression_params; compression_params.push_back(CV_IMWRITE_PNG_COMPRESSION); compression_params.push_back(9); imwrite("alpha.png", frame, compression_params);

[tiancovici]

I actually think it might be a driver installation issue, because I totally ignored the following when i did make load on librekinect

here's an example

pi@raspberrypi ~/librekinect $ sudo make load make unload_drivers make[1]: Entering directory '/home/pi/librekinect' sudo sh unload_drivers.sh Error: Module kinect is not currently loaded Error: Module gspca is not currently loaded Error: Module gspca_main is not currently loaded Error: Module gspca_kinect is not currently loaded Error: Module gspca is not currently loaded Error: Module kinect is not currently loaded make[1]: Leaving directory '/home/pi/librekinect' sudo modprobe videodev sudo insmod gspca.ko sudo insmod kinect.ko sudo chown -f -R root:root /dev/video* sudo chmod -f 755 /dev/video*

noticed others talked about it, but it was clear what the solution was. I have the latest linux driver firmware.

[lucyking]

I've see an issue long time ago that the librekinect author said that the "make unload_drivers" script is dirty,it will try remove the default modules all the time(even the modules is not loaded). I also see above Error on PC,but it do not matter,my code build and work well(though no run on Raspberry Pi 囧 ). so there should due to other issues.

[xxorde]

Q: Can we get both RGB and Depth? or is it one or the other? A: No.

I used the driver framework gspa. It makes developing video device drivers more easy. The problem is that it expects every video device to transmit one video stream. The kinect hast two. In order to make it work, I had to patch this framework to use the depth stream (by default it uses only the first stream, which is the rgb).

I made this driver for an autonomous vehicle and it satisfied my needs, but breaks many other things for example: other drivers using this framework ;)

To get a good an permanent solution gspca has to be changed to support 2 or more endpoints. Antonio Ospite has done an RGB driver, which helped me a lot to develop the depth part. When the framework supports is available the drivers can be merged or extended, but fore the moment the anther is no! Sorry.