Works with Microsoft Kinect?

[mp3guy]

You can get it to work by adding the freenect driver to OpenNI2

git clone https://github.com/OpenKinect/libfreenect.git cd libfreenect mkdir build cd build cmake .. -DBUILD_OPENNI2_DRIVER=ON make

ln -s lib/OpenNI2-FreenectDriver/libFreenectDriver.so ../../ElasticFusion/deps/OpenNI2/Bin/x64-Release/OpenNI2/Drivers/

[zry656565]

Any idea about running Kintinuous with Kinect v2.0?

I have installed libfreenect2, and successfully run the test program of libfreenect2: ./bin/Protonect.

But there is no libFreenectDriver.so in lib.

jerry@tu1504:~/libs/libfreenect2/build/lib$ ls
libfreenect2-openni2.so    libfreenect2.so      libfreenect2.so.0.2.0
libfreenect2-openni2.so.0  libfreenect2.so.0.2

And I tried to:

ln -s libfreenect2-openni2.so ~/lab/Kintinuous/deps/OpenNI2/Bin/x64-Release/OpenNI2/Drivers/

But that didn't work.

[mp3guy]

I haven't tried it with Kinect v2.0 I'm afraid. But I don't believe the libfreenect2 driver works with OpenNI2, you need to directly interface it.

[zry656565]

Okay, I'm trying to do this. If I success, I will record my steps here.

[veolata]

Great software, if Kinect v2 not compatible at this moment , any other camera support except Kinect v1?

[mp3guy]

No, but it's easy to add.

[robnsngh]

KinectV2 is working after installing freenect2 with OpenNI2 support. Depth intrinsic matrix can be passed as a file through cli.

[stephane-lb]

Oh well done ! Happy to hear that, have to play with !

[ac4fun]

@robnsngh it's a big news for me~ It cheers me up !

[ac4fun]

@zry656565 hey, buddy! have you succeeded in working with Kinect v2 using Kintinuous? I have build Kintinuous successfully on Ubuntu 14.04 and GTX970 and it can run with the demo klg . But it can't recognize the Kinect V2(I make sure the usb version is 3.0).

What' more, I tried to use the Logger2 but it did't work well . Could you please show me your record? @robnsngh

[Ben-Mack]

Hi @mp3guy , Can you confirm what sensor works and has the best resolution with Kintinuous? (I'm looking to buy one) Does the Intel Realsense F200 or Kinect v2 supported?

Thank you for such great open source project!

[zry656565]

@kingsleynuaa No, I cannot run Kintinuous with my Kinect v2 for now, and I bought a ASUS Xtion PRO Live for current research.

[JiamingSuen]

Hi, @robnsngh Would you please tell us your detailed steps of making Kinect v2 working? I've tried ln -s libfreenect2-openni2.so ~/lab/Kintinuous/deps/OpenNI2/Bin/x64-Release/OpenNI2/Drivers/ like @zry656565 did and it didn't work either. What do you mean by Depth intrinsic matrix can be passed as a file through cli.? How did you link the libFreenectDriver.so file for libfreenect2? Thanks a lot!

Ps. I know this is not a big deal but I'm still a little confused, are we talking about Kintinuous or ElasticFusion? Because this is the Kintinuous repo and @mp3guy linked the ElasticFusion lib(ln -s lib/OpenNI2-FreenectDriver/libFreenectDriver.so ../../ElasticFusion/deps/OpenNI2/Bin/x64-Release/OpenNI2/Drivers/)...

[speleophysics]

I'm attempting to get Kintinuous working with a Kinect. I've built the libfreenect driver and linked it to /usr/lib/OpenNI2/Drivers/. I assume this is the correct place to link, given that OpenNI2 is installed globally on my machine and not in a deps/ directory. However, I get a "no devices found" error when running Kintinuous. Is there something else I needed to do that I'm missing?

As a test, I did run glview from libfreenect, and it's working fine. So it's not a hardware problem.

[speleophysics]

To follow up on my previous query. After hours of trying, I never got it working on my machine. Now I've re-installed Ubuntu (16.04), and it does work on a fresh install. Not sure what was creating the problem before (on an old 14.04 install). I compiled libfreenect midway through the standard build procedure. If I remember correctly, it was after OpenNI2 and ffmpeg.

[riematrix]

@speleophysics Hi, did you get it working with Kinect2? I'm able to get the driver working and got both rgb and depth image with NiViewer2. But I got this following error while running Kintinuous:

Kintinuous: /opt/rgbd/Kintinuous/src/utils/OpenNI2Interface.cpp:119: OpenNI2Interface::OpenNI2Interface(int, int, int): Assertion `findMode(width, height, fps) && "Sorry, mode not supported!"' failed.

At first glance I thought Kintinuous is trying to match the width/height between rgb and depth image, which are not equal when the resolution is 960x540 or 1920x1080.

[speleophysics]

I am using an original connect, not a Kinect2.

[JunshengFu]

Hi @riematrix I came across the same problem as you did with Kinect 2. Did you solve the problem?

Point resolution: 11.7 millimetres Creating live capture... [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 10 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @2:11 with serial 016998250647 [Info] [Freenect2Impl] found 1 devices libva info: VA-API version 0.35.0 libva info: va_getDriverName() returns -1 libva error: va_getDriverName() failed with unknown libva error,driver_name=(null) [Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error [Info] [Freenect2DeviceImpl] opening... [Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 2016384 ir: 608*33792 [Info] [Freenect2DeviceImpl] opened [Info] [Freenect2DeviceImpl] starting... [Info] [Freenect2DeviceImpl] submitting rgb transfers... [Info] [Freenect2DeviceImpl] submitting depth transfers... [Info] [Freenect2DeviceImpl] started Kintinuous: /Kintinuous/src/utils/OpenNI2Interface.cpp:119: OpenNI2Interface::OpenNI2Interface(int, int, int): Assertion `findMode(width, height, fps) && "Sorry, mode not supported!"' failed.

[riematrix]

@JunshengFu No. But i believe there should be a solution.

[wine3603]

I got the same error with you buddy... This may come from kinect v2's resolution. Thomas @mp3guy , may I ask where can I change these settings in Kintinuous?

Kintinuous: /home/wine/Kintinuous/src/utils/OpenNI2Interface.cpp:119: OpenNI2Interface::OpenNI2Interface(int, int, int): Assertion `findMode(width, height, fps) && "Sorry, mode not supported!"' failed. Aborted (core dumped)

[mp3guy]

That function queries the OpenNI api to ask if the resolution you requested is supported by the driver. Currently the system only supports depth and rgb streams that are the same size.

[gaizixuan0128]

Hello.Does anybody get it working with KinectV1? After I builded it,and run the Kintinuous GUI It can't find the device: "gaizixuan@Lenovo:~/build$ ./Kintinuous Point resolution: 11.7 millimetres Creating live capture... failed! DeviceOpen using default: no devices found" I am a totally new learner,and really hope anybody could help me...

[JunshengFu]

@gaizixuan0128 I would suggest to debug as following steps 1) make sure the libfreenect and openni 2 are working with kinect 1 after installing 2) make sure the symbolic link are created probably. If there is a problem, instead of using the relative path as "ln -s lib/OpenNI2-FreenectDriver/libFreenectDriver.so ../../Kintinuous/deps/OpenNI2/Bin/x64-Release/OpenNI2/Drivers/", try to use the full path to create the symbolic link. Good luck!

[gaizixuan0128]

@JunshengFu Now the libfreenect and openni2 are working with kinect1,and the symbolic link already exists.But still get the same problem.... Could you explain how to use the full path to create? Thanks a lot!!!

[JunshengFu]

@gaizixuan0128 By full path, I mean that refer a file by the path starting from "home" directory, e.g "/home/..../libFreenectDriver.so". If you are using linux, you simply copy the file and paste it to a text editor to see its full path.

in this case it should be: ln -s /home/yourPathTo/libFreenectDriver.so /home/yourPathTo/Drivers/

[gaizixuan0128]

@JunshengFu Thank you!I will try again

[spyd3rweb]

@JunshengFu libfreenect2 supports Color and Depth video mode of 512 x 424; you'll just have to change the initialization resolution for Elastic Fusion, in MainController.cpp ln 37: Resolution::getInstance(512, 424);//Resolution::getInstance(640, 480)

[JunshengFu]

@spyd3rweb Thanks for pointing it out :-) !

[taoxiaobin]

@JunshengFu Hello, Are you succeed in getting it working with KinectV1? I tried it by your supposed steps, it can't find device. "Point resolution: 11.7 millimetres Creating live capture... failed! DeviceOpen using default: no devices found" Do you have any suggestion for it? Thanks!

[JunshengFu]

@taoxiaobin Yes, I got it working with Kinectv1. (1) Can you start Kinect successfully with openni2 or libfreenect? (2) I would recommend you to verify every dependence you installed for this project. For example, try to run their example code. Because, I built it again about a month ago, and I found some dependence libraries are updated, e.g. function names are changed. So, instead of downloading the latest dependence, I use the version which is compatible.

[roliveiradoteu]

@spyd3rweb

The change in resolution enables the use of the Kinect V2, however I can see 2 black horizontal strips in the RGB image, which then result in the point cloud having black texture.

Is there a way to overcome this issue?

[AutoIC]

It took awhile to figure out how to make Kintinuous working with Kinect V2, but finally it worked! (1) I started with a fresh install of Ubuntu 14.04 and run bash ../build.sh directly as shown in https://github.com/mp3guy/Kintinuous (2) Then the trick was really in fixing the dependencies in many of the library scripts that Kintinuous depends on, some of the most frequently seen libraries in my fixing process are the following: pcl/ Eigen/ boost/ pangolin/ For example, I had to change all " #include <pcl/ " to " #include </usr/include/pcl-1.7/pcl/ " (3) The next thing is to install the freenect2, I simply followed https://github.com/OpenKinect/libfreenect2 (4) Last but not least, I modified the Resolution in MainController.cpp as @spyd3rweb sugguested: Line 231: Resolution::get(512, 424); //Resolution::get(640, 480);

It seems mine also has the "black horizontal strips" problem as @roliveiradoteu 's

[wine3603]

@roliveiradoteu @AutoIC
He guys, my Kinect V2 works well now, did you modified the intrinsics as @spyd3rweb mentioned ? In the case of ElasticFusion, you should modify both the resolution and intrinsics.

[roliveiradoteu]

I am back testing the software with the kinect: i just noticed my data is inverted: the door is actually on the left not on the right :-1:

Using the correct intrinsics does not prevent the black stripes.

Elastic fusion is the same

[roliveiradoteu]

from what I can see the image produced by libfreenect2 is flipped, the code needs to be updated if it is to work with the Kinect2

[Allen-jl]

@wine3603 Would you mind telling us how you make your kinect2 run in ElasticFusion? THX a lot

[MarcSN311]

I am able to get my Kinect 2 working with NiViewer and libfreenect2. However I can't get Kintinuous to work:

~/Kintinuous/build % ./Kintinuous                          
Point resolution: 11.7 millimetres
Creating live capture... failed!
DeviceOpen using default: no devices found

Where do I have to put libfreenect2-openni2.so to make it work with Kintinuous?

[sotoglzz]

As @speleophysics issue, I'm having problems to get work Kintinuous with KinectV1, I'm in ubuntu 14.04, I compiled libfreenect successfully, and linked the drivers with:

ln -s lib/OpenNI2-FreenectDriver/libFreenectDriver.so ../../ElasticFusion/deps/OpenNI2/Bin/x64-Release/OpenNI2/Drivers/

I test the kinect with glview from libfreenect, and it works fine. Does anyone know how to fix this?? I really don't want to upgrade my machine to 16.04 as @speleophysics did

Thanks in advance

[aimeemccabe15]

Has anyone any more ideas on this? @MarcSN311 did you find a fix? I'm having the same issue as you. I'm using a KInect V2 with 16.04. I keep getting the error:

Point resolution: 11.7 millimetres Creating live capture... failed! DeviceOpen using default: no devices found

I have libFreenect2 installed and can successfully view the test feed through that.

[neur1n]

So this is another year of this issue. In short, I compiled libfreenect2 manually and Kintinuous is working with Kinect V2. Therefore I post it here in case someone needs it.

So I think you will need to install OpenNI2 before building libfreenect2, as while building libfreenect2 it will try to detect if OpenNI2 is installed. I'm not sure if the -DBUILD_OPENNI2_DRIVER=ON CMake flag is necessary, I typed it along with cmake .. anyway.

Then I think you will have libfreenect2-openni2.so and libfreenect2-openni2.so.0 in .../libfreenect2/build/lib. Then after sudo checkinstall these two libs will be located at /usr/local/lib/OpenNI2/Driver/. I myself copy them to /usr/lib/OpenNI2/Driver/, but I'm not sure if this is necessary.

The next step, since Kintinuous defines the resolution of depth camera as 640x480, you will need to change the according pairs in MainController.cpp and PangoVis.cpp` to 512 and 424, which is the resolution of Kinect V2's depth camera.

I think that's it. I might miss something, so feel free to ask questions and see if I can help. Thanks.

(By the way I cannot get a good point cloud however...)