search

giacomodabisias / object_recognition

A lot of different object recognition methods using pcl.
14 stars 10 forks source link

Tried with kinect #2

Open ahundt opened 9 years ago

ahundt commented 9 years ago

Hey, your code looked interesting so I decided to try it out. I was able to get it to compile and I'm hoping to get it to work with a kinect, but openni doesn't seem to want to detect it. Is that what you created it to work with?

Also, could you perhaps explain what the values in the models file that are read at startup mean? I know the first one is the filename of the model, but what are filter and icp_iterations? Are they parameters for the program to use or do they provide information about the model?

Thanks for putting this up. :-)

giacomodabisias commented 9 years ago

Hi! Its quite some time that I am not working on that project. I am using usually now a grabber without openni (only libfreenect) wince openni is quite useless. If you need it I can give it to you and you can integrate it. The values are quite complex. The filters are color filters which remove from the scene everything which is not similar in color to the model, icp_iterations are the number of icp iterations which should align the scene to the model. All parameters change the programs behaviour. I don't remember all but if you need I can have a look.

Have a nice day

2015-06-12 1:47 GMT+02:00 Andrew Hundt notifications@github.com:

Hey, your code looked interesting so I decided to try it out. I was able to get it to compile and I'm hoping to get it to work with a kinect, but openni doesn't seem to want to detect it. Is that what you created it to work with?

Also, could you perhaps explain what the values in the models file that are read at startup mean? I know the first one is the filename of the model, but what are filter and icp_iterations? Are they parameters for the program to use or do they provide information about the model?

Thanks for putting this up. :-)

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2.

G. Dabisias, PhD Student PERCRO, (Laboratory of Perceptual Robotics) Scuola Superiore Sant'Anna via Luigi Alamanni 13D, San Giuliano Terme 56010 (PI), Italy mob.: +39 3480839095

ahundt commented 9 years ago

Yes I wouldn't mind integrating the code you are talking about.

If it isn't too much trouble I would appreciate some more information on the parameters, otherwise I can try to figure it out.

giacomodabisias commented 9 years ago

Tomorrow I will send you the freenect grabber and explain a bit everything . Il 12/giu/2015 22:17, "Andrew Hundt" notifications@github.com ha scritto:

Yes I wouldn't mind integrating the code you are talking about.

If it isn't too much trouble I would appreciate some more information on the parameters, otherwise I can try to figure it out.

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-111603284 .

ahundt commented 9 years ago

Thanks! I appreciate that a lot. I'll also incorporate any of my changes to the library and submit pull requests.

ahundt commented 9 years ago

I also had a pre-existing model I wanted to try matching, currently without color. However once kinect data started flowing it quickly crashed with a null pointer exception.

ahundt commented 9 years ago

I can't provide details at the moment because I'm on my phone but I can provide info and look into it later

giacomodabisias commented 9 years ago

Sure, no problem Il 12/giu/2015 22:30, "Andrew Hundt" notifications@github.com ha scritto:

I can't provide details at the moment because I'm on my phone but I can provide info and look into it later

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-111605604 .

ahundt commented 9 years ago

I'm running the program with a simple 3" sphere in the pcd format, and according to the console output below the kinect is loading up successfully. However, it appears the result of (std::get < 0 > (found_models[i]))[0] in the call to pcl::transformPointCloud (*(model_list[i]), *rotated_model, (std::get < 0 > (found_models[i]))[0]); in visualizer::visualize() called from main() is null. Still need to track the reason for this back to the source.

stack trace

note: main is at the bottom, where it finally crashed by accessing a null value is at the top.

#0  0x000000010003cb6f in float vector[4] Eigen::internal::pload<float vector[4]>(Eigen::internal::unpacket_traits<float vector[4]>::type const*) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/arch/SSE/PacketMath.h:219
#1  0x000000010003cb35 in float vector[4] Eigen::internal::ploadt<float vector[4], 1>(Eigen::internal::unpacket_traits<float vector[4]>::type const*) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/GenericPacketMath.h:290
#2  0x000000010003cb18 in float vector[4] Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::packet<1>(long, long) const at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:181
#3  0x000000010003ca24 in void Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1>::copyPacket<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1, 1>(long, long, Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:522
#4  0x000000010003c887 in void Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1>::copyPacketByOuterInner<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1, 1>(long, long, Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:548
#5  0x000000010003c7f2 in Eigen::internal::assign_innervec_CompleteUnrolling<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::Matrix<float, 4, 4, 0, 4, 4>, 0, 16>::run(Eigen::Matrix<float, 4, 4, 0, 4, 4>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:206
#6  0x000000010003c76d in Eigen::internal::assign_impl<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::Matrix<float, 4, 4, 0, 4, 4>, 2, 2, 0>::run(Eigen::Matrix<float, 4, 4, 0, 4, 4>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:342
#7  0x000000010003c6bc in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::lazyAssign<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:499
#8  0x000000010003c5af in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::lazyAssign<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:414
#9  0x000000010003c50d in Eigen::internal::assign_selector<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::Matrix<float, 4, 4, 0, 4, 4>, false, false>::run(Eigen::Matrix<float, 4, 4, 0, 4, 4>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:520
#10 0x000000010003c4d8 in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::_set_noalias<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:621
#11 0x0000000100068b04 in void Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::_set_selector<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::Matrix<float, 4, 4, 0, 4, 4> const&, Eigen::internal::false_type const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:606
#12 0x0000000100068ac1 in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::_set<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:598
#13 0x0000000100065f18 in Eigen::Matrix<float, 4, 4, 0, 4, 4>::operator=(Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Matrix.h:156
#14 0x0000000100068a75 in Eigen::internal::transform_construct_from_matrix<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 2, 0, 3, 4, 4, 4>::run(Eigen::Transform<float, 3, 2, 0>*, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Geometry/Transform.h:1220
#15 0x0000000100068a3a in Eigen::Transform<float, 3, 2, 0>::Transform<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::EigenBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Geometry/Transform.h:274
#16 0x00000001000668dd in Eigen::Transform<float, 3, 2, 0>::Transform<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::EigenBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Geometry/Transform.h:275
#17 0x00000001000668a7 in void pcl::transformPointCloud<pcl::PointXYZRGB, float>(pcl::PointCloud<pcl::PointXYZRGB> const&, pcl::PointCloud<pcl::PointXYZRGB>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/include/pcl-1.7/pcl/common/transforms.h:194
#18 0x0000000100066355 in void pcl::transformPointCloud<pcl::PointXYZRGB>(pcl::PointCloud<pcl::PointXYZRGB> const&, pcl::PointCloud<pcl::PointXYZRGB>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/include/pcl-1.7/pcl/common/transforms.h:203
#19 0x0000000100176d89 in Visualizer::Visualize(std::__1::vector<boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGB> >, std::__1::allocator<boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGB> > > >, std::__1::vector<std::__1::tuple<std::__1::vector<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::aligned_allocator<Eigen::Matrix<float, 4, 4, 0, 4, 4> > >, std::__1::vector<std::__1::vector<pcl::Correspondence, Eigen::aligned_allocator<pcl::Correspondence> >, std::__1::allocator<std::__1::vector<pcl::Correspondence, Eigen::aligned_allocator<pcl::Correspondence> > > > >, std::__1::allocator<std::__1::tuple<std::__1::vector<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::aligned_allocator<Eigen::Matrix<float, 4, 4, 0, 4, 4> > >, std::__1::vector<std::__1::vector<pcl::Correspondence, Eigen::aligned_allocator<pcl::Correspondence> >, std::__1::allocator<std::__1::vector<pcl::Correspondence, Eigen::aligned_allocator<pcl::Correspondence> > > > > > >, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGB> >) at /Users/athundt/source/git/object_recognition/src/Visualizer.cpp:33
#20 0x0000000100037594 in main at /Users/athundt/source/git/object_recognition/pcl_multi_object_recognition.cpp:65
#21 0x00007fff8f2055c9 in start ()

console output

[1;33mFailed to find match for field 'rgb'.
Model read: sphere82mm3in_xyz_to_pcd.pcd
filter 0
all loaded
OpenNI2-FreenectDriver: Using libfreenect v0.5.2
OpenNI2-FreenectDriver: Found device freenect://0
OpenNI initialization successful
freenect://0
OpenNI2-FreenectDriver: Opening device freenect://0
Unable to claim interface -7
[Stream 70] Negotiated packet size 1920
write_register: 0x0105 <= 0x00
write_register: 0x0006 <= 0x00
write_register: 0x0012 <= 0x03
write_register: 0x0013 <= 0x01
write_register: 0x0014 <= 0x1e
write_register: 0x0006 <= 0x02
write_register: 0x0017 <= 0x00
Ir sensor creation successful
Ir activation successful
write_register: 0x0006 <= 0x00
[Stream 70] Negotiated packet size 1920
write_register: 0x0105 <= 0x00
write_register: 0x0006 <= 0x00
write_register: 0x0012 <= 0x03
write_register: 0x0013 <= 0x01
write_register: 0x0014 <= 0x1e
write_register: 0x0006 <= 0x02
write_register: 0x0017 <= 0x00
[Stream 80] Negotiated packet size 1920
write_register: 0x000c <= 0x00
write_register: 0x000d <= 0x01
write_register: 0x000e <= 0x1e
write_register: 0x0005 <= 0x01
[Stream 70] Invalid magic ffff
[Stream 70] Lost 1 packets
[Stream 70] Lost 1 total packets in 8 frames (0.125000 lppf)
[Stream 70] Lost 3 packets
[Stream 70] Lost 4 total packets in 8 frames (0.500000 lppf)
[Stream 70] Inconsistent flag 75 with 238 packets in buf (242 total), resyncing...
[Stream 70] Invalid magic 23ff
[Stream 70] Invalid magic 22ff
[Stream 70] Invalid magic 23ff
[Stream 70] Invalid magic 24ff
[Stream 70] Lost 4 packets
[Stream 70] Lost 8 total packets in 18 frames (0.444444 lppf)
[Stream 70] Lost 3 packets
[Stream 70] Lost 11 total packets in 18 frames (0.611111 lppf)
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 8 packets
[Stream 70] Lost 19 total packets in 18 frames (1.055556 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 7 packets
[Stream 70] Lost 26 total packets in 18 frames (1.444444 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 23ff
[Stream 70] Invalid magic 22ff
[Stream 70] Invalid magic 23ff
[Stream 70] Lost 7 packets
[Stream 70] Lost 33 total packets in 18 frames (1.833333 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 8 packets
[Stream 70] Lost 41 total packets in 18 frames (2.277778 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 22ff
[Stream 70] Invalid magic 25ff
[Stream 70] Invalid magic 24ff
[Stream 70] Invalid magic 24ff
[Stream 70] Lost 7 packets
[Stream 70] Lost 48 total packets in 18 frames (2.666667 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 23ff
[Stream 70] Invalid magic 23ff
[Stream 70] Invalid magic 26ff
[Stream 70] Invalid magic 25ff
[Stream 70] Invalid magic 24ff
[Stream 70] Invalid magic 24ff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 11 packets
[Stream 70] Lost 59 total packets in 18 frames (3.277778 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 2eff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 7 packets
[Stream 70] Lost 66 total packets in 18 frames (3.666667 lppf)
[Stream 70] Lost too many packets, resyncing...
write_register: 0x0047 <= 0x00
Color sensor creation successful
Color sensor activation successful
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 8 packets
[Stream 70] Lost 74 total packets in 32 frames (2.312500 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 80] Invalid magic a8c7
[Stream 80] Lost 1 packets
[Stream 80] Lost 1 total packets in 4 frames (0.250000 lppf)
[Stream 80] Lost 4 packets
[Stream 80] Lost 5 total packets in 4 frames (1.250000 lppf)
[Stream 80] Inconsistent flag 85 with 56 packets in buf (162 total), resyncing...
[Stream 80] Invalid magic 936c
[Stream 80] Lost 1 packets
[Stream 80] Lost 6 total packets in 8 frames (0.750000 lppf)
[Stream 80] Lost 3 packets
[Stream 80] Lost 9 total packets in 8 frames (1.125000 lppf)
[Stream 80] Inconsistent flag 81 with 77 packets in buf (162 total), resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 5 packets
[Stream 70] Lost 79 total packets in 36 frames (2.194444 lppf)
[Stream 70] Inconsistent flag 75 with 202 packets in buf (242 total), resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Lost 1 packets
[Stream 70] Lost 80 total packets in 38 frames (2.105263 lppf)
[Stream 70] Lost 3 packets
[Stream 70] Lost 83 total packets in 38 frames (2.184211 lppf)
[Stream 70] Inconsistent flag 75 with 45 packets in buf (242 total), resyncing...
[Stream 80] Invalid magic e1c9
[Stream 80] Lost 1 packets
[Stream 80] Lost 10 total packets in 9 frames (1.111111 lppf)
[Stream 80] Lost 4 packets
[Stream 80] Lost 14 total packets in 9 frames (1.555556 lppf)
[Stream 80] Inconsistent flag 82 with 161 packets in buf (162 total), resyncing...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 8 packets
[Stream 70] Lost 91 total packets in 40 frames (2.275000 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 80] Invalid magic a696
[Stream 80] Lost 1 packets
[Stream 80] Lost 15 total packets in 10 frames (1.500000 lppf)
[Stream 80] Lost 5 packets
[Stream 80] Lost 20 total packets in 10 frames (2.000000 lppf)
[Stream 80] Inconsistent flag 82 with 161 packets in buf (162 total), resyncing...
calculating model normals... 
model size 119988
calculating scene normals... 
finding uniform sampled keypoints...
    found 2579 keypoints in the scene and 20000 in the model
Starting to cluster...
    Found 0 model instance/instances
giacomodabisias commented 9 years ago

I don't like this : Unable to claim interface -7 Do you see the kinect output in the window? I mean the cloud?

2015-06-13 13:08 GMT+02:00 Andrew Hundt notifications@github.com:

I'm running the program with a simple 3" sphere in the pcd format, and according to the console output below the kinect is loading up successfully. However, it appears the result of (std::get < 0 > (found_models[i]))[0] in the call to pcl::transformPointCloud ((model_list[i]), rotated_model, (std::get < 0 > (found_models[i]))[0]); in visualizer::visualize() called from main() is null. Still need to track the reason for this back to the source. stack trace

note: main is at the bottom, where it finally crashed by accessing a null value is at the top.

0 0x000000010003cb6f in float vector[4] Eigen::internal::pload<float vector[4]>(Eigen::internal::unpackettraits<float vector[4]>::type const) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/arch/SSE/PacketMath.h:219

1 0x000000010003cb35 in float vector[4] Eigen::internal::ploadt<float vector[4], 1>(Eigen::internal::unpackettraits<float vector[4]>::type const) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/GenericPacketMath.h:290

2 0x000000010003cb18 in float vector[4] Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::packet<1>(long, long) const at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:181

3 0x000000010003ca24 in void Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1>::copyPacket<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1, 1>(long, long, Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:522

4 0x000000010003c887 in void Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1>::copyPacketByOuterInner<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1, 1>(long, long, Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:548

5 0x000000010003c7f2 in Eigen::internal::assign_innervec_CompleteUnrolling<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::Matrix<float, 4, 4, 0, 4, 4>, 0, 16>::run(Eigen::Matrix<float, 4, 4, 0, 4, 4>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:206

6 0x000000010003c76d in Eigen::internal::assign_impl<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::Matrix<float, 4, 4, 0, 4, 4>, 2, 2, 0>::run(Eigen::Matrix<float, 4, 4, 0, 4, 4>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:342

7 0x000000010003c6bc in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::lazyAssign<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:499

8 0x000000010003c5af in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::lazyAssign<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:414

9 0x000000010003c50d in Eigen::internal::assign_selector<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::Matrix<float, 4, 4, 0, 4, 4>, false, false>::run(Eigen::Matrix<float, 4, 4, 0, 4, 4>&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Assign.h:520

10 0x000000010003c4d8 in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::_set_noalias<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:621

11 0x0000000100068b04 in void Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::_set_selector<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::Matrix<float, 4, 4, 0, 4, 4> const&, Eigen::internal::false_type const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:606

12 0x0000000100068ac1 in Eigen::Matrix<float, 4, 4, 0, 4, 4>& Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >::_set<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/PlainObjectBase.h:598

13 0x0000000100065f18 in Eigen::Matrix<float, 4, 4, 0, 4, 4>::operator=(Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Core/Matrix.h:156

14 0x0000000100068a75 in Eigen::internal::transform_construct_from_matrix<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 2, 0, 3, 4, 4, 4>::run(Eigen::Transform<float, 3, 2, 0>*, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Geometry/Transform.h:1220

15 0x0000000100068a3a in Eigen::Transform<float, 3, 2, 0>::Transform<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::EigenBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Geometry/Transform.h:274

16 0x00000001000668dd in Eigen::Transform<float, 3, 2, 0>::Transform<Eigen::Matrix<float, 4, 4, 0, 4, 4> >(Eigen::EigenBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> > const&) at /usr/local/Cellar/eigen/3.2.4/include/eigen3/Eigen/src/Geometry/Transform.h:275

17 0x00000001000668a7 in void pcl::transformPointCloud<pcl::PointXYZRGB, float>(pcl::PointCloudpcl::PointXYZRGB const&, pcl::PointCloudpcl::PointXYZRGB&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/include/pcl-1.7/pcl/common/transforms.h:194

18 0x0000000100066355 in void pcl::transformPointCloudpcl::PointXYZRGB(pcl::PointCloudpcl::PointXYZRGB const&, pcl::PointCloudpcl::PointXYZRGB&, Eigen::Matrix<float, 4, 4, 0, 4, 4> const&) at /usr/local/include/pcl-1.7/pcl/common/transforms.h:203

19 0x0000000100176d89 in Visualizer::Visualize(std::1::vectorboost::shared_ptr<pcl::PointCloud, std::__1::allocatorboost::shared_ptr<pcl::PointCloud > >, std::1::vector<std::1::tuple<std::1::vector<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::aligned_allocator<Eigen::Matrix<float, 4, 4, 0, 4, 4> > >, std::1::vector<std::1::vector<pcl::Correspondence, Eigen::aligned_allocator >, std::1::allocator<std::1::vector<pcl::Correspondence, Eigen::aligned_allocator > > > >, std::1::allocator<std::1::tuple<std::1::vector<Eigen::Matrix<float, 4, 4, 0, 4, 4>, Eigen::aligned_allocator<Eigen::Matrix<float, 4, 4, 0, 4, 4> > >, std::1::vector<std::1::vector<pcl::Correspondence, Eigen::aligned_allocator >, std::1::allocator<std::__1::vector<pcl::Correspondence, Eigen::aligned_allocator > > > > > >, boost::shared_ptrpcl::PointCloud) at /Users/athundt/source/git/object_recognition/src/Visualizer.cpp:33

20 0x0000000100037594 in main at /Users/athundt/source/git/object_recognition/pcl_multi_object_recognition.cpp:65

21 0x00007fff8f2055c9 in start ()

console output

[1;33mFailed to find match for field 'rgb'. �[0;mModel read: sphere82mm3in_xyz_to_pcd.pcd �[0;mfilter 0 all loaded OpenNI2-FreenectDriver: Using libfreenect v0.5.2 OpenNI2-FreenectDriver: Found device freenect://0 OpenNI initialization successful freenect://0 OpenNI2-FreenectDriver: Opening device freenect://0 Unable to claim interface -7 [Stream 70] Negotiated packet size 1920 write_register: 0x0105 <= 0x00 write_register: 0x0006 <= 0x00 write_register: 0x0012 <= 0x03 write_register: 0x0013 <= 0x01 write_register: 0x0014 <= 0x1e write_register: 0x0006 <= 0x02 write_register: 0x0017 <= 0x00 Ir sensor creation successful Ir activation successful write_register: 0x0006 <= 0x00 [Stream 70] Negotiated packet size 1920 write_register: 0x0105 <= 0x00 write_register: 0x0006 <= 0x00 write_register: 0x0012 <= 0x03 write_register: 0x0013 <= 0x01 write_register: 0x0014 <= 0x1e write_register: 0x0006 <= 0x02 write_register: 0x0017 <= 0x00 [Stream 80] Negotiated packet size 1920 write_register: 0x000c <= 0x00 write_register: 0x000d <= 0x01 write_register: 0x000e <= 0x1e write_register: 0x0005 <= 0x01 [Stream 70] Invalid magic ffff [Stream 70] Lost 1 packets [Stream 70] Lost 1 total packets in 8 frames (0.125000 lppf) [Stream 70] Lost 3 packets [Stream 70] Lost 4 total packets in 8 frames (0.500000 lppf) [Stream 70] Inconsistent flag 75 with 238 packets in buf (242 total), resyncing... [Stream 70] Invalid magic 23ff [Stream 70] Invalid magic 22ff [Stream 70] Invalid magic 23ff [Stream 70] Invalid magic 24ff [Stream 70] Lost 4 packets [Stream 70] Lost 8 total packets in 18 frames (0.444444 lppf) [Stream 70] Lost 3 packets [Stream 70] Lost 11 total packets in 18 frames (0.611111 lppf) [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 8 packets [Stream 70] Lost 19 total packets in 18 frames (1.055556 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 7 packets [Stream 70] Lost 26 total packets in 18 frames (1.444444 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic 23ff [Stream 70] Invalid magic 22ff [Stream 70] Invalid magic 23ff [Stream 70] Lost 7 packets [Stream 70] Lost 33 total packets in 18 frames (1.833333 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 8 packets [Stream 70] Lost 41 total packets in 18 frames (2.277778 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic 22ff [Stream 70] Invalid magic 25ff [Stream 70] Invalid magic 24ff [Stream 70] Invalid magic 24ff [Stream 70] Lost 7 packets [Stream 70] Lost 48 total packets in 18 frames (2.666667 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic 23ff [Stream 70] Invalid magic 23ff [Stream 70] Invalid magic 26ff [Stream 70] Invalid magic 25ff [Stream 70] Invalid magic 24ff [Stream 70] Invalid magic 24ff [Stream 70] Invalid magic ffff [Stream 70] Lost 11 packets [Stream 70] Lost 59 total packets in 18 frames (3.277778 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic 2eff [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 7 packets [Stream 70] Lost 66 total packets in 18 frames (3.666667 lppf) [Stream 70] Lost too many packets, resyncing... write_register: 0x0047 <= 0x00 Color sensor creation successful Color sensor activation successful [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 8 packets [Stream 70] Lost 74 total packets in 32 frames (2.312500 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 80] Invalid magic a8c7 [Stream 80] Lost 1 packets [Stream 80] Lost 1 total packets in 4 frames (0.250000 lppf) [Stream 80] Lost 4 packets [Stream 80] Lost 5 total packets in 4 frames (1.250000 lppf) [Stream 80] Inconsistent flag 85 with 56 packets in buf (162 total), resyncing... [Stream 80] Invalid magic 936c [Stream 80] Lost 1 packets [Stream 80] Lost 6 total packets in 8 frames (0.750000 lppf) [Stream 80] Lost 3 packets [Stream 80] Lost 9 total packets in 8 frames (1.125000 lppf) [Stream 80] Inconsistent flag 81 with 77 packets in buf (162 total), resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 5 packets [Stream 70] Lost 79 total packets in 36 frames (2.194444 lppf) [Stream 70] Inconsistent flag 75 with 202 packets in buf (242 total), resyncing... [Stream 70] Invalid magic ffff [Stream 70] Lost 1 packets [Stream 70] Lost 80 total packets in 38 frames (2.105263 lppf) [Stream 70] Lost 3 packets [Stream 70] Lost 83 total packets in 38 frames (2.184211 lppf) [Stream 70] Inconsistent flag 75 with 45 packets in buf (242 total), resyncing... [Stream 80] Invalid magic e1c9 [Stream 80] Lost 1 packets [Stream 80] Lost 10 total packets in 9 frames (1.111111 lppf) [Stream 80] Lost 4 packets [Stream 80] Lost 14 total packets in 9 frames (1.555556 lppf) [Stream 80] Inconsistent flag 82 with 161 packets in buf (162 total), resyncing... [Stream 70] Invalid magic ffff [Stream 70] Invalid magic ffff [Stream 70] Lost 8 packets [Stream 70] Lost 91 total packets in 40 frames (2.275000 lppf) [Stream 70] Lost too many packets, resyncing... [Stream 80] Invalid magic a696 [Stream 80] Lost 1 packets [Stream 80] Lost 15 total packets in 10 frames (1.500000 lppf) [Stream 80] Lost 5 packets [Stream 80] Lost 20 total packets in 10 frames (2.000000 lppf) [Stream 80] Inconsistent flag 82 with 161 packets in buf (162 total), resyncing... calculating model normals... model size 119988 calculating scene normals... finding uniform sampled keypoints... found 2579 keypoints in the scene and 20000 in the model Starting to cluster... Found 0 model instance/instances

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-111698779 .

G. Dabisias, PhD Student PERCRO, (Laboratory of Perceptual Robotics) Scuola Superiore Sant'Anna via Luigi Alamanni 13D, San Giuliano Terme 56010 (PI), Italy mob.: +39 3480839095

ahundt commented 9 years ago

No I'm not able to see the kinect output in the window. I googled around and it seems to be a problem with Libusb not being able to access the device.

However when I directly run freenect-glpclview I'm able to see the cloud. Perhaps I should skip to using libfreenect directly? Do you know of any existing code that might be good for that?

giacomodabisias commented 9 years ago

Hi, yes, use directly libfreenect without openni; I stopped using openni a long time ago. I have a grabber at work so I can give it to you on monday if you want.

2015-06-13 18:52 GMT+02:00 Andrew Hundt notifications@github.com:

No I'm not able to see the kinect output in the window. I googled around and it seems to be a problem with Libusb not being able to access the device.

However when I directly run freenect-glpclview I'm able to see the cloud. Perhaps I should skip to using libfreenect directly? Do you know of any existing code that might be good for that?

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-111729410 .

G. Dabisias, PhD Student PERCRO, (Laboratory of Perceptual Robotics) Scuola Superiore Sant'Anna via Luigi Alamanni 13D, San Giuliano Terme 56010 (PI), Italy mob.: +39 3480839095

ahundt commented 9 years ago

That would be fantastic, thanks! Will it be a separate repository or will you just merge the code into this one?

giacomodabisias commented 9 years ago

Separate, but its easy to insert if you want. Remember me on Thursday and I will send it to you. I am making also a simple grabber for the f200 if you need Il 16/giu/2015 20:28, "Andrew Hundt" notifications@github.com ha scritto:

That would be fantastic, thanks! Will it be a separate repository or will you just merge the code into this one?

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-112521850 .

ahundt commented 9 years ago

I'd definitely be interested in both. Could you send me a link to the f200 device you are using in case I decide to buy one?

giacomodabisias commented 9 years ago

http://click.intel.com/intel-realsense-developer-kit.html new version: http://click.intel.com/intel-realsense-developer-kit-r200.html

2015-06-16 22:23 GMT+02:00 Andrew Hundt notifications@github.com:

I'd definitely be interested in both. Could you send me a link to the f200 device you are using in case I decide to buy one?

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-112557719 .

G. Dabisias, PhD Student PERCRO, (Laboratory of Perceptual Robotics) Scuola Superiore Sant'Anna via Luigi Alamanni 13D, San Giuliano Terme 56010 (PI), Italy mob.: +39 3480839095

ahundt commented 9 years ago

Thanks for the links! Looks like the r200 is the longer range one I would be interested in.

Also, you mentioned I should remind you on Thursday about sending the kinect frame grabber. I'd appreciate if you could send it. If it can be put up on github that would be wonderful, but also my email address is ATHundt@gmail.com. Thanks!

giacomodabisias commented 9 years ago

Just created it. There is also an example of use. https://github.com/giacomodabisias/freenect_cloud_grabber

2015-06-18 4:03 GMT+02:00 Andrew Hundt notifications@github.com:

Thanks for the links! Looks like the r200 is the longer range one I would be interested in.

Also, you mentioned I should remind you on Thursday about sending the kinect frame grabber. I'd appreciate if you could send it. If it can be put up on github that would be wonderful, but also my email address is ATHundt@gmail.com. Thanks!

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-113010282 .

G. Dabisias, PhD Student PERCRO, (Laboratory of Perceptual Robotics) Scuola Superiore Sant'Anna via Luigi Alamanni 13D, San Giuliano Terme 56010 (PI), Italy mob.: +39 3480839095

ahundt commented 9 years ago

awesome! I will check it out, thanks

ahundt commented 9 years ago

Ok I have it compiling and will test it when I have a kinect with me to try it out. I'm planning to integrate the library into this repository if that's okay. Would it be best to add it as a sub tree of this one or is there another mechanism you would prefer?

Also should I simply scrap the openNI code or keep it as an option?

giacomodabisias commented 9 years ago

Hi, I would integrate it and remove openni, its just an overly over libfreenect. I will have a look at your changes tomorrow. Have a nice weekend Il 19/giu/2015 23:47, "Andrew Hundt" notifications@github.com ha scritto:

Ok I have it compiling and will test it when I have a kinect with me to try it out. I'm planning to integrate the library into this repository if that's okay. Would it be best to add it as a sub tree of this one or is there another mechanism you would prefer?

Also should I simply scrap the openNI code or keep it as an option?

— Reply to this email directly or view it on GitHub https://github.com/giacomodabisias/object_recognition/issues/2#issuecomment-113652230 .

ahundt commented 9 years ago

Cool, I updated it to work with mac, and added your frame grabber to object_recognition as a subtree with the following command:

git subtree add --prefix=modules/freenect_cloud_grabber git@github.com:ahundt/freenect_cloud_grabber.git master

Don't worry that I used my fork, it will still be compatible with your tree. It just included my commits that added OS X compatibility.

The other key commands you will want to pull in updates are:

git remote add freenect_cloud_grabber https://github.com/giacomodabisias/freenect_cloud_grabber.git
git subtree pull --prefix modules/freenect_cloud_grabber freenect_cloud_grabber master

However, there still seems to be a problem when loading models, somehow they appear to be empty. I have a simple models.txt with the following contents:

sphere82mm3in_xyz_to_pcd.pcd 0.9 10

The pcd file contains a simple sphere with 82mm diameter I made in openscad then converted with meshlab to an intermediate format, then used pcl converters to put it in the pcd format.

With that data I get the following errors:

Failed to find match for field 'rgb'.
Model read: sphere82mm3in_xyz_to_pcd.pcd
filter 0.9
all loaded
upload_firmware: failed to find firmware file.
upload_firmware failed: -2
calculating model normals... 
model size 119988
calculating scene normals... 
[pcl::NormalEstimation::compute] input_ is empty!
finding uniform sampled keypoints...
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
   found 0 keypoints in the scene and 20000 in the model
Starting to cluster...
[pcl::BOARDLocalReferenceFrameEstimation::compute] input_ is empty!
[pcl::BOARDLocalReferenceFrameEstimation::initCompute] Init failed.
   Found 0 model instance/instances

Specifically the line (std::get < 0 > (found_models[i]))[0] in Visualizer.cpp line 33 seems to return a null value.

ahundt commented 9 years ago

Sorry for the double post, but I pushed "Comment" too early and had to edit the previous comment to complete the information.

ahundt commented 9 years ago

Also, I forgot to mention that the frame grabber demo seems to work great! Doesn't connect 100% of the time, but I'm pretty sure that is libfreenect's fault and not a problem in your code. Thanks for creating it!

giacomodabisias commented 9 years ago

Usually when you start libfreenect you need to close it and restart it. No idea why