search

flann-lib / flann

Fast Library for Approximate Nearest Neighbors
http://people.cs.ubc.ca/~mariusm/flann
Other
2.24k stars 649 forks source link

Can't get the cuda test working ! #240

Open Emie31 opened 9 years ago

Emie31 commented 9 years ago

Hello !

Thanks a lot for the nice code you provide ! I've just installed it, and I want to use the cuda version of it.. So I first use the cmake option -DBUILD_CUDA_LIB, and I add to change the CUDA_NVCC_FLAGS so that it actually compiles with my architecture (-arch=sm_20). Now I try to get the tests working, but I'm running into an issue the HDF5 lib.. and I don't know how to deal with it !

Here is the error I get:

[100%] Runnint gtest test(s) flann_cuda_test
[==========] Running 8 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from Flann_3D
[ RUN      ] Flann_3D.KDTreeSingleTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeSingleTest (3 ms)
[ RUN      ] Flann_3D.KDTreeCudaTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeCudaTest (2 ms)
[ RUN      ] Flann_3D.TestRadiusSearch
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.TestRadiusSearch (0 ms)
[----------] 3 tests from Flann_3D (5 ms total)

[----------] 5 tests from Flann_3D_Random_Cloud
[ RUN      ] Flann_3D_Random_Cloud.Test1NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.797032 seconds)
Building kd-tree index...done (0.028326 seconds)
Searching KNN...done (0.000142 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:252: Failure
Expected: (precision) >= (0.99), actual: 0 vs 0.99
Precision: 0
[  FAILED  ] Flann_3D_Random_Cloud.Test1NN (832 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.795608 seconds)
Building kd-tree index...done (0.004154 seconds)
Searching KNN...done (0.000139 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:272: Failure
Expected: (precision) >= (0.99), actual: 5e-05 vs 0.99
Precision: 5e-05
[  FAILED  ] Flann_3D_Random_Cloud.Test4NN (801 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NNGpuBuffers
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.794307 seconds)
Building kd-tree index...done (0.003875 seconds)
Searching KNN...Segmentation fault (core dumped)
make[3]: *** [test/CMakeFiles/test_flann_cuda_test] Error 139
make[2]: *** [test/CMakeFiles/test_flann_cuda_test.dir/all] Error 2
make[1]: *** [test/CMakeFiles/test_flann_cuda_test.dir/rule] Error 2
make: *** [test_flann_cuda_test] Error 2

I've tried to install HDF5 using apt-get install libhdf5-mpi-dev, and also by compiling the source (from the library github) but nothing worked!

What am I missing ? I'm really new to this lib.. so I guess I missed some stupid step somewhere... Could you help me on how to have this test working ?

Many Thanks!

hoding123 commented 9 years ago

Hi. It seems that you have written to the wrong guy. Sorry for not being able to help.

Good luck with you hdf5 installation!

2015-05-13 18:42 GMT+08:00 Emie31 notifications@github.com:

Hello !

Thanks a lot for the nice code you provide ! I've just installed it, and I want to use the cuda version of it.. So I first use the cmake option -DBUILD_CUDA_LIB, and I add to change the CUDA_NVCC_FLAGS so that it actually compiles with my architecture (-arch=sm_20). Now I try to get the tests working, but I'm running into an issue the HDF5 lib.. and I don't know how to deal with it !

Here is the error I get:

[100%] Runnint gtest test(s) flann_cuda_test [==========] Running 8 tests from 2 test cases. [----------] Global test environment set-up. [----------] 3 tests from Flann_3D [ RUN ] Flann_3D.KDTreeSingleTest Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:

000: ../../../src/H5D.c line 334 in H5Dopen2(): not found

major: Dataset
minor: Object not found

001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object

major: Symbol table
minor: Object not found

002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed

major: Symbol table
minor: Object not found

003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed

major: Symbol table
minor: Callback failed

004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist

major: Symbol table
minor: Object not found

unknown file: Failure C++ exception with description "Error opening dataset in file." thrown in SetUp(). [ FAILED ] Flann_3D.KDTreeSingleTest (3 ms) [ RUN ] Flann_3D.KDTreeCudaTest Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:

000: ../../../src/H5D.c line 334 in H5Dopen2(): not found

major: Dataset
minor: Object not found

001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object

major: Symbol table
minor: Object not found

002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed

major: Symbol table
minor: Object not found

003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed

major: Symbol table
minor: Callback failed

004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist

major: Symbol table
minor: Object not found

unknown file: Failure C++ exception with description "Error opening dataset in file." thrown in SetUp(). [ FAILED ] Flann_3D.KDTreeCudaTest (2 ms) [ RUN ] Flann_3D.TestRadiusSearch Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:

000: ../../../src/H5D.c line 334 in H5Dopen2(): not found

major: Dataset
minor: Object not found

001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object

major: Symbol table
minor: Object not found

002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed

major: Symbol table
minor: Object not found

003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed

major: Symbol table
minor: Callback failed

004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist

major: Symbol table
minor: Object not found

unknown file: Failure C++ exception with description "Error opening dataset in file." thrown in SetUp(). [ FAILED ] Flann_3D.TestRadiusSearch (0 ms) [----------] 3 tests from Flann_3D (5 ms total)

[----------] 5 tests from Flann_3D_Random_Cloud [ RUN ] Flann_3D_Random_Cloud.Test1NN creating random point cloud (10000 points)...done Building linear index...done (1e-06 seconds) Searching KNN...done (0.797032 seconds) Building kd-tree index...done (0.028326 seconds) Searching KNN...done (0.000142 seconds) /home/guy/builds/flann/test/flann_cuda_test.cu:252: Failure Expected: (precision) >= (0.99), actual: 0 vs 0.99 Precision: 0 [ FAILED ] Flann_3D_Random_Cloud.Test1NN (832 ms) [ RUN ] Flann_3D_Random_Cloud.Test4NN creating random point cloud (10000 points)...done Building linear index...done (1e-06 seconds) Searching KNN...done (0.795608 seconds) Building kd-tree index...done (0.004154 seconds) Searching KNN...done (0.000139 seconds) /home/guy/builds/flann/test/flann_cuda_test.cu:272: Failure Expected: (precision) >= (0.99), actual: 5e-05 vs 0.99 Precision: 5e-05 [ FAILED ] Flann_3D_Random_Cloud.Test4NN (801 ms) [ RUN ] Flann_3D_Random_Cloud.Test4NNGpuBuffers creating random point cloud (10000 points)...done Building linear index...done (1e-06 seconds) Searching KNN...done (0.794307 seconds) Building kd-tree index...done (0.003875 seconds) Searching KNN...Segmentation fault (core dumped) make[3]: * [test/CMakeFiles/test_flann_cuda_test] Error 139 make[2]: * [test/CMakeFiles/test_flann_cuda_test.dir/all] Error 2 make[1]: * [test/CMakeFiles/test_flann_cuda_test.dir/rule] Error 2 make: * [test_flann_cuda_test] Error 2

I've tried to install HDF5 using apt-get install libhdf5-mpi-dev, and also by compiling the source (from the library github) but nothing worked!

What am I missing ? I'm really new to this lib.. so I guess I missed some stupid step somewhere... Could you help me on how to have this test working ?

Many Thanks!

— Reply to this email directly or view it on GitHub https://github.com/mariusmuja/flann/issues/240.

Emie31 commented 9 years ago

Hi again,

Sorry, for my unclear question ! I'm trying to use the GPU version of flann to get all the 3D points inside a sphere. The CPU version of the library is working well, but I can't get the GPU version working.

for the GPU version In my code I just add : 

#define FLANN_USE_CUDA
#include <flann/flann.h>
std::vector<double> v_ptr(vertices.size() * 3);
    for (unsigned int i = 0; i < vertices.size(); i++)
    {
        for (unsigned int j = 0; j < 3; j++)
        {
            v_ptr[i * 3 + j] = vertices[i].p()[j];
        }
    }
    flann::Matrix<double> dataset(&v_ptr[0], vertices.size(), 3);
    flann::Matrix<double> query(&v_ptr[0], vertices.size(), 3);

    flann::Matrix<int> indices(new int[query.rows * 3], query.rows, 3);
    flann::Matrix<double> dists(new double[query.rows * 3], query.rows, 3);
    // construct an randomized kd-tree index using 4 kd-trees
    flann::Index<L2_Simple<double> > index(dataset, flann::KDTreeCuda3dIndexParams());
    index.buildIndex();

and I linked to libflann, libflann_cpp and libflann_cuda but I keep getting the 2 following errors: 

In function `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::~KDTreeCuda3dIndex()':
tmpxft_0000492e_00000000-4_Cuda_GLS.cudafe1.cpp:(.text._ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEED2Ev[_ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEED5Ev]+0x3f): undefined reference to `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::clearGpuBuffers()'
In function `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::buildIndex()':
tmpxft_0000492e_00000000-4_Cuda_GLS.cudafe1.cpp:(.text._ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEE10buildIndexEv[_ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEE10buildIndexEv]+0xaf): undefined reference to `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::uploadTreeToGpu()'

So I was guessing the cuda library might not be working. To check that I compile the flann_cuda_test (and it compiles fine). However when I try to run it, the test doesn't pass: 

guy@chewbacca:~/builds/flann/test$ ./flann_cuda_test 
[==========] Running 8 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from Flann_3D
[ RUN      ] Flann_3D.KDTreeSingleTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeSingleTest (7 ms)
[ RUN      ] Flann_3D.KDTreeCudaTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeCudaTest (2 ms)
[ RUN      ] Flann_3D.TestRadiusSearch
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.TestRadiusSearch (1 ms)
[----------] 3 tests from Flann_3D (10 ms total)

[----------] 5 tests from Flann_3D_Random_Cloud
[ RUN      ] Flann_3D_Random_Cloud.Test1NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.727318 seconds)
Building kd-tree index...done (0.035195 seconds)
Searching KNN...done (0.000147 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:252: Failure
Expected: (precision) >= (0.99), actual: 0 vs 0.99
Precision: 0
[  FAILED  ] Flann_3D_Random_Cloud.Test1NN (764 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.727468 seconds)
Building kd-tree index...done (0.004335 seconds)
Searching KNN...done (0.000138 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:272: Failure
Expected: (precision) >= (0.99), actual: 2.5e-05 vs 0.99
Precision: 2.5e-05
[  FAILED  ] Flann_3D_Random_Cloud.Test4NN (733 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NNGpuBuffers
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.725263 seconds)
Building kd-tree index...done (0.004063 seconds)
Searching KNN...Segmentation fault (core dumped)

but it looks like it is a different issue.. Do you have any idea of what I'm doing wrong and on how I can get the cuda lib working ?

Thanks, Emilie.

Emie31 commented 9 years ago

Sorry for all the messages.. I finally found my mistake in my code. We cannot used the cuda kdtree with double, but only with float (as there is no implementation of the class for the double template in the library) so I changed my code so that : 

 flann::Index<L2_Simple<double> > index(dataset, flann::KDTreeCuda3dIndexParams());

becomes :

 flann::Index<L2_Simple<float> > index(dataset, flann::KDTreeCuda3dIndexParams());

and it compiles now.

alexsusu commented 9 years ago

Hello, I followed your advice from this thread and I was able to compile the flann CUDA library.

Basically I gave:
  unset CPLUS_INCLUDE_PATH
  export CUDA_NVCC_FLAGS=-arch=sm_20
  cmake -DBUILD_CUDA_LIB=ON .
  make

(I also need to have installed Google Test framework - e.g., from https://github.com/google/googletest )

I am curious - if I understand correctly (from the source file name kdtree_cuda_3d_index.cu, etc) the KDtree is written for 3 dimensions. Can I easily extend it to 4 dimensions?

Thanks, Alex

Leo-LiHao commented 7 years ago

Hello, I got the same error! I found that in flann_cuda_test.cu: 117 flann::load_from_file(match,"cloud.h5","indices"); but

$ h5ls -r cloud.h5 
/                        Group 
/dataset                 Dataset {177264, 3}
/dataset_padded          Dataset {177264, 5}
/distances               Dataset {19696, 5}
/match                   Dataset {19696, 5}
/query                   Dataset {19696, 3}

There doesn't exist 'indices'