search

Duanyll / CANConv

Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening
GNU General Public License v3.0
36 stars 3 forks source link

some problems about building libKMCUDA.so #3

Open peiyaoooo opened 2 months ago

peiyaoooo commented 2 months ago

Hi Yule, Thank you for your impressive work. But there are some problems when I run the released code. The libKMCUDa.so is built successfully with your guidance. But it faces "segmentation fault (core dumped)" when I import libKMCUDA. It is a tough issue. kmeans_cuda from official KMCUDA function needs the input in the form of numpy array. And you add a torch.cc file to realize the conversion between torch tensor and numpy array, right? you use the PYBIND11_MODULE in torch.cc file and name this conversion function the same name with "kmeans_cuda". I am confused. Is there any crash between the kmeans cuda function in torch.cc and kmcuda.cc? Besides, it seems no command for PYBIND11 in the CMakelists.txt. Is this conversion function really included in the libKMCUDA.so file? Hope for your help and Thank you for your time and attention again.

Duanyll commented 2 months ago

Thanks for your feedback. Can you show me which version of PyTorch and operation system are you using?

From here the numpy binding and pytorch binding won't appear in the same libKMCUDA.so, controlled by the DISABLE_PYTORCH flag in CMake (Not set by default).

Note that you don't need to link to pybind11 in CMakeLists.txt manually, since TORCH_PYTHON_LIBRARY already contains it.

Duanyll commented 2 months ago

I just confirmed that everything works fine in the environment defined by the Dockerfile. Also please use the build.sh to copy the built .so file to correct location.

peiyaoooo commented 1 month ago

I just confirmed that everything works fine in the environment defined by the Dockerfile. Also please use the build.sh to copy the built .so file to correct location.

Thank you for your answer! But I setup the environment with your dockerfile and bulit .so file via "bash build.sh". So the pytorch verision == 2.1.1. The operation system is on the Ubuntu 9.4.0-1ubuntu1~20.04.2.

If convenient, could you share your image with me? I tried many solutions to solve the segmentation fault issue but failed. I also used ldd to check the dependence and nm to check the functions of the .so file. Everything is okay but import failure.

Thank you for your attention and your time!

peiyaoooo commented 1 month ago

Hi yule, I try the official code of KMCUDA in numpy version and the compiled .so file can be imported successfully. So it may be something wrong with pybind_module part I guess.

Duanyll commented 1 month ago

Please notice that when you use the official kmcuda, you have to edit here to load the data to cpu and convert to numpy array before passing it to kmeans_cuda. (torch.cc reduced such redundant data transfer). Also please manually check the changed parts between the forked repo and the official one. The forked repo contains some bugfixes to prevent crashing when processing some inputs.

peiyaoooo commented 1 month ago

Please notice that when you use the official kmcuda, you have to edit here to load the data to cpu and convert to numpy array before passing it to kmeans_cuda. (torch.cc reduced such redundant data transfer). Also please manually check the changed parts between the forked repo and the official one. The forked repo contains some bugfixes to prevent crashing when processing some inputs.

Yes， I got that point. But what is your operation system? Is it same to mine?

The operation system is on the Ubuntu 9.4.0-1ubuntu1~20.04.2

peiyaoooo commented 1 month ago

Hi yule, when I comment the 'centroids' default setting of pybind11_module, the .so file could be compiled without segmentation fault. The error may be caused from "py::arg("centroids") = torch::empty(0)".

Duanyll commented 1 month ago

Thank you for your valuable feedback. I will look into it.

发件人: peiyaoooo @.> 发送时间: 2024年9月22日 21:24 收件人: Duanyll/CANConv @.> 抄送: Duanyll @.>; Comment @.> 主题: Re: [Duanyll/CANConv] some problems about building libKMCUDA.so (Issue #3)

Hi yule, when I comment the 'centroids' default setting of pybind11_module, the .so file could be compiled without segmentation fault. The error may be caused from "py::arg("centroids") = torch::empty(0)".

― Reply to this email directly, view it on GitHubhttps://github.com/Duanyll/CANConv/issues/3#issuecomment-2366787512, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AF5RE4JF76YJYGG4ZRQ4S6LZX3AKHAVCNFSM6AAAAABOO5BQ3KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRWG44DONJRGI. You are receiving this because you commented.Message ID: @.***>