Doesn't work ( Propagation )

[Disa-Kizonda]

[ WARN:0@0.026] global loadsave.cpp:248 cv::findDecoder imread('./cache/images/32.jpg'): can't open/read file: check file path/integrity [ WARN:0@0.027] global loadsave.cpp:248 cv::findDecoder imread('./cache/depths/32.png'): can't open/read file: check file path/integrity

I have only 0-4.jpg in /images and 0.png in /depths

[GongMingCarmen]

same question!!!

[nk4517]

costs.dmb
depths.dmb
normals.dmb

These files should be generated by the Propagation submodule executable if it successfully completes.

graphics_utils.py

        cv2.imwrite(os.path.join(cdata_image_path, str(idx+1)+".jpg"), src_img)
        write_cam_txt(os.path.join(cdata_camera_path, str(idx+1)+".txt"), src_K.detach().cpu().numpy(), src_w2c.detach().cpu().numpy(),
                                                                            [depth_min, (depth_max-depth_min)/192.0, 192.0, depth_max])
    # c++ api for depth propagation
    propagation_command = r'./submodules/Propagation/Propagation ./cache 0 "1 2 3 4" ' + str(patch_size)
    os.system(propagation_command)

def generate_edge_mask(propagated_depth, patch_size):
    # img gradient

The code directly executes an Propagation executable through os.system() without verifying if the utility exists or contemplating the result of this operation.

Here's my hack for compiling the Propagation submodule with MSVC 2022, along with compiled .exe and OpenCV .dll files https://github.com/nk4517/GaussianPro/tree/propagation_win32

[Disa-Kizonda]

@1

For some reason, when I add this code to main.cpp it starts working without errors and saves .dmb files If I remove it, it gives errors again

[kcheng1021]

Could you please provide your hardware and environment information? I would like to reproduce the bug and work towards resolving the issue as quickly as possible.

[Disa-Kizonda]

@kcheng1021 windows 10 1050ti cuda 11.8

opencv 4.9.0 python 3.11.1

no conda

[Disa-Kizonda]

Still errors

[kcheng1021]

I believe there might be an issue with reading the file names, possibly due to different character settings on different machines. To troubleshoot this, could you please try adding the following code after line 99 in the file "/submodule/Propagation/main.cpp": ref_id = 0; src_ids_str = "1 2 3 4"; This will hardcode the values to see if training is successful. @Disa-Kizonda

[kcheng1021]

Still errors

Please provide me with the error information. I will also find a windows computer to try to reproduce the bug.

[Disa-Kizonda]

Same as before PS D:\dasdsa\GaussianPro> .\submodules\Propagation\Propagation.exe ./cache 0 "1 2 3 4" 20

[ WARN:0@0.018] global loadsave.cpp:248 cv::findDecoder imread_('./cache/images/32.jpg'): can't open/read file: check file path/integrity

[ WARN:0@0.018] global loadsave.cpp:248 cv::findDecoder imread_('./cache/depths/32.png'): can't open/read file: check file path/integrity

OpenCV: terminate handler is called! The last OpenCV error is: OpenCV(4.9.0) Error: Bad argument (Matrix operand is an empty matrix.) in cv::checkOperandsExist, file C:\GHA-OCV-1_work\ci-gha-workflow\ci-gha-workflow\opencv\modules\core\src\matrix_expressions.cpp, line 24

But when i add this on line 104 std::cout << problem.ref_image_id << std::endl; it's starting to work and saves dmb files

Propagation.zip

[kcheng1021]

1. Can you please print the problem.ref_image_id at line 425 of Propagation.cpp? Let's see when the number 32 appears.
2. So, if you add something at line 104 and retrain, is the training finished?
3. the VRAM of a 1050ti might not be sufficient to complete a training session with 30,000 iterations. Similar to 3DGS, which requires 24GB of VRAM, if you want to successfully train on a 1050ti, you may need to reduce the number of training images and increase the value of percent_dense.

[Disa-Kizonda]

PS D:\dasdsa\GaussianPro> .\submodules\Propagation\Propagation.exe ./cache 0 "1 2 3 4" 20 32 [ WARN:0@0.018] global loadsave.cpp:248 cv::findDecoder imread('./cache/images/32.jpg'): can't open/read file: check file path/integrity [ WARN:0@0.019] global loadsave.cpp:248 cv::findDecoder imread('./cache/depths/32.png'): can't open/read file: check file path/integrity

Yes if i run train.py training is complete

And thanks for the advice with percent_dense

[kcheng1021]

You may need to debug further to identify where exactly outputs the number 32, instead of simply using cout to print it. I am also trying to reproduce this bug and provide you with a prompt response as soon as possible.

[Disa-Kizonda]

I changed ref_id >> problem.ref_image_id; to problem.ref_image_id = ref_id; in main.cpp like here https://github.com/kcheng1021/GaussianPro/issues/8#issuecomment-1976135963

now it's working

[kcheng1021]

I changed ref_id >> problem.ref_image_id; to problem.ref_image_id = ref_id; in main.cpp like here #8 (comment)

now it's working

Yeah! I just found it too. This could be the differences in features between different versions of C++. Thank you very much for your sharing and debuging. So if you found the issue is solved, Please close the issue.

[Disa-Kizonda]

Thank you

[kcheng1021]

@Disa-Kizonda also thanks for your help to improve the project.

[rwijmashell]

Hi, I'm still having this issue, even after applying this fix. Is there any way to find out what's causing this? The docker build seems to be successful.

This is my setup (Docker image on Azure ML):

Python 3.9 Cuda 11.8 Pytorch 2.0.1 GPU: Nvidia A100

[kcheng1021]

Hi, I'm still having this issue, even after applying this fix. Is there any way to find out what's causing this? The docker build seems to be successful.

This is my setup (Docker image on Azure ML):

Python 3.9 Cuda 11.8 Pytorch 2.0.1 GPU: Nvidia A100

Did you use this fix? https://github.com/kcheng1021/GaussianPro/issues/8#issuecomment-1976686681

[rwijmashell]

This is what it looks like (after seeing the fix here I simply pulled in the new changes you made in one of your latest commits):

    Problem problem;
    problem.ref_image_id = ref_id;

[rwijmashell]

@kcheng1021 This is the docker build log, does this indicate any issues?

13 [ 9/11] RUN cd GaussianPro/submodules/Propagation && cmake . && make

13 0.572 CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):

13 0.572 Compatibility with CMake < 3.5 will be removed from a future version of

13 0.572 CMake.

13 0.572

13 0.572 Update the VERSION argument value or use a ... suffix to tell

13 0.572 CMake that the project does not need compatibility with older versions.

13 0.572

13 0.572

13 0.788 -- The C compiler identification is GNU 11.4.0

13 0.866 -- The CXX compiler identification is GNU 11.4.0

13 0.885 -- Detecting C compiler ABI info

13 0.954 -- Detecting C compiler ABI info - done

13 0.967 -- Check for working C compiler: /usr/bin/cc - skipped

13 0.968 -- Detecting C compile features

13 0.968 -- Detecting C compile features - done

13 0.981 -- Detecting CXX compiler ABI info

13 1.047 -- Detecting CXX compiler ABI info - done

13 1.057 -- Check for working CXX compiler: /usr/bin/c++ - skipped

13 1.057 -- Detecting CXX compile features

13 1.058 -- Detecting CXX compile features - done

13 1.059 CMake Warning (dev) at CMakeLists.txt:4 (find_package):

13 1.059 Policy CMP0146 is not set: The FindCUDA module is removed. Run "cmake

13 1.059 --help-policy CMP0146" for policy details. Use the cmake_policy command to

13 1.059 set the policy and suppress this warning.

13 1.059

13 1.059 This warning is for project developers. Use -Wno-dev to suppress it.

13 1.059

13 1.073 -- Performing Test CMAKE_HAVE_LIBC_PTHREAD

13 1.129 -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success

13 1.131 -- Found Threads: TRUE

13 1.141 -- Found CUDA: /usr/local/cuda (found suitable version "11.8", minimum required is "6.0")

13 1.153 -- Found OpenCV: /usr (found version "4.5.4")

13 1.335 -- Found OpenMP_C: -fopenmp (found version "4.5")

13 1.400 -- Found OpenMP_CXX: -fopenmp (found version "4.5")

13 1.400 -- Found OpenMP: TRUE (found version "4.5")

13 1.405 -- Configuring done (0.8s)

13 1.412 -- Generating done (0.0s)

13 1.412 -- Build files have been written to: /app/GaussianPro/submodules/Propagation

13 1.429 [ 25%] Building NVCC (Device) object CMakeFiles/Propagation.dir/Propagation_generated_Propagation.cu.o

13 2.664 /app/GaussianPro/submodules/Propagation/Propagation.cu(329): warning #550-D: variable "cost" was set but never used

13 2.664

13 5.152 ptxas info : 218048 bytes gmem, 72 bytes cmem[3]

13 5.152 ptxas info : Compiling entry function '_Z14RedPixelFilterPK6CameraP6float4Pf' for 'sm_86'

13 5.152 ptxas info : Function properties for _Z14RedPixelFilterPK6CameraP6float4Pf

13 5.152 88 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads

13 5.152 ptxas info : Used 29 registers, 376 bytes cmem[0]

13 5.152 ptxas info : Compiling entry function '_Z16BlackPixelFilterPK6CameraP6float4Pf' for 'sm_86'

13 5.152 ptxas info : Function properties for _Z16BlackPixelFilterPK6CameraP6float4Pf

13 5.152 88 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads

13 5.152 ptxas info : Used 29 registers, 376 bytes cmem[0]

13 5.152 ptxas info : Compiling entry function '_Z17GetDepthandNormalP6CameraP6float416PatchMatchParams' for 'sm_86'

13 5.152 ptxas info : Function properties for _Z17GetDepthandNormalP6CameraP6float416PatchMatchParams

13 5.152 0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads

13 5.152 ptxas info : Used 21 registers, 424 bytes cmem[0]

13 5.152 ptxas info : Compiling entry function '_Z14RedPixelUpdateP18cudaTextureObjectsS0_P6CameraP6float4PfP17curandStateXORWOWPj16PatchMatchParamsi' for 'sm_86'

13 5.152 ptxas info : Function properties for _Z14RedPixelUpdateP18cudaTextureObjectsS0_P6CameraP6float4PfP17curandStateXORWOWPj16PatchMatchParamsi

13 5.152 1856 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads

13 5.152 ptxas info : Used 111 registers, 468 bytes cmem[0], 16 bytes cmem[2]

13 5.152 ptxas info : Compiling entry function '_Z16BlackPixelUpdateP18cudaTextureObjectsS0_P6CameraP6float4PfP17curandStateXORWOWPj16PatchMatchParamsi' for 'sm_86'

13 5.152 ptxas info : Function properties for _Z16BlackPixelUpdateP18cudaTextureObjectsS0_P6CameraP6float4PfP17curandStateXORWOWPj16PatchMatchParamsi

13 5.152 1856 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads

13 5.152 ptxas info : Used 111 registers, 468 bytes cmem[0], 16 bytes cmem[2]

13 5.152 ptxas info : Compiling entry function 'Z20RandomInitializationP18cudaTextureObjectsP6CameraP6float4PfP17curandStateXORWOWPj16PatchMatchParamsS5' for 'sm_86'

13 5.152 ptxas info : Function properties for Z20RandomInitializationP18cudaTextureObjectsP6CameraP6float4PfP17curandStateXORWOWPj16PatchMatchParamsS5

13 5.152 256 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads

13 5.152 ptxas info : Used 76 registers, 464 bytes cmem[0]

13 7.212 [ 50%] Building CXX object CMakeFiles/Propagation.dir/Propagation.cpp.o

13 7.865 /app/GaussianPro/submodules/Propagation/Propagation.cpp: In function ‘void StoreColorPlyFileBinaryPointCloud(const string&, const std::vector&)’:

13 7.865 /app/GaussianPro/submodules/Propagation/Propagation.cpp:354:41: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘std::vector::size_type’ {aka ‘long unsigned int’} [-Wformat=]

13 7.865 354 | fprintf(outputPly, "element vertex %d\n",pc.size());

13 7.865 | ~^ ~~~~~

13 7.865 | | |

13 7.865 | int std::vector::size_type {aka long unsigned int}

13 7.865 | %ld

13 7.872 /app/GaussianPro/submodules/Propagation/Propagation.cpp: In member function ‘void Propagation::CudaSpaceInitialization(const string&, const Problem&)’:

13 7.872 /app/GaussianPro/submodules/Propagation/Propagation.cpp:503:62: warning: unused parameter ‘dense_folder’ [-Wunused-parameter]

13 7.872 503 | void Propagation::CudaSpaceInitialization(const std::string &dense_folder, const Problem &problem)

13 7.872 | ~~~~~^~~~~~

13 7.872 /app/GaussianPro/submodules/Propagation/Propagation.cpp:503:91: warning: unused parameter ‘problem’ [-Wunused-parameter]

13 7.872 503 | void Propagation::CudaSpaceInitialization(const std::string &dense_folder, const Problem &problem)

13 7.872 | ~~~^~~

13 8.121 [ 75%] Building CXX object CMakeFiles/Propagation.dir/main.cpp.o

13 8.797 /app/GaussianPro/submodules/Propagation/main.cpp: In function ‘void ProcessProblem(const string&, const Problem&, bool, int)’:

13 8.797 /app/GaussianPro/submodules/Propagation/main.cpp:48:9: warning: variable ‘temp’ set but not used [-Wunused-but-set-variable]

13 8.797 48 | int temp = pro.GetPatchSize();

13 8.797 | ^~~~

13 9.017 [100%] Linking CXX executable Propagation

13 9.455 [100%] Built target Propagation

[kcheng1021]

It seems that there are many warnings and infos appearing that I didn't encounter during compilation. I suggest running it in the environment that we have tested.

[kcheng1021]

Besides, make sure the compiled propagation is located in the path "submodule/Propagation/". @rwijmashell