Crash when transposing A matrix

[patriciogonzalezvivo]

Hi, the main.py python script crashes at this line part of compute_linear_system_matrices() as part of the poisson blending process.

    self.eigen_solver.A = self.A.transpose().dot(self.A)

Any idea how should I debug or solve it?

[patriciogonzalezvivo]

Breaking this up in steps, seams crash happens on the dot product

        blocks.append(self.fidelity_weight * scipy.sparse.eye(blocks[0].shape[1]))
        mat_A = self.concatenate_csr_matrices_by_row(blocks)
        self.A = mat_A
        if self.eigen_solver is not None:
            print("A_ = self.A")
            A_ = self.A
            print("self.A.transpose().dot(A_)")
            B_ = A_.transpose()
            print("self.dot")
            C_ = B_.dot(self.A)
            print("Assign it to self.eigen_solver.A")
            self.eigen_solver.A = C_

        # self.x_grad_mat = x_grad_mat
        # self.y_grad_mat = y_grad_mat
        print("Done computing matrices")

Produce

I1228 12:35:09.807940  6531 depthmap_stitcher.cpp:233] Output the aligned coefficients to file :data/debug/data/debug/depthmapAlignPy_coeff.json
return alignment coefficient and depth maps...
2023-12-28 12:35:09,821 - __main__ - INFO - 4) blender to ERP image
2023-12-28 12:35:09,821 - utility.blending - DEBUG - stitch the tangent image 0
2023-12-28 12:35:09,939 - utility.blending - DEBUG - stitch the tangent image 1
2023-12-28 12:35:10,051 - utility.blending - DEBUG - stitch the tangent image 2
2023-12-28 12:35:10,162 - utility.blending - DEBUG - stitch the tangent image 3
2023-12-28 12:35:10,271 - utility.blending - DEBUG - stitch the tangent image 4
2023-12-28 12:35:10,381 - utility.blending - DEBUG - stitch the tangent image 5
2023-12-28 12:35:10,417 - utility.blending - DEBUG - stitch the tangent image 6
2023-12-28 12:35:10,452 - utility.blending - DEBUG - stitch the tangent image 7
2023-12-28 12:35:10,488 - utility.blending - DEBUG - stitch the tangent image 8
2023-12-28 12:35:10,521 - utility.blending - DEBUG - stitch the tangent image 9
2023-12-28 12:35:10,557 - utility.blending - DEBUG - stitch the tangent image 10
2023-12-28 12:35:10,593 - utility.blending - DEBUG - stitch the tangent image 11
2023-12-28 12:35:10,629 - utility.blending - DEBUG - stitch the tangent image 12
2023-12-28 12:35:10,667 - utility.blending - DEBUG - stitch the tangent image 13
2023-12-28 12:35:10,702 - utility.blending - DEBUG - stitch the tangent image 14
2023-12-28 12:35:10,739 - utility.blending - DEBUG - stitch the tangent image 15
2023-12-28 12:35:10,856 - utility.blending - DEBUG - stitch the tangent image 16
2023-12-28 12:35:10,969 - utility.blending - DEBUG - stitch the tangent image 17
2023-12-28 12:35:11,078 - utility.blending - DEBUG - stitch the tangent image 18
2023-12-28 12:35:11,191 - utility.blending - DEBUG - stitch the tangent image 19
A_ = self.A
self.A.transpose().dot(A_)
self.dot
[1]    6531 killed     python main.py -i data/0001.jpg --persp_monodepth=midas2 

Any ideas what could be happening here?

[manurare]

Hello,

It is weird that it doesn't throw anything more descriptive. My guess is that you are running out of RAM. For example if I run this code in my laptop it also gets killed:

A = scipy.sparse.diags(np.arange(100000000))
A.transpose().dot(A)

Of course this matrix would belong to an image of 100 megapixels. But it only stores 1 float per row. I am not sure about the sizes of your images, but the A matrix for Poisson blending stores more than 1 float per row.

[patriciogonzalezvivo]

I'm using this image which is the same provided by on the original repo as an example

2023-12-29 12:50:18,502 - utility.blending - DEBUG - stitch the tangent image 18
2023-12-29 12:50:18,628 - utility.blending - DEBUG - stitch the tangent image 19
self.A (85983232, 2097152)
transpose()
(2097152, 85983232)
dot()
[1]    6341 killed     python main.py -i data/0001.jpg --persp_monodepth=midas2 

The crash happens in at the dot product. A matrix before that operation is (85983232, 2097152). Do you mind telling me what is this transposition and dot product about?

[manurare]

Hi. For the solver you need a square A matrix. That's why you need to multiply A by its transpose