search

poolio / bilateral_solver

Python code for the fast bilateral solver
MIT License
231 stars 56 forks source link

Pixels with zero confidence!? #2

Closed nischnei closed 7 years ago

nischnei commented 7 years ago

For depth upsampling I would like to get rid of the prior upsampling step (linear or bilinear upsampling e.g. to get the same resolution as the target image). I tried to tackle this by setting the confidences for those inputs to zero. However, due to the following division:

// Flat initialization y0 = self.grid.splat(xw) / w_splat

the operation fails. Any recommendations?

jonbarron commented 7 years ago

I would guess that you're getting a divide-by-zero issue? If that's the case then you can just clamp w_splat to be at least numerical epsilon: something like np.maximum(w_splat.max, 0.00001). Can you clarify if it's a divide-by-zero error?

nischnei commented 7 years ago

Yes, it's a divide-by-zero issue. I was hoping to avoid the "numerical epsilon" as it includes (in my case) default values which are just totally wrong (-1). For zero confidence it would make sense to just incorporate the smoothness term - which would be fine if I regard equation (1) from the paper. I will go through the math and code and hope to provide a fix if anyone has the same issue.

jonbarron commented 7 years ago

I wouldn't worry about clamping to epsilon having the effect of letting zero-confidence pixels change the output. I believe the numerator is actually zero too for these cases, so all you're doing is fixing a divide-by-zero error.

RoyiAvital commented 6 years ago

Is there a place the Optimized Code is available to replicate the speed in the article?

jonbarron commented 6 years ago

No, sorry, that code is too heavily entangled with Google's code repository for us to easily open-source it.

On Wed, Sep 5, 2018 at 12:37 AM Royi notifications@github.com wrote:

Is there a place the Optimized Code is available to replicate the speed in the article?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/poolio/bilateral_solver/issues/2#issuecomment-418630284, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2StoROj_Ro4DfqfW6SU_oscTrlYldAks5uX39GgaJpZM4KexX7 .

RoyiAvital commented 6 years ago

Could you shed light about how was the performance achieved? As in the Python code it seems you explicitly define matrix A which in real world problem might no feasible (Too large).

jonbarron commented 6 years ago

Explicitly defining the A matrix isn't intractable (as evidenced by the fact that this python code runs), it's the W matrix in the paper that's too large and dense to store in memory. In our production code we effectively construct A as well, we just use PCG to approximately solve it rather than using a direct sparse matrix solver as we do in the python reference code.

On Wed, Sep 5, 2018 at 12:17 PM Royi notifications@github.com wrote:

Could you shed light about how was the performance achieved? As in the Python code it seems you explicitly define matrix A which in real world problem might no feasible (Too large).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/poolio/bilateral_solver/issues/2#issuecomment-418849509, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2Stt4ligOyICfkebECey_L4OtKsKzeks5uYCM1gaJpZM4KexX7 .

RoyiAvital commented 6 years ago

I see. I thought A would be also dense matrix.

By the way, the Python Code does use PCG to solve the problem. Though it seems it uses the simpler conditioning if I got it right.

jonbarron commented 6 years ago

Oh, you're right! My mistake. We just didn't implement the hierarchical preconditioning in this version.

On Wed, Sep 5, 2018 at 1:56 PM Royi notifications@github.com wrote:

I see. I thought A would be also dense matrix.

By the way, the Python Code https://github.com/poolio/bilateral_solver/blob/master/notebooks/bilateral_solver.ipynb does use PCG to solve the problem. Though it seems you use the simpler conditioning if I got it right.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/poolio/bilateral_solver/issues/2#issuecomment-418878512, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2StgwOrNOni97Wzm-VKCHdBLcz5bA7ks5uYDpmgaJpZM4KexX7 .