Gaps at brick boundaries in deep volume rendering

[digbeta]

This appears to have been opened previously with issue #9 and still appears to be present. I see this even after calling UpdateApron() as noted in issue #9.

I've seen this in my code and also verified it's present in the latest commit in the 3D print example. See the image below, where you can see the vertical and horizontal line along brick boundaries:

Here's another view, showing before topology is visble, note the dotted vertical line visible on what appears to be the brick boundary:

After enabling topology:

[digbeta]

Is it possible that the brick functions aren't being called when a ray hits what it thinks is an apron boundary? Maybe something here:

        b = (((int(dda.p.z) << gvdb->dim[lev]) + int(dda.p.y)) << gvdb->dim[lev]) + int(dda.p.x);   // bitmaskpos
        if ( isBitOn ( gvdb, node, b ) ) {                      // check vdb bitmask for voxel occupancy        

I'm looking into things and my guess is that it's not an issue with the brick function itself, but possibly we're hitting a boundary condition incorrectly and not firing the brick function during the DDA...

Any ideas?

[NBickford-NV]

Hi digbeta!

I have a few ideas on what this issue might be due to, but unfortunately I might not have time to get to it this week. Just wanted to give you a heads-up that I've seen this, though!

(Edit: Also a duplicate of issue #75)

[digbeta]

No problem - thanks, Neil!

On Mon, Aug 3, 2020 at 5:13 PM Neil Bickford notifications@github.com wrote:

Hi digbeta!

I have a few ideas on what this issue might be due to, but unfortunately I might not have time to get to it this week. Just wanted to give you a heads-up that I've seen this, though!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NVIDIA/gvdb-voxels/issues/99#issuecomment-668244795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AECSDEPP3JMDIX4GXW6Y6YDR64SAPANCNFSM4POCOHDA .

[icoderaven]

Pitching in to say that I've seen the visual artifact on my side as well!

[digbeta]

Hi, @NeilBickford-NV -- any chance you've had some time to look into this and/or share some ideas on how to address the gaps? Thanks in advance!

[NBickford-NV]

Hi digbeta! Unfortunately, I haven't had time to look into this yet; my best guess is that it's probably going to be something in https://github.com/NVIDIA/gvdb-voxels/blob/master/source/gvdb_library/kernels/cuda_gvdb_dda.cuh or the procedure in rayCast in https://github.com/NVIDIA/gvdb-voxels/blob/master/source/gvdb_library/kernels/cuda_gvdb_raycast.cuh.

[digbeta]

Got it - thanks, Neil!

[NBickford-NV]

Sorry for the long wait on this (had to cycle around to some other tasks first), but I think I've managed to fix two of the main sources of artifacts here! They wound up being in a place I didn't expect - within the brick raymarching methods themselves. Here's a screenshot of what the first view in this thread looks like with these bugs fixed:

There are still some artifacts that can be found on gDepthMap and gInteractiveGL; these appear to be due to the value of epsilon (which is used in rayCast to nudge samples inside nodes), and setting a larger value of epsilon using VolumeGVDB::SetEpsilon(0.01f, 256) appears to solve these issues.

However, I have some ideas for a different version of HDDAState that might remove the need to specify epsilon entirely, at the expense of possibly requiring functions that look at the internal variables of HDDAState to be changed.

Here's a longer explanation, from the commit text (https://github.com/NVIDIA/gvdb-voxels/commit/88a0895f74ccb3e9698c8267d1d03b41a3b7190c):

For the artifacts in issue #99, it turned out that there were two issues in raySurfaceTrilinearBrick. One was that a line that quantized samples to discrete increments of t was commented out. Since the value of t when entering a brick otherwise depends on where within a voxel the ray entered the brick (I think), this led to visible bands the size of voxels. rayDeepBrick also had this problem, except there the issue was that quantization was performed after determining the voxel position, instead of before.

The other issue in raySurfaceTrilinearBrick was that the loop to skip empty voxels inside bricks (which appears to have been intended as an optimization, but I'm not sure how effective it was) was incorrect - it would always step by SCN_DIRECTSTEP at least once, even if the initial sample was inside a brick.

Together, these two issues appear to have been responsible for visible banding along the planes separating bricks, and for visible gaps between bricks in the volume view of g3DPrint and several other samples.

However, volume rendering isn't totally artifact-free on all samples: in some samples, positioning one's camera just right and at a particular range of distances - such that one of the planes between bricks is greatly foreshortened - will show floating-point glitches between bricks. (This only seems to happen for some planes and on some samples - e.g. I haven't seen it on g3DPrint yet - but is possible to reliably reproduce on gDepthMap and gInteractiveGL). This appears to be due to the value of epsilon, which is used in rayCast to move samples slightly into nodes. In these cases, setting epsilon to a larger value - e.g. 0.01f - using VolumeGVDB::SetEpsilon(0.01f, 256) - appears to fix these artifacts, although it adds some subtle banding artifacts that look like circles in gDepthMap. As such, I've only applied this fix to gResample and gSprayDeposit, where it doesn't appear to add new visible artifacts.

I'll probably take a look at replacing HDDAState with a new approach that expresses index-space position differently; hopefully this should avoid having to use an epsilon, although functions that access HDDAState's class members directly will probably have to be modified.

This also fixes some problems with gSprayDeposit! It turns out that some artifacts were because not all aprons were being updated. Another issue is that gvdbRaytrace ignored the shading method passed to it and uses tricubic intersection instead. Because tricubic intersection technically requires an apron size of 2 (which isn't supported at the moment), this lead to rays sometimes intersecting the boundaries between bricks, which presented as adding density to spaces in the air. Changing gvdbRaytrace to use trilinear intersection instead fixes this - but the correct solution would be to respect the shading parameter passed to the function.

Also note that because we now correctly quantize t-values, one can see circular bands around the camera. These should go away with smaller steps, or maybe there's a way to hide them by e.g. offsetting each pixel's t-value within each brick by a random value (e.g. using blue noise!). Though I also wonder if those sorts of fixes maybe fall within the scope of using a different type of volume rendering approach entirely (e.g. using a Monte Carlo approach instead of a deterministic approach).

[digbeta]

Thanks, Neil! I appreciate you digging in and tracking this down. I hadn’t had a chance to dig into it either so thanks for the efforts getting it resolved.

Thank you!

On Wed, Nov 11, 2020 at 11:30 PM Neil Bickford notifications@github.com wrote:

Sorry for the long wait on this (had to cycle around to some other tasks first), but I think I've managed to fix two of the main sources of artifacts here! They wound up being in a place I didn't expect - within the brick raymarching methods themselves. Here's a screenshot of what the first view in this thread looks like with these bugs fixed:

[image: image] https://user-images.githubusercontent.com/57467222/98895295-a263b780-245b-11eb-811e-969c9f3c091d.png

There are still some artifacts that can be found on gDepthMap and gInteractiveGL; these appear to be due to the value of epsilon (which is used in rayCast to nudge samples inside nodes), and setting a larger value of epsilon using VolumeGVDB::SetEpsilon(0.01f, 256) appears to solve these issues.

However, I have some ideas for a different version of HDDAState that might remove the need to specify epsilon entirely, at the expense of possibly requiring functions that look at the internal variables of HDDAState to be changed.

Here's a longer explanation, from the commit text (88a0895 https://github.com/NVIDIA/gvdb-voxels/commit/88a0895f74ccb3e9698c8267d1d03b41a3b7190c ):

For the artifacts in issue #99 https://github.com/NVIDIA/gvdb-voxels/issues/99, it turned out that there were two issues in raySurfaceTrilinearBrick.

One was that a line that quantized samples to discrete increments of t was commented out. Since the

value of t when entering a brick otherwise depends on where within a voxel the ray entered the brick

(I think), this led to visible bands the size of voxels. rayDeepBrick also had this problem, except

there the issue was that quantization was performed after determining the voxel position, instead of

before.

The other issue in raySurfaceTrilinearBrick was that the loop to skip empty voxels inside bricks

(which appears to have been intended as an optimization, but I'm not sure how effective it was)

was incorrect - it would always step by SCN_DIRECTSTEP at least once, even if the initial sample

was inside a brick.

Together, these two issues appear to have been responsible for visible banding along the planes

separating bricks, and for visible gaps between bricks in the volume view of g3DPrint and several

other samples.

However, volume rendering isn't totally artifact-free on all samples: in some samples, positioning

one's camera just right and at a particular range of distances - such that one of the planes between

bricks is greatly foreshortened - will show floating-point glitches between bricks. (This only seems

to happen for some planes and on some samples - e.g. I haven't seen it on g3DPrint yet - but is

possible to reliably reproduce on gDepthMap and gInteractiveGL). This appears to be due to the

value of epsilon, which is used in rayCast to move samples slightly into nodes. In these cases,

setting epsilon to a larger value - e.g. 0.01f - using VolumeGVDB::SetEpsilon(0.01f, 256) - appears

to fix these artifacts, although it adds some subtle banding artifacts that look like circles in gDepthMap.

As such, I've only applied this fix to gResample and gSprayDeposit, where it doesn't appear to add new

visible artifacts.

I'll probably take a look at replacing HDDAState with a new approach that expresses index-space position

differently; hopefully this should avoid having to use an epsilon, although functions that access HDDAState's

class members directly will probably have to be modified.

This also fixes some problems with gSprayDeposit! It turns out that some artifacts were because not

all aprons were being updated. Another issue is that gvdbRaytrace ignored the shading method passed

to it and uses tricubic intersection instead. Because tricubic intersection technically requires an

apron size of 2 (which isn't supported at the moment), this lead to rays sometimes intersecting the

boundaries between bricks, which presented as adding density to spaces in the air. Changing gvdbRaytrace

to use trilinear intersection instead fixes this - but the correct solution would be to respect the

shading parameter passed to the function.

Also note that because we now correctly quantize t-values, one can see circular bands around the camera. These should go away with smaller steps, or maybe there's a way to hide them by e.g. offsetting each pixel's t-value within each brick by a random value (e.g. using blue noise!). Though I also wonder if those sorts of fixes maybe fall within the scope of using a different type of volume rendering approach entirely (e.g. using a Monte Carlo approach instead of a deterministic approach).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NVIDIA/gvdb-voxels/issues/99#issuecomment-725828967, or unsubscribe https://github.com/notifications/unsubscribe-auth/AECSDENLKP2LG6SZQYSM5ETSPNQGHANCNFSM4POCOHDA .

[digbeta]

Hi, Neil,

I went ahead and started testing out the changes specifically on the rayDeepBrick function. I can confirm the gaps go away, but I did also then get circular bands as you explained in your notes. However, when I went ahead and changed direct step to be a smaller value (0.5 -> 0.1 or 0.01), the banding goes away, but more artifacts reappear on brick boundaries again. Here's an image with the following steps() call:

m_scene->SetSteps(0.1f, 16, 0.1f); and SetEpsilon(0.01, 256);

Here's the box with a value of 0.2f for directstep:

The banding isn't terrible.... but not great. Any ideas for correcting this? Thank you!

[NBickford-NV]

Sorry for the wait on this! The answer's a bit long.

For the banding, one solution would be to offset the t-values of each ray within the brick traversal function by a random value (e.g. using white noise - or even better, blue noise). This is sort of the same trick as in percentage-closer filtering with shadows, where it breaks up banding by replacing it with noise. This would probably be easiest implemented in a fork (or especially if you have your own brick traversal function that you're passing as a pointer!) I'm not 100% sure if GVDB was intended to have randomness, but if it's allowed, then there's a lot of neat things that can be done with Monte Carlo sampling (e.g. delta tracking can be used to render translucent volumes more noisy, but much less expensively).

For the lines at small DirectSteps, I think this is probably floating-point error and will need to be tuned per-scene at the moment, unfortunately. I did an initial prototype of the mixed-traversal intersection method, but it didn't quite work, so it'll probably be going back to the drawing board + coming sometime in 2021.

[digbeta]

Closing this out as the issue has been fixed.