Is it possible generate a large depthmap chunk-by-chunk?

[bytesbay]

Im doing an app that makes a depthmap of a large planet. non of the GPUs can handle this , so is there any way to generate a chunk considering the surrounded chunks? Because without considering the height - the edges between chunks will jump too much and will cause visual glitches.

[graemeniedermayer]

I've been thinking about this recently. My first attempt would be to add a margin to each chunk so you have overlapping depth maps and then blend the depth maps together afterwards.

It is pretty challenging to have each chunk completely independent, because depth algorithms use global image context. A white boarder won't always be the same distance away even if it has the same pixel values.

There should be some sort of controlnet idea that applies to depthmaps. The network structures and ideas feel close enough that the tile controlnet should be applicable to depthmaps. But this also seems like a pretty heavy idea to explore. If someone has done this please share.

Maybe tiling the depthmap individually and then using stable diffusion + controlnet to merge them. might be an option.

[bytesbay]

@graemeniedermayer Thanks for your comment. Do you know is it possible to set margins with this extension? Maybe ill contribute , but i dont know from where to start

[semjon00]

I would first generate a depthmap of a downscaled image. Then progressively merge this depthmap with depthmaps taken from only parts of the image - so less downscaling is necessary. The benefit of such approach is that the depthmap geometry consistency probably will be better.

[bytesbay]

@semjon00 Yep i also thought about this. Do you know any method how to upscale a depth map? Using this module

[semjon00]

No, there is no such thing currently, but it could be a great addition.