BillFSmith / TilingZoeDepth

144 stars 22 forks source link

Splitting images further for even higher, more realistic definition #6

Open gituser123456789000 opened 6 months ago

gituser123456789000 commented 6 months ago

Through testing, I see that splitting images further and running each piece through your Tiled ZoeDepth and then recombining the pieces, your program seems definitely more capable at pulling out more detail. It needs the rest of what your program does to even everything out and compare it against the original, so the result is cohesive, as the rest of your program does. Here are some examples to show that more detail is there for the taking if you made a version of the program that does 2x, 4x, 8x, 64x, etc as many splits as it does now. I'd expect the coding would be more complex, and the result to take x times longer, but for better quality, who cares about the time, within reason.

This is the original image we're testing with: https://media.discordapp.net/attachments/829513391529656424/1197009038639104110/Input.png

This is the depth buffer pulled directly from the game as a sort of ideal result: https://media.discordapp.net/attachments/829513391529656424/1197126300930936862/OMG_ground_truth_depth_buffer.png

Here's the original result from your Tiled ZoeDepth: https://media.discordapp.net/attachments/829513391529656424/1197628494847430686/Input_d_TiledZeoDepth.png

And here are the results of splitting the image in 2, 4, 8 and running each piece through Tiled ZoeDepth, then recombining them: https://media.discordapp.net/attachments/829513391529656424/1197628495103271003/split2.png https://media.discordapp.net/attachments/829513391529656424/1197628495329767514/split4.png https://media.discordapp.net/attachments/829513391529656424/1197628495594016859/split8.png

You start seeing detail in the rims of the trailer, more individual tree branches, background trees, the front end of the Hummer start starts taking shape and separated from the front tire, the g_n on his back, definitely the pouch he's wearing on his hip and its strap, the bodies, etc.

As I've said before, you have the best defined depth maps I've seen from an automated program. I'd like to see it taken further. Version 4 with options for 8x as many splits, or more. Even just 2x more splits made a significant difference.

Edit to add: Here's the detail that can be brought out with 8x upscaling and 8x additional splitting: https://media.discordapp.net/attachments/829513391529656424/1197695958004269176/Input_8xUpscaled_Split8x_scaled_down_to_original_resolution.png