search

nagadomi / nunif

Misc; latest version of waifu2x; 2D video to stereo 3D video conversion
MIT License
1.58k stars 142 forks source link

New metric depth model issue #146

Closed donrikk closed 4 months ago

donrikk commented 4 months ago

for some reason all maps inlovling v2 are now destroyed these are a few examples of a variety of models between any n v2 models and regular v2 small through large models now after the update. all of these examples are N v2 models just to clarify just differnt depth model sizes like base small and large

image image image image image

loawizard commented 4 months ago

Is this normal?

donrikk commented 4 months ago

not even close to normal notice his hospital gown in the last example here. his entire shirt is being repreesented as extremely far background when it should be at least mid ground and that happens like every other depth frame so the chest is constatly bouncing in and out of foreground level. its def not performing properly i highly doubt this was the intended look for the map.

nagadomi commented 4 months ago

I suspect it is a known problem with V2 recognizing glass and mirrors as flat surfaces. I have confirmed that it is the same result as the official code.

donrikk commented 4 months ago

@nagadomi yeah that i expect ive seen that and i tend to ignore it as at some point thatll get better but if u look at the example above his shirt not a glass surface or mirror just a regular shirt is being project to the background in black so in this example abov his torso is popping out of foreground and all the awy into the background leaving him as a floating head and arm. iobviously i dont think it has anythingto do with you but i do think the depth anything crew may have miss the mark on the accuracy of the map the majorty of frames look like water paint there is almost zer oacuracy at all in tyhe model which is weird becuase if any s is in it then it should at the very least have some foreground detal but it doesnt. i may have to try to see if i can some way inform the depth anything devs about this as i dont think they intended this map to do what its doing. im not expecting it to be the greeatest thing in the world but as of currently v1 N model is better than v2 and that fundementally makes no sense to me lol

nagadomi commented 4 months ago

may be related issue https://github.com/DepthAnything/Depth-Anything-V2/issues/10 https://github.com/LiheYoung/Depth-Anything/issues/201

kaelsonofkrypto commented 4 months ago

hi Nagadomi, something is definitely wrong with the any sv2 model in iw3, i just checked it compared to hugging face v1, v2, and iw3, for some reason the any sv2 has lost complete detail, here is a screen shot of them side my side broken any sv2

kaelsonofkrypto commented 4 months ago

here is a link to the original source rgb png i used https://drive.google.com/file/d/1N-CaueINcpr6USzKmTHQc9O9M8zucgYY/view?usp=sharing

nagadomi commented 4 months ago

@kaelsonofkrypto Try the same settings as demo. Depth Model = ANY_V2_S, Depth Resolution = 518, Foreground Scale = 0, Edge Fix = OFF Export

Edit: I changed the title as it might cause trouble for issues linked from here.