(BUG) Quality went downhill in latest version

[HamletEagle]

Hello,

Today I downloaded the latest version and tried to run the code, but the result is significantly worse than with the code from around 3 weeks ago.

Here is the command that I run with the new code: depthflow input --image ./image.png main -w 1080 -h 1920 -s 8 -f 30.0 -q 100 -r -o ./output.avi --format avi -f 30 Here is the command that I run with the old code: broken depthflow parallax --image image.png main --format avi -q 100 -s 8 -w 1080 -h 1920 -f 30.0 -r -o ./output.avi

The new and low-quality result (see the distortion) can be seen here: https://streamable.com/ri72fk

The old result can be seen here. While it was not perfect, there was significantly less "stretching". https://streamable.com/y05h7u

Here is the image, in case you want to test (it was too big to upload here): https://we.tl/t-yXmBj7zOjZ

(ignore the image quality, it is something I quickly generated with stable diffusion to test the new version)

[Tremeschin]

This old version likely still used ZoeDepth for the depth maps, but I found DepthAnything to yield more accurate results, especially in anime (+it uses from transformers instead of torch hub) so I changed to it

The thing is, the new one is "thinner" on the edges, so there's this extra foreground leak to the background, which is a side effect of it being more precise funnily enough 🤔

Good news that applying some simple filtering to the depth map solves it to "fatten" the edges as I discovered yesterday:

https://github.com/BrokenSource/DepthFlow/issues/12#issuecomment-2046173262

I can give you later when I'm on pc a file to revert to zoedepth, or soon-ish I'll commit these changes (+new hooks to upscale the input or depth map)

Need some R&D to see if it removes too much fine detail, if it can be done on the shader itself at an ok computational cost, or cv2 is faster

it's fixable, dw :)

[Tremeschin]

Hi, I've just finished a big internal reorganization of imports, plus hooking up upscalers to depthflow, and the fix on last comment of the depth maps being too precise (I might have missed an import/functionality or two, sorry if there's any new immediate bugs)

Pretty much all projects had modifications, there's instructions on the website on upgrading, or reclone everything :) !

depthflow input -i image.png --upscale A B main

Upscales the input image by A times and the depth map by B times, use 1 for no upscale (default to both); currently no caching of the upscales but this code logistic for caching is the next thing I'll work on

Upscaling the depth map can do wonders, I have some ideas on smarter places to have it, but it's only efficient/effective on <= 1080p images, not this case

Expand to see yours input image on new version (sharp angle, lots of displacement)


[HamletEagle]

Okay, I tried the new version. I had to install some python packages by hand, but after this, I was able to run it.

The result is much better than before. There is still some distortion around the face of the character on the left, but not more than the original version.

Thanks for the fix! Btw, when do you think you will be able to add the custom animation functionality similar to Leiapix? No rush, just asking.

[Tremeschin]

The result is much better than before. There is still some distortion around the face of the character on the left, but not more than the original version.

Nice!, I'll eventually have the depth estimator changeable between depth anything and zoedepth (+potentially others), so we can have back the old visuals

Btw, when do you think you will be able to add the custom animation functionality similar to Leiapix? No rush, just asking.

No problem in asking !

I've got a life situation going on so I'm making less progress than usual, but I was playing just yesterday with dynamic CLIs with typer (because once we select a preset we want to maybe change its parameters, and add two+ presets together?)

I could make a mock up in a day or two but I'm working through this slowly so it is extensible not only for depthflow but maybe shaderflow scenes in general, it's probably the next big feature (or proper upscaling/depth caching)

[Tremeschin]

I'll close as the original issue is resolved, but I will revisit soon the preset system ^failedinitial versions