parkchamchi / DepthViewer

Unity program that creates 3D scenes for VR using MiDaS model
MIT License
66 stars 5 forks source link

Depthviewer only generates 518x518 for DAN #26

Open ThatDocBoi opened 7 months ago

ThatDocBoi commented 7 months ago

Hey, I am trying to convert a bunch of images, however I've noticed that if I use the depth.py it generates much better depth. Looking at the depth files it appears that using depth.py downscales the image so that 1 side is 518 and the other is bigger, but the viewer only does 518x518. Since the viewer does not see the previously generated .depthfiles, this makes it so that for each image, the depth file needs to be opened, then the image file, every time.

Would it be possible to have a way to automatically read the generated depth files or make the viewer generate it like how depth.py does please?

Thank you.

parkchamchi commented 2 months ago

Well it's been long since I haven't seen the code so I have to test it, but in the Unity C# script when it detects depth-anything model it sets the fallback to 518x518: #

...Which set the SentisDepthModel's WidthFallback and HeightFallback

But as "fallback" implies, this is only used when it could not (or I could not) detect the model size from the model itself:

        //Set these to (518, 518) for Depth-Anything models

        var heightSym = _model.inputs[0].shape[2];
        var widthSym = _model.inputs[0].shape[3];

        if (heightSym.isValue)
            _height  = heightSym.value;
        else {
            Debug.Log($"SentisDepthModel: Height cannot be inferred, falling back to {_heightFallback}");
            _height = _heightFallback;
        }

        if (widthSym.isValue)
            _width = widthSym.value;
        else {
            Debug.Log($"SentisDepthModel: Width cannot be inferred, falling back to {_widthFallback}");
            _width = _widthFallback;
        }

        _output = new float[_width*_height];

#

So two questions arise:

I'll take a look into it.

(btw sorry for the long hiatus)