zhyever / PatchFusion

[CVPR 2024] An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimation
https://zhyever.github.io/patchfusion/
MIT License
958 stars 64 forks source link

AttributeError: 'dict' object has no attribute 'pretrained_resource' #34

Open chaoji90 opened 5 months ago

chaoji90 commented 5 months ago

AttributeError: 'dict' object has no attribute 'pretrained_resource'occurred when I executed model = PatchFusion.from_pretrained(model_name).to(DEVICE).eval() It looks like the dictionary is accessed through dots. Any suggestions on how to deal with it? Is it a version problem?

zhyever commented 4 months ago

Tbh, I have no idea about this issue. Anyone has a similar case and can help?

graysuit commented 4 months ago

@zhyever I have same issue. Its on: https://github.com/zhyever/PatchFusion/blob/main/estimator/models/patchfusion.py#L77


C:\Users\gray\Desktop\PatchFusion\tools>test_single_forward.py
config.json: 100%|██████████████████████████████████████████████████████████████████| 3.65k/3.65k [00:00<00:00, 3.64MB/s]
C:\Users\gray\AppData\Local\Programs\Python\Python312\Lib\site-packages\huggingface_hub\file_download.py:157: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\gray\.cache\huggingface\hub\models--Zhyever--patchfusion_depth_anything_vitl14. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)
Traceback (most recent call last):
  File "C:\Users\gray\Desktop\PatchFusion\tools\test_single_forward.py", line 12, in <module>
    model = PatchFusion.from_pretrained(model_name).to(DEVICE).eval()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gray\AppData\Local\Programs\Python\Python312\Lib\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gray\AppData\Local\Programs\Python\Python312\Lib\site-packages\huggingface_hub\hub_mixin.py", line 511, in from_pretrained
    instance = cls._from_pretrained(
               ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gray\AppData\Local\Programs\Python\Python312\Lib\site-packages\huggingface_hub\hub_mixin.py", line 727, in _from_pretrained
    model = cls(**model_kwargs)
            ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gray\Desktop\PatchFusion\tools\estimator\models\patchfusion.py", line 77, in __init__
    config.coarse_branch.pretrained_resource = None
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'pretrained_resource'

C:\Users\gray\Desktop\PatchFusion\tools>
zhyever commented 4 months ago

While I don't know what is going on, one hack implementation is as simple as replacing this config.coarse_branch.pretrained_resource = None with config.coarse_branch['pretrained_resource'] = None. L78 should be similar. I'm not sure if there are more bugs after these two lines of codes.

Btw, I recommand to check the installed huggingface_hub version and align it with our offical one huggingface-hub==0.20.1

graysuit commented 4 months ago

Sorry if it looks dump I'm new to huggingface etc things and I copied estimator etc folders to tools directory Does that affect ?

My purpose is to create high resolution depth map image from an image and after generating 3d relief sculpture stl for that I'm also planning to bundle into pyinstaller Will it work and am I in right direction ? Currently using dotnet (C#/VB) as main language and trying to wrap it, as dotnet doesn't have such libs

zhyever commented 4 months ago

No worries. This interface is somehow annoying because it's changing some detailed implementations during updates...

It would affect if you change the folder structure. Please leave them there. (as far as I see)

The current issue is that the huggingface-hub might not work on purpose, so it would be impossible to download everything automatically in your case. The solution is to use another way to build the model locally but it might need more manual work (download checkpoints, change configs, etc). I will first try to do it on my machine and then publish another test_single_forward_wo_net.py later. No worries, it would be friendly to people who are new to these tricky stuffs.

I'm not sure if the path after getting high resolution depth map is correct. But I believe patchfusion is always a good choice for high resolution metric depth estimation. Feel free to have a try and hopefully everything goes well and you can finally achieve your goals!

graysuit commented 4 months ago

Hi, truly appreciate your help I tried refactoring like you suggested and errors gone.

Now I supplied it 500x500px image, but my pc got hang up: image

xFormers not available
xFormers not available
C:\Users\gray\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\functional.py:512: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ..\aten\src\ATen\native\TensorShape.cpp:3588.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]

Not sure if PatchFusion does heavy calculations ? and isn't suitable for general CPU ? I have 8gb ram, core i3

zhyever commented 4 months ago

Hi, glad to hear it works. We recommand to use gpu to run patchfusion. It's kind of heavy for cpu

graysuit commented 4 months ago

Hi, glad to hear it works. We recommand to use gpu to run patchfusion. It's kind of heavy for cpu

No issue, I can use my friend's PC

I tried demo. But output seems low detailed compared to samples on paper.

For example I tried horse image 1500px square. input - Copy

But output didn't have details like eye, nose etc. clipimage I was expecting output which could be made like below: image

Is there any settings I need to change. Also after that, will I able to generate 3D model ? For example this sample looks very good. I consider brightness as Z and constructed 3D STL model and looks to be working. Not sure if its correct way and have any recommendations ? clipimage

image

zhyever commented 2 months ago

Sorry for the late reply!

It might be too hard for the model to predict depth for this figure. It could be out-of-distribution in some sense. Please let me know if you find a way to get better results. Really interested in a successful solution.

graysuit commented 2 months ago

Please let me know if you find a way to get better results. Really interested in a successful solution.

Unfortunately, I'm not working on project anymore. But best clean results were generated from vistasculpt. I analyzed its working, it was using dinov2 I failed to know mechanism though.

image I attached output stl, you can check: horse.zip