Fails to download AdaBins

[kirilledelman]

After rendering a couple frames, it dies with:

2022-02-17 01:32:25.422 | DEBUG    | pytti.workhorse:update:503 - Time: 0.0000 seconds
2022-02-17 01:32:25.435 | DEBUG    | pytti.LossAug.DepthLoss:init_AdaBins:18 - Loading AdaBins...
2022-02-17 01:32:25.439 | DEBUG    | adabins.infer:__init__:81 - /root/.cache/adabins/AdaBins_nyu.pt
2022-02-17 01:32:25.444 | DEBUG    | adabins.model_io:dl_adabins:8 - Attempting to fetch AdaBins pretrained weights...
2022-02-17 01:32:25.447 | DEBUG    | adabins.model_io:dl_adabins:15 - using destination path: /root/.cache/adabins/
2022-02-17 01:32:25.449 | DEBUG    | adabins.model_io:dl_adabins:24 - downloading from: https://drive.google.com/uc?id=1lvyZZbC9NLcS8a__YPcUP7rDiIpbRpoF

    Cannot retrieve the public link of the file. You may need to change
    the permission to 'Anyone with the link', or have had many accesses. 

You may still be able to access the file from the browser:

     https://drive.google.com/uc?id=1lvyZZbC9NLcS8a__YPcUP7rDiIpbRpoF 

2022-02-17 01:32:25.859 | DEBUG    | adabins.model_io:dl_adabins:27 - gdown response: None
2022-02-17 01:32:25.860 | DEBUG    | adabins.model_io:dl_adabins:8 - Attempting to fetch AdaBins pretrained weights...
2022-02-17 01:32:25.866 | DEBUG    | adabins.model_io:dl_adabins:15 - using destination path: /root/.cache/adabins/
2022-02-17 01:32:25.869 | DEBUG    | adabins.model_io:dl_adabins:24 - downloading from: https://drive.google.com/uc?id=1zgGJrkFkJbRouqMaWArXE4WF_rhj-pxW

    Cannot retrieve the public link of the file. You may need to change
    the permission to 'Anyone with the link', or have had many accesses. 

You may still be able to access the file from the browser:

     https://drive.google.com/uc?id=1zgGJrkFkJbRouqMaWArXE4WF_rhj-pxW 

2022-02-17 01:32:26.055 | DEBUG    | adabins.model_io:dl_adabins:27 - gdown response: None
Access denied with the following error:
Access denied with the following error:
Loading base model ()...Downloading: "https://github.com/rwightman/gen-efficientnet-pytorch/archive/master.zip" to /root/.cache/torch/hub/master.zip
Downloading: "https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-weights/tf_efficientnet_b5_ap-9e82fae8.pth" to /root/.cache/torch/hub/checkpoints/tf_efficientnet_b5_ap-9e82fae8.pth
Done.
Removing last two layers (global_pool & classifier).
Building Encoder-Decoder model..Done.
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-8-55db5d020140>](https://localhost:8080/#) in <module>()
      4 
      5 # function wraps step 2.3 of the original p5 notebook
----> 6 _main(cfg)

13 frames
[/usr/local/lib/python3.7/dist-packages/torch/serialization.py](https://localhost:8080/#) in __init__(self, name, mode)
    209 class _open_file(_opener):
    210     def __init__(self, name, mode):
--> 211         super(_open_file, self).__init__(open(name, mode))
    212 
    213     def __exit__(self, *args):

FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/adabins/AdaBins_nyu.pt'

[dmarx]

This isn't actually a bug with with pytti-tools: the download URL is being rate-limited because it's receiving too many requests. It should resolve itself in a few hours, definitely should work again tomorrow. For the time being, you can use either of the two URLs that appear in that error message to download it through your browser, then just move/copy the weights to the location in the error: /root/.cache/adabins/AdaBins_nyu.pt

[kirilledelman]

@dmarx is there a way to update the path that it's trying to pull it from directly in code (in my own copy of this notebook)? I'd point it to my own drive's location of it. I searched for this URL in the source and didn't find it.

Additionally, Colab notebook doesn't allow me to create a folder called ".cache", probably because it starts with a period, so I'm stuck.

[dmarx]

y'know what? I spoke too soon. Everything I said above is accurate, but I think it's my fault that the downlaod URL is being overloaded with requests.

---

dev notes:

Issue: models downloaded during CLI runs are being downlaoded to the outputs folder for the run rather than the user cache.

proposed resolution: use the warmup script to detect the correct cache location and write it to a location accessible during CLI runs (new config key?). Access this location when invoking the requsite model download

Alt resolution: block the run if requsitie models not already downloaded. prompt use to invoke a separate command to download the models then re-attempt the run.

[dmarx]

Also yes, I'll try to add some mechanism to let the user specify the location of the weights

[dmarx]

1. add a directory to the config for specifying where pretrained models will download
2. if model has been downloaded to cache already, add a link to this location? Copy to this location?

...part of the issue here is that the model download piece currently lives in the adabins code. Which makes sense out of context, but for our purposes, we want to specify where the model downloads to. so I guess skip step (2) above and instead do something more like:

1. add a directory to the config for specifying where pretrained models will download
2. specify this directory as a function argument when calling adabins (rather than relying on communicating everything through config files). it's then adabins responsibility to try to load from that directory or download to it

[dmarx]

Ok, looked into this a bit more. I had another user reporting that pytti was downloading a model repeatedly. Turns out, their issue wasn't with adabins but with vqgan. The issue with vqgan is really obvious and I'll be fixing that shortly. Thankfully, that issue doesn't impact AdaBins, so the problem you're experiencing is unrelated and I don't believe was caused by pytti-tools requesting that URL repeatedly when it didn't need to, so closing issue.

As I mentioned, I'll be adding a mechanism for the user to specify a directory to save/load weights in the near future. Once this is in place, I'll also add instructions or something to promote users who have a google drive mounted to locate their model directory on the drive. This should further reduce model downloads by users of this particular tool. Thanks again for your patience, and let me know if you continue to have issues or spot any other problems.

[kirilledelman]

Thank you, I appreciate you looking into this. Hoping to be able to run this amazing tool again soon. I was only able to run the example and was floored by the results, before starting to get that error.