search

Jack000 / glid-3-xl

1.4B latent diffusion model fine tuning
MIT License
260 stars 49 forks source link

ModuleNotFoundError: No module named 'ldm.models'; 'ldm' is not a package & Is Conitive Face API Required? #3

Open Pathos14489 opened 2 years ago

Pathos14489 commented 2 years ago

I'm having an issue running this using what's available here. I've gotten this far but I'm just not sure how to proceed at this point. Is cognitive face API required? can it be removed? I'd personally rather not have every single sample going to microsoft if I could personally help it.

Using device: cuda:0
[2022-04-28 00:47:32] /home/user/Projects/glid-3-xl/sample.py  -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   eprint(line:60) :: Error when calling Cognitive Face API:
        status_code: 401
        code: 401
        message: Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.

[2022-04-28 00:47:32] /home/user/Projects/glid-3-xl/sample.py  -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   eprint(line:60) :: img_url:https://raw.githubusercontent.com/Microsoft/Cognitive-Face-Windows/master/Data/detection1.jpg
[2022-04-28 00:47:32] /home/user/Projects/glid-3-xl/sample.py  -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   eprint(line:60) :: Error when calling Cognitive Face API:
        status_code: 401
        code: 401
        message: Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.

[2022-04-28 00:47:32] /home/user/Projects/glid-3-xl/sample.py  -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   eprint(line:60) :: img_url:/data1/mingmingzhao/label/data_sets_teacher_1w/47017613_1510574400_out-video-jzc70f41fa6f7145b4b66738f81f082b65_f_1510574403268_t_1510575931221.flv_0001.jpg
[]
Traceback (most recent call last):
  File "/home/user/Projects/glid-3-xl/sample.py", line 283, in <module>
    ldm = torch.load(args.kl_path, map_location="cpu")
  File "/home/user/.local/lib/python3.10/site-packages/torch/serialization.py", line 712, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/user/.local/lib/python3.10/site-packages/torch/serialization.py", line 1046, in _load
    result = unpickler.load()
  File "/home/user/.local/lib/python3.10/site-packages/torch/serialization.py", line 1039, in find_class
    return super().find_class(mod_name, name)
ModuleNotFoundError: No module named 'ldm.models'; 'ldm' is not a package```

Distro: PopOS 22.01 NVIDIA-SMI: 

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.86       Driver Version: 470.86       CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
 NVIDIA GeForce RTX 2080 Ti

Python: 3.10.4

limiteinductive commented 2 years ago

you installed the wrong package: ldm from pypi is not the right package. You should clone this repo https://github.com/CompVis/latent-diffusion and pip install .

Pathos14489 commented 2 years ago

ffs I'm an actual complete idiot, thank you.

jdluzen commented 2 years ago

I'm trying to follow your directions of cloning the repo and pip installing like so: pip install git+https://github.com/CompVis/latent-diffusion.git@main#egg=ldm but I get an error:

WARNING: Discarding git+https://github.com/CompVis/latent-diffusion.git@main#egg=ldm. Requested latent-diffusion from git+https://github.com/CompVis/latent-diffusion.git@main#egg=ldm has inconsistent name: filename has 'ldm', but metadata has 'latent-diffusion'
ERROR: Could not find a version that satisfies the requirement ldm (unavailable) (from versions: 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.0.8, 0.0.9, 0.1.0, 0.1.1, 0.1.3)
ERROR: No matching distribution found for ldm (unavailable)

Ideas?

ben-selas commented 2 years ago

I'd advice you to clone the repository and then install locally. Is that what you already did? (wrong account, but I'm limiteinductive)

wes-kay commented 2 years ago

@jdluzen he's saying clone then do pip install . also https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/index-identity if you want the face api.

@ben-selas @limiteinductive I'm getting the same error sorry I don't fully understand do we have to install: https://github.com/CompVis/latent-diffusion/tree/main/models/ldm from the main model then run the sample on glide? I did that Successfully installed latent-diffusion-0.0.1 from the latent-diffusion setup, and running the glide sample ModuleNotFoundError: No module named 'ldm' now.

pip list also doesn't show ldm installed.

ben-selas commented 2 years ago

@wes-kay Could you print the output of your pip freeze after you installed CompVis latent-diffusion package?

wes-kay commented 2 years ago

@ben-selas

On windows

The install

`Successfully built latent-diffusion WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) Installing collected packages: latent-diffusion Attempting uninstall: latent-diffusion WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) Found existing installation: latent-diffusion 0.0.1 Uninstalling latent-diffusion-0.0.1: Successfully uninstalled latent-diffusion-0.0.1 WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) Successfully installed latent-diffusion-0.0.1 WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages)

`

Pip freeze `WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages) absl-py==1.1.0 ai-tools==0.3.9 aiohttp==3.8.1 aiosignal==1.2.0 antlr4-python3-runtime==4.9.3 async-generator==1.10 async-timeout==4.0.2 atomicwrites==1.4.0 attrs==21.4.0 axial-positional-embedding==0.2.1 blobfile==1.3.1 braceexpand==0.1.7 cachetools==5.2.0 certifi==2022.5.18.1 cffi==1.15.0 charset-normalizer==2.0.12 click==8.1.3 clip==0.2.0 cognitive-face==1.5.0 colorama==0.4.4 cryptography==37.0.2 Cython==0.29.30 DALL-E==0.1 dalle-pytorch==1.6.4 dlib==19.24.0 einops==0.4.1 filelock==3.7.1 frozenlist==1.3.0 fsspec==2022.5.0 ftfy==6.1.1 google-auth==2.8.0 google-auth-oauthlib==0.4.6 grpcio==1.46.3 -e git+https://github.com/Jack000/glid-3-xl@a0b5be4b04378d4d4779240d3e0a599360c1a133#egg=guided_diffusion h11==0.13.0 huggingface-hub==0.7.0 idna==3.3 imageio==2.19.3 iniconfig==1.1.1 latent-diffusion @ file:///...../latent-diffusion Library==0.0.0 Markdown==3.3.7 multidict==6.0.2 mypy==0.961 mypy-extensions==0.4.3 networkx==2.8.4 numpy==1.22.4 oauthlib==3.2.0 omegaconf==2.2.2 opencv-python==4.6.0.66 outcome==1.2.0 packaging==21.3 Pillow==9.1.1 pluggy==1.0.0 protobuf==3.19.4 py==1.11.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycparser==2.21 pycryptodomex==3.14.1 pyDeprecate==0.3.2 pyOpenSSL==22.0.0 pyparsing==3.0.9 PySocks==1.7.1 pytest==7.1.2 pytorch-lightning==1.6.4 PyWavelets==1.3.0 PyYAML==6.0 regex==2022.6.2 requests==2.28.0 requests-oauthlib==1.3.1 rotary-embedding-torch==0.1.5 rsa==4.8 scikit-image==0.19.3 scipy==1.8.1 selenium==4.2.0 six==1.16.0 sniffio==1.2.0 sortedcontainers==2.4.0 taming-transformers-rom1504==0.0.6 tensorboard==2.9.1 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tifffile==2022.5.4 tokenizers==0.12.1 tomli==2.0.1 torch==1.11.0 torchmetrics==0.9.1 torchvision==0.12.0 tqdm==4.64.0 transformers==4.19.4 trio==0.21.0 trio-websocket==0.9.2 typing_extensions==4.2.0 urllib3==1.26.9 wcwidth==0.2.5 webdataset==0.2.5 Werkzeug==2.1.2 wsproto==1.1.0 xmltodict==0.12.0 yarl==1.7.2 youtokentome==1.0.6 zprint==0.0.11

`

limiteinductive commented 2 years ago

@wes-kay I'd advise you to use python 3.8 for deep learning packages (although I don't think it is the issue here). When you open your python console and type import ldm does it throw an error?

wes-kay commented 2 years ago

@limiteinductive It's not installed. As the setup in the latent diffusion doesn't contain the install for the package: https://github.com/CompVis/latent-diffusion/blob/main/setup.py

"ldm from pypi is not the right package" Where do we get the correct ldm?

limiteinductive commented 2 years ago

@limiteinductive It's not installed. As the setup in the latent diffusion doesn't contain the install for the package: https://github.com/CompVis/latent-diffusion/blob/main/setup.py

"ldm from pypi is not the right package" Where do we get the correct ldm?

Did you try installing it using conda and the environment.yaml file like advertised in the Readme of the CompVis package?

wes-kay commented 2 years ago

:shame: I installed Conda but forgot to actually run the yaml... Thank you for your contribution and time!

wes-kay commented 2 years ago

@limiteinductive PROMISE I will leave you alone after this. I installed ld with conda and I can run the ld sample, but now trying to run glide:

(ldm) PathTo\glid-3-xl>python sample.py --model_path finetune.pt --batch_size 6 --num_batches 6 --text "a cyberpunk girl with a scifi neuralink device on her head"
Traceback (most recent call last):
  File "sample.py", line 19, in <module>
    from dalle_pytorch import DiscreteVAE, VQGanVAE
ModuleNotFoundError: No module named 'dalle_pytorch'

I can just install this with pip normally or do I need to do something else?

limiteinductive commented 2 years ago

@wes-kay ha ha, don't worry. Yes sure you can install it. If it bugs it means you probably need to install a newer version of pytorch in your ldm env. We should probably refactor the glid-3-xl to have a better install script

wes-kay commented 2 years ago

Thank you for all your help it's working now but throwing a AssertionError: Torch not compiled with CUDA enabled :D Had to update pytorch conda install pytorch torchvision -c pytorch

wes-kay commented 2 years ago

Okay fixed that, with a re install and update of torch, now getting 

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 3.42 GiB already allocated; 0 bytes free; 3.49 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

:D Playing around with the model_params didn't fix it.

limiteinductive commented 2 years ago

@wes-kay bad news: your GPU is doesn't have enough VRAM. You should use a service like google colab, or kaggle notebooks if you wanna run this package

wes-kay commented 2 years ago

@limiteinductive Thought as much, final question, I spun up a deep learning EC2 image

but getting:

/glid-3-xl$ python sample.py --clip_guidance --model_path finetune.pt --batch_size 1 --num_batches 12 --text "a cyberpunk girl with a scifi neuralink device on her head | trending on artstation"
Using device: cpu
Killed

Do I need higher specs?

EDIT: Looks like their "Deep" learning doesn't have GPUs RuntimeError: No HIP GPUs are available

limiteinductive commented 2 years ago

yeah it seems so. You probably won't get a GPU for less than 0.25/hour

Pareesa commented 1 year ago

you installed the wrong package: ldm from pypi is not the right package. You should clone this repo https://github.com/CompVis/latent-diffusion and pip install .

I still have the same problem and I have done pip install . locally. This is the error: eprint(line:60) :: Error when calling Cognitive Face API: status_code: 401 code: 401 message: Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.

eprint(line:60) :: img_url:https://raw.githubusercontent.com/Microsoft/Cognitive-Face-Windows/master/Data/detection1.jpg