Open kalle07 opened 1 week ago
MNeMoNiCuZ
commented
1 week ago The models I changed to don't seem to support Python 3.12 for some reason, due to numpy. I don't know why, but it worked when I used Python 3.10.
You may need to re-setup your environment. Sorry!
kalle07
commented
1 week ago iv installed with python 3.11 ;) so only 3.10? i dont have 3.12 (installed at all)
MNeMoNiCuZ
commented
1 week ago I tried just now with python 3.11.5 and it works fine.
Did you remove your venv and set it up again?
1. -V:3.12 at * C:\Python312\python.exe
2. -V:3.11 at C:\Python311\python.exe
3. -V:3.10 at C:\Python310\python.exe
4. -V:3.8 at C:\Python38\python.exe
Select a Python version by number (Press Enter for default = '1'): 2
Using Python version 3.11
Enter the name for your virtual environment (Press Enter for default 'venv'):
Creating virtual environment named venv...
Creating .gitignore in the venv folder...
Generating venv_activate.bat...
Generating venv_update.bat for a one-time pip upgrade...
Do you want to upgrade pip now? (Y/N) (Press Enter for default 'Y'):
Upgrading pip and activating the virtual environment...
Activating virtual environment venv and upgrading pip...
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (23.2.1)
Collecting pip
Obtaining dependency information for pip from https://files.pythonhosted.org/packages/d4/55/90db48d85f7689ec6f81c0db0622d704306c5284850383c090e6c7195a5c/pip-24.2-py3-none-any.whl.metadata
Downloading pip-24.2-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.2-py3-none-any.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 8.3 MB/s eta 0:00:00
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 23.2.1
Uninstalling pip-23.2.1:
Successfully uninstalled pip-23.2.1
Successfully installed pip-24.2
Pip has been upgraded in the virtual environment venv.
To deactivate, manually type 'deactivate'.
uv pip package installer:
uv is a Python package that provides enhanced capabilities for managing Python environments and dependencies.
Do you want to install 'uv' package? (Y/N) (Press Enter for default 'Y'):
Installing 'uv' package...
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting uv
Downloading uv-0.4.4-py3-none-win_amd64.whl.metadata (11 kB)
Downloading uv-0.4.4-py3-none-win_amd64.whl (12.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.3/12.3 MB 36.8 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.4.4
requirements.txt found.
Do you wish to run 'uv pip install -r requirements.txt'? (Y/N) (Press Enter for default 'Y'):
User selected: Y
Installing requirements from requirements.txt...
Resolved 72 packages in 420ms
Prepared 1 package in 31ms
Installed 72 packages in 2.50s
+ accelerate==0.34.0
+ aiofiles==23.2.1
+ annotated-types==0.7.0
+ anyio==4.4.0
+ bitsandbytes==0.43.3
+ certifi==2024.8.30
+ charset-normalizer==3.3.2
+ click==8.1.7
+ colorama==0.4.6
+ contourpy==1.3.0
+ cycler==0.12.1
+ fastapi==0.112.2
+ ffmpy==0.4.0
+ filelock==3.15.4
+ fonttools==4.53.1
+ fsspec==2024.6.1
+ gradio==4.42.0
+ gradio-client==1.3.0
+ h11==0.14.0
+ httpcore==1.0.5
+ httpx==0.27.2
+ huggingface-hub==0.24.3
+ idna==3.8
+ importlib-resources==6.4.4
+ jinja2==3.1.4
+ kiwisolver==1.4.7
+ markdown-it-py==3.0.0
+ markupsafe==2.1.5
+ matplotlib==3.9.2
+ mdurl==0.1.2
+ mpmath==1.3.0
+ networkx==3.3
+ numpy==2.1.1
+ orjson==3.10.7
+ packaging==24.1
+ pandas==2.2.2
+ pillow==10.4.0
+ protobuf==5.28.0
+ psutil==5.9.8
+ pydantic==2.8.2
+ pydantic-core==2.20.1
+ pydub==0.25.1
+ pygments==2.18.0
+ pyparsing==3.1.4
+ python-dateutil==2.9.0.post0
+ python-multipart==0.0.9
+ pytz==2024.1
+ pyyaml==6.0.2
+ regex==2024.7.24
+ requests==2.32.3
+ rich==13.8.0
+ ruff==0.6.3
+ safetensors==0.4.4
+ semantic-version==2.10.0
+ sentencepiece==0.2.0
+ shellingham==1.5.4
+ six==1.16.0
+ sniffio==1.3.1
+ spaces==0.29.6
+ starlette==0.38.4
+ sympy==1.13.2
+ tokenizers==0.19.1
+ tomlkit==0.12.0
+ torch==2.4.0
+ tqdm==4.66.5
+ transformers==4.43.3
+ typer==0.12.5
+ typing-extensions==4.12.2
+ tzdata==2024.1
+ urllib3==2.2.2
+ uvicorn==0.30.6
+ websockets==12.0
Requirements installed.
Listing installed packages...
Package Version
------------------- -----------
accelerate 0.34.0
aiofiles 23.2.1
annotated-types 0.7.0
anyio 4.4.0
bitsandbytes 0.43.3
certifi 2024.8.30
charset-normalizer 3.3.2
click 8.1.7
colorama 0.4.6
contourpy 1.3.0
cycler 0.12.1
fastapi 0.112.2
ffmpy 0.4.0
filelock 3.15.4
fonttools 4.53.1
fsspec 2024.6.1
gradio 4.42.0
gradio_client 1.3.0
h11 0.14.0
httpcore 1.0.5
httpx 0.27.2
huggingface-hub 0.24.3
idna 3.8
importlib_resources 6.4.4
Jinja2 3.1.4
kiwisolver 1.4.7
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.9.2
mdurl 0.1.2
mpmath 1.3.0
networkx 3.3
numpy 2.1.1
orjson 3.10.7
packaging 24.1
pandas 2.2.2
pillow 10.4.0
pip 24.2
protobuf 5.28.0
psutil 5.9.8
pydantic 2.8.2
pydantic_core 2.20.1
pydub 0.25.1
Pygments 2.18.0
pyparsing 3.1.4
python-dateutil 2.9.0.post0
python-multipart 0.0.9
pytz 2024.1
PyYAML 6.0.2
regex 2024.7.24
requests 2.32.3
rich 13.8.0
ruff 0.6.3
safetensors 0.4.4
semantic-version 2.10.0
sentencepiece 0.2.0
setuptools 65.5.0
shellingham 1.5.4
six 1.16.0
sniffio 1.3.1
spaces 0.29.6
starlette 0.38.4
sympy 1.13.2
tokenizers 0.19.1
tomlkit 0.12.0
torch 2.4.0
tqdm 4.66.5
transformers 4.43.3
typer 0.12.5
typing_extensions 4.12.2
tzdata 2024.1
urllib3 2.2.2
uv 0.4.4
uvicorn 0.30.6
websockets 12.0
Setup complete. Your virtual environment is ready.
To deactivate the virtual environment, type 'deactivate'.
(venv) C:\Users\MNeMiC\Desktop\joy-caption-batch>pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
Looking in indexes: https://download.pytorch.org/whl/cu121, https://pypi.ngc.nvidia.com
Requirement already satisfied: torch in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (2.4.0)
Collecting torchvision
Downloading https://download.pytorch.org/whl/cu121/torchvision-0.19.1%2Bcu121-cp311-cp311-win_amd64.whl (5.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.8/5.8 MB 35.4 MB/s eta 0:00:00
Collecting torchaudio
Downloading https://download.pytorch.org/whl/cu121/torchaudio-2.4.1%2Bcu121-cp311-cp311-win_amd64.whl (4.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.1/4.1 MB 35.4 MB/s eta 0:00:00
Requirement already satisfied: filelock in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torch) (3.15.4)
Requirement already satisfied: typing-extensions>=4.8.0 in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torch) (4.12.2)
Requirement already satisfied: sympy in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torch) (1.13.2)
Requirement already satisfied: networkx in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torch) (3.3)
Requirement already satisfied: jinja2 in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torch) (3.1.4)
Requirement already satisfied: fsspec in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torch) (2024.6.1)
Requirement already satisfied: numpy in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torchvision) (2.1.1)
Collecting torch
Downloading https://download.pytorch.org/whl/cu121/torch-2.4.1%2Bcu121-cp311-cp311-win_amd64.whl (2444.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 GB 36.0 MB/s eta 0:00:00
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from torchvision) (10.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from jinja2->torch) (2.1.5)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\mnemic\desktop\joy-caption-batch\venv\lib\site-packages (from sympy->torch) (1.3.0)
Installing collected packages: torch, torchvision, torchaudio
Attempting uninstall: torch
Found existing installation: torch 2.4.0
Uninstalling torch-2.4.0:
Successfully uninstalled torch-2.4.0
Successfully installed torch-2.4.1+cu121 torchaudio-2.4.1+cu121 torchvision-0.19.1+cu121
(venv) C:\Users\MNeMiC\Desktop\joy-caption-batch>py batch.py
Captioning Batch Images Initializing...
Downloading image_adapter.pt from Hugging Face Space...
Downloaded image_adapter.pt successfully.
Captioning Initializing
Found 1 files to process in C:\Users\MNeMiC\Desktop\joy-caption-batch\input
Loading CLIP
Loading tokenizer
Loading LLM
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████| 4/4 [00:09<00:00, 2.46s/it]
Loading image adapter
C:\Users\MNeMiC\Desktop\joy-caption-batch\batch.py:135: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
image_adapter.load_state_dict(torch.load(CHECKPOINT_PATH / "image_adapter.pt", map_location="cpu"))
Processing images: 0%| | 0/1 [00:00<?, ?it/s]C:\Users\MNeMiC\Desktop\joy-caption-batch\venv\Lib\site-packages\transformers\models\siglip\modeling_siglip.py:573: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:555.)
attn_output = torch.nn.functional.scaled_dot_product_attention(
Processing images: 100%|█████████████████████████████████████████████████████████████| 1/1 [00:09<00:00, 9.11s/it]
Processing all images in the input folder
(venv) C:\Users\MNeMiC\Desktop\joy-caption-batch>python --version
Python 3.11.5what is that hardlink error? (same numpy error with 3.10)
h:\caption\joy-caption-batch>venv_create.bat Installed Pythons found by py Launcher for Windows *
Do you want to upgrade pip now? (Y/N) (Press Enter for default 'Y'): Upgrading pip and activating the virtual environment... Activating virtual environment venv and upgrading pip... Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Requirement already satisfied: pip in .\venv\lib\site-packages (22.3.1) Collecting pip Downloading pip-24.2-py3-none-any.whl (1.8 MB) ---------------------------------------- 1.8/1.8 MB 6.8 MB/s eta 0:00:00 Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 22.3.1 Uninstalling pip-22.3.1: Successfully uninstalled pip-22.3.1 Successfully installed pip-24.2 Pip has been upgraded in the virtual environment venv. To deactivate, manually type 'deactivate'.
uv pip package installer:
uv is a Python package that provides enhanced capabilities for managing Python environments and dependencies.
Do you want to install 'uv' package? (Y/N) (Press Enter for default 'Y'):
Installing 'uv' package...
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting uv
Downloading uv-0.4.4-py3-none-win_amd64.whl.metadata (11 kB)
Downloading uv-0.4.4-py3-none-win_amd64.whl (12.3 MB)
---------------------------------------- 12.3/12.3 MB 7.0 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.4.4
requirements.txt found.
Do you wish to run 'uv pip install -r requirements.txt'? (Y/N) (Press Enter for default 'Y'):
User selected: Y
Installing requirements from requirements.txt...
Resolved 72 packages in 321ms
░░░░░░░░░░░░░░░░░░░░ [0/72] Installing wheels... warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
If the cache and target directories are on different filesystems, hardlinking may not be supported.
If this is intentional, set export UV_LINK_MODE=copy or use --link-mode=copy to suppress this warning.
Installed 72 packages in 7.18s
accelerate 0.34.0 aiofiles 23.2.1 annotated-types 0.7.0 anyio 4.4.0 bitsandbytes 0.43.3 certifi 2024.8.30 charset-normalizer 3.3.2 click 8.1.7 colorama 0.4.6 contourpy 1.3.0 cycler 0.12.1 fastapi 0.112.2 ffmpy 0.4.0 filelock 3.15.4 fonttools 4.53.1 fsspec 2024.6.1 gradio 4.42.0 gradio_client 1.3.0 h11 0.14.0 httpcore 1.0.5 httpx 0.27.2 huggingface-hub 0.24.3 idna 3.8 importlib_resources 6.4.4 Jinja2 3.1.4 kiwisolver 1.4.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.9.2 mdurl 0.1.2 mpmath 1.3.0 networkx 3.3 numpy 2.1.1 orjson 3.10.7 packaging 24.1 pandas 2.2.2 pillow 10.4.0 pip 24.2 protobuf 5.28.0 psutil 5.9.8 pydantic 2.8.2 pydantic_core 2.20.1 pydub 0.25.1 Pygments 2.18.0 pyparsing 3.1.4 python-dateutil 2.9.0.post0 python-multipart 0.0.9 pytz 2024.1 PyYAML 6.0.2 regex 2024.7.24 requests 2.32.3 rich 13.8.0 ruff 0.6.3 safetensors 0.4.4 semantic-version 2.10.0 sentencepiece 0.2.0 setuptools 65.5.0 shellingham 1.5.4 six 1.16.0 sniffio 1.3.1 spaces 0.29.6 starlette 0.38.4 sympy 1.13.2 tokenizers 0.19.1 tomlkit 0.12.0 torch 2.4.0 tqdm 4.66.5 transformers 4.43.3 typer 0.12.5 typing_extensions 4.12.2 tzdata 2024.1 urllib3 2.2.2 uv 0.4.4 uvicorn 0.30.6 websockets 12.0 Setup complete. Your virtual environment is ready. To deactivate the virtual environment, type 'deactivate'. (venv) h:\caption\joy-caption-batch>
MNeMoNiCuZ
commented
1 week ago I think the hardlink error is benign.
After running that, did you install pytorch properly for the correct version of cuda?
nvcc --version
And then the right cuda-matching version from here: https://pytorch.org/
Example: If CUDA is 12.1, it's this one:
kalle07
commented
1 week ago i dont know exact how is all working together ... but iv intsalled a1111 WebUI_Forge kohya text-generation-webui fooocus
and all work fine i will wait until the installation is a bit more stable ;) maybe you should add not the whole cuda but the libs (I always notice that when some what is installed with cuda)
MNeMoNiCuZ
commented
1 week ago It's unlikely to get much more stable. I'm not adding cuda, you have to do that part yourself.
Python / Cuda / Torch versions are often troublesome. It's a shame the system isn't more automatic/streamlined.
There are some other versions you can try. I've seen links to it on Reddit. Search for joy caption there and you should have some alternatives. Good luck!
kalle07
commented
1 week ago i know cuda, all is installed 11 and 12 ;)
i meant like that (allways that is installed if i use cuda pyton software i mentioned)
https://github.com/oobabooga/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.2.83+cu121-cp311-cp311-win_amd64.whl; platform_system == "Windows" and python_version == "3.11" https://github.com/oobabooga/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.2.83+cu121-cp310-cp310-win_amd64.whl; platform_system == "Windows" and python_version == "3.10" https://github.com/oobabooga/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.2.83+cu121-cp311-cp311-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.11" https://github.com/oobabooga/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.2.83+cu121-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.10"
MNeMoNiCuZ
commented
1 week ago Yeah I'm not sure what I can do to help, sorry :) I'll let you know if I figure something out.
To confirm about hardlink warning message: The hardlink error previously mentioned is benign. It just means you are installing files on a different drive or location than the target location. Which means they have to be copied, rather than softlinked.
why numpy 1 / 2 ?
iv installed 2 LLMs and 4 image generators based on git python stuff, all runngin fine with cuda and tensor
what is that ?
(venv) h:\caption\joy-caption-batch>nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Jun_13_19:42:34_Pacific_Daylight_Time_2023 Cuda compilation tools, release 12.2, V12.2.91 Build cuda_12.2.r12.2/compiler.32965470_0
(venv) h:\caption\joy-caption-batch>
(venv) h:\caption\joy-caption-batch>py batch.py
A module that was compiled using NumPy 1.x cannot be run in NumPy 2.1.1 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "h:\caption\joy-caption-batch\batch.py", line 4, in
from torch import nn
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch__init.py", line 2120, in
from torch._higher_order_ops import cond
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch_higher_order_ops__init__.py", line 1, in
from .cond import cond
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch_higher_order_ops\cond.py", line 5, in
import torch._subclasses.functional_tensor
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch_subclasses\functional_tensor.py", line 42, in
class FunctionalTensor(torch.Tensor):
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch_subclasses\functional_tensor.py", line 258, in FunctionalTensor
cpu = _conversion_method_template(device=torch.device("cpu"))
h:\caption\joy-caption-batch\venv\Lib\site-packages\torch_subclasses\functional_tensor.py:258: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_numpy.cpp:84.)
cpu = _conversion_method_template(device=torch.device("cpu"))
Captioning Batch Images Initializing...
image_adapter.pt already exists.
Captioning Initializing
Found 1 files to process in h:\caption\joy-caption-batch\input
Loading CLIP
Traceback (most recent call last):
File "h:\caption\joy-caption-batch\batch.py", line 119, in
clip_model.to("cuda")
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch\nn\modules\module.py", line 1174, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch\nn\modules\module.py", line 780, in _apply
module._apply(fn)
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch\nn\modules\module.py", line 780, in _apply
module._apply(fn)
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch\nn\modules\module.py", line 805, in _apply
param_applied = fn(param)
^^^^^^^^^
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch\nn\modules\module.py", line 1160, in convert
return t.to(
^^^^^
File "h:\caption\joy-caption-batch\venv\Lib\site-packages\torch\cuda\ init__.py", line 305, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
(venv) h:\caption\joy-caption-batch>