search

hollowstrawberry / kohya-colab

Accessible Google Colab notebooks for Stable Diffusion Lora training, based on the work of kohya-ss and Linaqruf
GNU General Public License v3.0
561 stars 80 forks source link

No tensorflow-python? #73

Closed Metalhead33 closed 6 months ago

Metalhead33 commented 6 months ago 
💿 Checking dataset...
📁MyDrive/Lora_Dataset/metalhead33_only/
📈 Found 123 images with 2 repeats, equaling 246 steps.
📉 Divide 246 steps by 2 batch size to get 123.0 steps per epoch.
🔮 There will be 10 epochs, for around 1230 total training steps.

🏭 Installing dependencies...

fatal: destination path '/content/kohya-trainer' already exists and is not an empty directory.
HEAD is now at 9a67e0d Merge pull request #610 from lubobill1990/patch-1
27 packages can be upgraded. Run 'apt list --upgradable' to see them.
aria2 is already the newest version (1.36.0-1).
0 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
WARNING: Ignoring invalid distribution -ensorflow (/usr/local/lib/python3.10/dist-packages)
Processing /content/kohya-trainer
  Preparing metadata (setup.py) ... done
Requirement already satisfied: accelerate==0.15.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 1)) (0.15.0)
Requirement already satisfied: transformers==4.26.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 2)) (4.26.0)
Requirement already satisfied: ftfy==6.1.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 3)) (6.1.1)
Requirement already satisfied: albumentations==1.3.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (1.3.1)
Requirement already satisfied: opencv-python==4.8.0.76 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 5)) (4.8.0.76)
Requirement already satisfied: einops==0.6.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 6)) (0.6.0)
Requirement already satisfied: diffusers==0.10.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (0.10.2)
Requirement already satisfied: pytorch-lightning==1.9.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (1.9.0)
Requirement already satisfied: bitsandbytes==0.41.3.post2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (0.41.3.post2)
Requirement already satisfied: tensorflow==2.14.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 10)) (2.14.0)
Requirement already satisfied: safetensors==0.4.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 11)) (0.4.1)
Requirement already satisfied: toml==0.10.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (0.10.2)
Requirement already satisfied: voluptuous==0.13.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (0.13.1)
Requirement already satisfied: lion_pytorch==0.0.6 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 14)) (0.0.6)
Requirement already satisfied: dadaptation==3.1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (3.1)
Requirement already satisfied: prodigyopt==1.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 16)) (1.0)
Requirement already satisfied: lycoris_lora==0.1.4 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 17)) (0.1.4)
Requirement already satisfied: xformers==0.0.22.post7 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 18)) (0.0.22.post7)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.15.0->-r requirements.txt (line 1)) (1.23.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.15.0->-r requirements.txt (line 1)) (23.2)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.15.0->-r requirements.txt (line 1)) (5.9.5)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate==0.15.0->-r requirements.txt (line 1)) (6.0.1)
Requirement already satisfied: torch>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.15.0->-r requirements.txt (line 1)) (2.1.0+cu121)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (0.19.4)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (2023.6.3)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (2.31.0)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (0.13.3)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.26.0->-r requirements.txt (line 2)) (4.66.1)
Requirement already satisfied: wcwidth>=0.2.5 in /usr/local/lib/python3.10/dist-packages (from ftfy==6.1.1->-r requirements.txt (line 3)) (0.2.12)
Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from albumentations==1.3.1->-r requirements.txt (line 4)) (1.11.4)
Requirement already satisfied: scikit-image>=0.16.1 in /usr/local/lib/python3.10/dist-packages (from albumentations==1.3.1->-r requirements.txt (line 4)) (0.19.3)
Requirement already satisfied: qudida>=0.0.4 in /usr/local/lib/python3.10/dist-packages (from albumentations==1.3.1->-r requirements.txt (line 4)) (0.0.4)
Requirement already satisfied: opencv-python-headless>=4.1.1 in /usr/local/lib/python3.10/dist-packages (from albumentations==1.3.1->-r requirements.txt (line 4)) (4.8.1.78)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.10/dist-packages (from diffusers==0.10.2->-r requirements.txt (line 7)) (7.0.0)
Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from diffusers==0.10.2->-r requirements.txt (line 7)) (9.4.0)
Requirement already satisfied: fsspec[http]>2021.06.0 in /usr/local/lib/python3.10/dist-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (2023.6.0)
Requirement already satisfied: torchmetrics>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (1.2.1)
Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (4.5.0)
Requirement already satisfied: lightning-utilities>=0.4.2 in /usr/local/lib/python3.10/dist-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (0.10.0)
Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (1.4.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (1.6.3)
Requirement already satisfied: flatbuffers>=23.5.26 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (23.5.26)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (0.5.4)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (0.2.0)
Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (3.9.0)
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (16.0.6)
Requirement already satisfied: ml-dtypes==0.2.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (0.2.0)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (3.3.0)
Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (3.20.3)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (67.7.2)
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (2.4.0)
Requirement already satisfied: wrapt<1.15,>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (1.14.1)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (0.34.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (1.60.0)
Requirement already satisfied: tensorboard<2.15,>=2.14 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (2.14.1)
Requirement already satisfied: tensorflow-estimator<2.15,>=2.14.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (2.14.0)
Requirement already satisfied: keras<2.15,>=2.14.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.14.0->-r requirements.txt (line 10)) (2.14.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->accelerate==0.15.0->-r requirements.txt (line 1)) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->accelerate==0.15.0->-r requirements.txt (line 1)) (3.2.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->accelerate==0.15.0->-r requirements.txt (line 1)) (3.1.2)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.4.0->accelerate==0.15.0->-r requirements.txt (line 1)) (2.1.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow==2.14.0->-r requirements.txt (line 10)) (0.42.0)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (3.9.1)
Requirement already satisfied: scikit-learn>=0.19.1 in /usr/local/lib/python3.10/dist-packages (from qudida>=0.0.4->albumentations==1.3.1->-r requirements.txt (line 4)) (1.2.2)
Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.10/dist-packages (from scikit-image>=0.16.1->albumentations==1.3.1->-r requirements.txt (line 4)) (2.31.6)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.10/dist-packages (from scikit-image>=0.16.1->albumentations==1.3.1->-r requirements.txt (line 4)) (2023.12.9)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-image>=0.16.1->albumentations==1.3.1->-r requirements.txt (line 4)) (1.5.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (2.17.3)
Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (1.0.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (3.5.1)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (3.0.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.26.0->-r requirements.txt (line 2)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.26.0->-r requirements.txt (line 2)) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.26.0->-r requirements.txt (line 2)) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.26.0->-r requirements.txt (line 2)) (2023.11.17)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata->diffusers==0.10.2->-r requirements.txt (line 7)) (3.17.0)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (1.9.4)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (1.4.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (4.0.3)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (5.3.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (0.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (1.3.1)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.19.1->qudida>=0.0.4->albumentations==1.3.1->-r requirements.txt (line 4)) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.19.1->qudida>=0.0.4->albumentations==1.3.1->-r requirements.txt (line 4)) (3.2.0)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=1.0.1->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.4.0->accelerate==0.15.0->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (0.5.1)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow==2.14.0->-r requirements.txt (line 10)) (3.2.2)
Building wheels for collected packages: library
  Building wheel for library (setup.py) ... done
  Created wheel for library: filename=library-0.0.0-py3-none-any.whl size=84080 sha256=1f1e3ac9298ae53366fb01e0068adacce4bbf7459d9bc5616d7a1db3ddcf6fdd
  Stored in directory: /root/.cache/pip/wheels/d8/ef/ac/94db7d75dc3a8e81533bc2d001d79467c693ec69ed02bbee59
Successfully built library
WARNING: Ignoring invalid distribution -ensorflow (/usr/local/lib/python3.10/dist-packages)
Installing collected packages: library
  Attempting uninstall: library
    Found existing installation: library 0.0.0
    Uninstalling library-0.0.0:
      Successfully uninstalled library-0.0.0
Successfully installed library-0.0.0

---------------------------------------------------------------------------

ModuleNotFoundError                       Traceback (most recent call last)

[<ipython-input-10-7cc52c843274>](https://localhost:8080/#) in <cell line: 510>()
    508                      "### There will be several files, you should try the latest version (the file with the largest number next to it)"))
    509 
--> 510 main()

11 frames

[/usr/local/lib/python3.10/dist-packages/tensorflow/__init__.py](https://localhost:8080/#) in <module>
     36 import typing as _typing
     37 
---> 38 from tensorflow.python.tools import module_util as _module_util
     39 from tensorflow.python.util.lazy_loader import LazyLoader as _LazyLoader
     40 

ModuleNotFoundError: No module named 'tensorflow.python'

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------
Metalhead33 commented 6 months ago

Solved, somehow. I have no idea how though.