Multiple Errors During Setup in Colab

[ninjatacoshell]

While running the setup in Google colab, the following error appears:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tokenizers 0.15.2 requires huggingface_hub<1.0,>=0.16.4, but you have huggingface-hub 0.14.1 which is incompatible.
transformers 4.38.2 requires huggingface-hub<1.0,>=0.19.3, but you have huggingface-hub 0.14.1 which is incompatible.

Setup continues, but later produces another error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 2.2.1+cu121 requires torch==2.2.1, but you have torch 2.0.1 which is incompatible.
torchtext 0.17.1 requires torch==2.2.1, but you have torch 2.0.1 which is incompatible.
torchvision 0.17.1+cu121 requires torch==2.2.1, but you have torch 2.0.1 which is incompatible.
…
WARNING: The following packages were previously imported in this runtime:
  [pydevd_plugins]
You must restart the runtime in order to use newly installed versions.

This is accompanied by a pop-up window that says,

Restart session
WARNING: The following packages were previously imported in this runtime:`
  [pydevd_plugins]
You must restart the runtime in order to use newly installed versions.
Restarting will lose all runtime state, including local variables.

Whether the session is restarted or not, the setup continues until local and public URLs are generated. The gradio instance works, but once the "Train Model" button is pushed a third error occurs:

Traceback (most recent call last):
  File "/usr/local/bin/accelerate", line 5, in <module>
    from accelerate.commands.accelerate_cli import main
  File "/usr/local/lib/python3.10/dist-packages/accelerate/commands/accelerate_cli.py", line 19, in <module>
    from accelerate.commands.estimate import estimate_command_parser
  File "/usr/local/lib/python3.10/dist-packages/accelerate/commands/estimate.py", line 34, in <module>
    import timm
  File "/usr/local/lib/python3.10/dist-packages/timm/__init__.py", line 2, in <module>
    from .models import create_model, list_models, is_model, list_modules, model_entrypoint, \
  File "/usr/local/lib/python3.10/dist-packages/timm/models/__init__.py", line 1, in <module>
    from .beit import *
  File "/usr/local/lib/python3.10/dist-packages/timm/models/beit.py", line 49, in <module>
    from timm.data import IMAGENET_DEFAULT_MEAN, IMAGENET_DEFAULT_STD
  File "/usr/local/lib/python3.10/dist-packages/timm/data/__init__.py", line 5, in <module>
    from .dataset import ImageDataset, IterableImageDataset, AugMixDataset
  File "/usr/local/lib/python3.10/dist-packages/timm/data/dataset.py", line 12, in <module>
    from .parsers import create_parser
  File "/usr/local/lib/python3.10/dist-packages/timm/data/parsers/__init__.py", line 1, in <module>
    from .parser_factory import create_parser
  File "/usr/local/lib/python3.10/dist-packages/timm/data/parsers/parser_factory.py", line 3, in <module>
    from .parser_image_folder import ParserImageFolder
  File "/usr/local/lib/python3.10/dist-packages/timm/data/parsers/parser_image_folder.py", line 11, in <module>
    from timm.utils.misc import natural_key
  File "/usr/local/lib/python3.10/dist-packages/timm/utils/__init__.py", line 2, in <module>
    from .checkpoint_saver import CheckpointSaver
  File "/usr/local/lib/python3.10/dist-packages/timm/utils/checkpoint_saver.py", line 15, in <module>
    from .model import unwrap_model, get_state_dict
  File "/usr/local/lib/python3.10/dist-packages/timm/utils/model.py", line 8, in <module>
    from torchvision.ops.misc import FrozenBatchNorm2d
  File "/usr/local/lib/python3.10/dist-packages/torchvision/__init__.py", line 6, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
  File "/usr/local/lib/python3.10/dist-packages/torchvision/_meta_registrations.py", line 4, in <module>
    import torch._custom_ops
ModuleNotFoundError: No module named 'torch._custom_ops'

This final error occurs whether the runtime is restarted or not.

UPDATE ON 9/23/2024

Now I get the following error if I restart the session:

Traceback (most recent call last):
  File "/usr/local/bin/accelerate", line 5, in <module>
    from accelerate.commands.accelerate_cli import main
  File "/usr/local/lib/python3.10/dist-packages/accelerate/__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 34, in <module>
    from huggingface_hub import split_torch_state_dict_into_shards
ImportError: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (/usr/local/lib/python3.10/dist-packages/huggingface_hub/__init__.py)

[effegma-ai]

I am experiencing the same problem.

[sunnysharma12]

Hi, I also have same issues.

@camenduru - is there any plan to keep this Google Colab updated?

Is there any other reliable Colab based on Kohya?

[monsterhunters]

@sunnysharma12 @ninjatacoshell @effegma-ai , you can use colab created by me --> https://github.com/monsterhunters/Lora-Training-GUI

[sunnysharma12]

@sunnysharma12 @ninjatacoshell @effegma-ai , you can use colab created by me --> https://github.com/monsterhunters/Lora-Training-GUI

Thanks @monsterhunters - it worked for me.