controlnet incompatibility due to Timm version

[knzro]

Describe the bug I am having an incompatibility with timm<=0.9.5 required by controlnet, it seems that specific version makes the interrogation process fail. To Reproduce Steps to reproduce the behavior:

1. install timm==0.9.5
2. select wd interrogator
3. Load dataset
4. See error:

   [tag-editor] Preprocess completed
   0%|                                                                                            | 0/2 [00:02<?, ?it/s]
   Traceback (most recent call last):
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
   output = await app.get_blocks().process_api(
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
   result = await self.call_function(
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
   prediction = await anyio.to_thread.run_sync(
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
   return await get_asynclib().run_sync_in_worker_thread(
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
   return await future
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
   result = context.run(func, *args)
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
   response = f(*args, **kwargs)
   File "D:\wifediff\stable-diffusion-webui\extensions\stable-diffusion-webui-dataset-tag-editor\scripts\tag_editor_ui\block_load_dataset.py", line 158, in load_files_from_dir
   dte_instance.load_dataset(
   File "D:\wifediff\stable-diffusion-webui\extensions\stable-diffusion-webui-dataset-tag-editor\scripts\dataset_tag_editor\dte_logic.py", line 797, in load_dataset
   tg.start()
   File "D:\wifediff\stable-diffusion-webui\extensions\stable-diffusion-webui-dataset-tag-editor\scripts\dataset_tag_editor\taggers_builtin.py", line 115, in start
   self.tagger_inst.load()
   File "D:\wifediff\stable-diffusion-webui\extensions\stable-diffusion-webui-dataset-tag-editor\scripts\dataset_tag_editor\interrogators\waifu_diffusion_tagger_timm.py", line 56, in load
   self.model.load_state_dict(state_dict)
   File "D:\wifediff\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load_state_dict
   raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
   RuntimeError: Error(s) in loading state_dict for VisionTransformer:
       Missing key(s) in state_dict: "cls_token".
       size mismatch for pos_embed: copying a param with shape torch.Size([1, 784, 768]) from checkpoint, the shape in current model is torch.Size([1, 197, 768]).

Expected behavior Dataset is tagged

Screenshots

Environment (please complete the following information):

• OS: windows
• Browser: firefox
• Version of SD WebUI: v1.9.3, by AUTOMATIC1111
• Version of this app: commit d6c7dcef

Additional context The issue went away by updating timm, the problem being that version 0.9.5 seems to be a requisite for controlnet thus the workaround requiring to edit the controlnet extension requirements file or it will be downgraded and set to that version on launch. timm-1.0.8 seems to work fine and controlnet hasn't imploded yet.

[toshiaki1729]

It is not possible to address every issues caused by other extensions. so I recommend using the standalone version.