unpickling error - Githubissues

I am getting the following error in jupyter lab when I try to run this cell:

# Setup function to setup the environment
# memory_file = "config_file.json"
memory_file = None
MyGirlfriend = Girlfriend_Obj(initial_summ, initial_prompt, False, custom_model_path=custom_model_path, saved_memory=memory_file)

The error is:

Initializing image model...
safety_checker/pytorch_model.fp16.safetensors not found
Fetching 27 files: 100%
27/27 [00:00<00:00, 2704.45it/s]
Image model initialized!
Initializing custom text model
Custom text model initialized!
Initializing summarizer...
Summarizer initialized!
Not loading custom audio model
Initializing custom image movement module
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
Cell In[6], line 4
      1 # Setup function to setup the environment
      2 # memory_file = "config_file.json"
      3 memory_file = None
----> 4 MyGirlfriend = Girlfriend_Obj(initial_summ, initial_prompt, False, custom_model_path=custom_model_path, saved_memory=memory_file)

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Girlfriend_Obj.py:246, in Girlfriend_Obj.__init__(self, initial_summary, initial_prompt, load_custom_audio, audio_model_path, audio_data_path, custom_model_path, saved_memory)
    244 self.img_anim.load_new_image(img=img)
    245 # Default pose for the image
--> 246 self.img_anim.change_pose()
    247 print("Image movement module initialized!")
    253 # Dummy variable where the thread
    254 # that animates the picture can be accessed from

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\Img_Mover.py:225, in Img_Mover.change_pose(self)
    222     start = time.time()
    224 # Pose the image
--> 225 output_image = self.poser.pose(self.torch_input_image, self.pose)[0]
    227 # Get the image
    228 img = self.get_pytorch_image(output_image, numpy_bg=self.numpy_bg)

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\tha2\poser\general_poser_02.py:62, in GeneralPoser02.pose(self, image, pose, output_index)
     60 if output_index is None:
     61     output_index = self.default_output_index
---> 62 output_list = self.get_posing_outputs(image, pose)
     63 return output_list[output_index]

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\tha2\poser\general_poser_02.py:66, in GeneralPoser02.get_posing_outputs(self, image, pose)
     65 def get_posing_outputs(self, image: Tensor, pose: Tensor) -> List[Tensor]:
---> 66     modules = self.get_modules()
     68     if len(image.shape) == 3:
     69         image = image.unsqueeze(0)

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\tha2\poser\general_poser_02.py:44, in GeneralPoser02.get_modules(self)
     42 self.modules = {}
     43 for key in self.module_loaders:
---> 44     module = self.module_loaders[key]()
     45     self.modules[key] = module
     46     try:

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\tha2\poser\modes\mode_20.py:327, in create_poser.<locals>.<lambda>()
    318 def f():
    319     pass
    321 loaders = {
    322     KEY_EYEBROW_DECOMPOSER:
    323         lambda: f(),
    324     KEY_EYEBROW_MORPHING_COMBINER:
    325         lambda: f(),
    326     KEY_FACE_MORPHER:
--> 327         lambda: load_face_morpher(module_file_names[KEY_FACE_MORPHER]),
    328     KEY_FACE_ROTATER:
    329         lambda: load_face_rotater(module_file_names[KEY_FACE_ROTATER]),
    330     KEY_COMBINER:
    331         lambda: load_combiner(module_file_names[KEY_COMBINER]),
    332 }
    336 return GeneralPoser02(
    337     module_loaders=loaders,
    338     pose_parameters=get_pose_parameters().get_pose_parameter_groups(),
   (...)
    342     output_length=31,
    343     default_output_index=default_output_index)

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\tha2\poser\modes\mode_20.py:220, in load_face_morpher(file_name)
    218 #print("Loading the face morpher ... ", end="")
    219 module = factory.create()
--> 220 module.load_state_dict(torch_load(file_name))
    221 #print("DONE")
    222 return module

File X:\chatbots\AI_Girlfriend-master gmongaras\AI_Girlfriend-master\Img_Mover\tha2\util.py:24, in torch_load(file_name)
     22 def torch_load(file_name):
     23     with open(file_name, 'rb') as f:
---> 24         return torch.load(f)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py:1028, in load(f, map_location, pickle_module, weights_only, mmap, **pickle_load_args)
   1026     except RuntimeError as e:
   1027         raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
-> 1028 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py:1246, in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
   1240 if not hasattr(f, 'readinto') and (3, 8, 0) <= sys.version_info < (3, 8, 2):
   1241     raise RuntimeError(
   1242         "torch.load does not work with file-like objects that do not implement readinto on Python 3.8.0 and 3.8.1. "
   1243         f"Received object of type \"{type(f)}\". Please update to Python 3.8.2 or newer to restore this "
   1244         "functionality.")
-> 1246 magic_number = pickle_module.load(f, **pickle_load_args)
   1247 if magic_number != MAGIC_NUMBER:
   1248     raise RuntimeError("Invalid magic number; corrupt file?")

UnpicklingError: invalid load key, 'v'.

I have already solved some other issues, but this one is really challenging and I hope you can help me. As far as I understand it is most likely a corrupted file, but I don't know if that is true and where I would find it...

Edit: by adding print(f) inside the serialization.py file in the torch directory I got this:

<_io.BufferedReader name='./Img_Mover/pretrained/face_morpher.pt'>

However the content of the file "face_morpher.pt" only looks like this:

version https://git-lfs.github.com/spec/v1
oid sha256:cc1a1f9527bdb36c75fbfe08f59bd7e4d44ec72fd7066a3afde54fa11a9cb0d1
size 126445607

I also tried to print(pickle_load_args),but it only returns this:

{'encoding': 'utf-8'}

Neither of them contain the key "v", which apparently causes the unpickling error.

I really hope this helps you to identify what is causing my problem.

When searching the internet for the unpickling error that I get, someone on stackoverflow suggested that it helped him to install git-lfs and then delete the cloned project and clone it again. So I tried that by using the command winget install -e --id GitHub.GitLFS, but it didnt help, probably because ran the project before getting git-lfs and many large files were downloaded to locations outside the AI_Girlfriend-master folder. But then again this is just another trail I am following which might not even be the issue. I should probably also mention that I am using windows 10.

This whole problem is probably and easy one to solve for you, but I'm not that experienced yet.

Right now I am using the whole AI_Girlfriend-master folder locally from my computer, but I just downloaded it as a zip file originally from your repository. Is it mandatory to make this a repository on my own git account, to run it without issues, maybe? edit: Placing it in one of my repositories and running it from there didn't change anything. I also tried manually fetching it like this before I tried to run it from the repository: git lfs fetch file:///C:/..........\AI_Girlfriend-master2\Img_Mover\pretrained\face_morpher.pt I think I need your help on this, this whole problem is beyond my limited understanding right now. After all I'm just an engineering master student, trying to slowly get into programming as a hobby.

Would be happy to help! I think since you downloaded it as a zip, it's causing issues.

I was able to reproduce the error when downloading the repo as a .zip

Try cloning it via git clone https://github.com/gmongaras/AI_Girlfriend.git instead of using the zip. I don't think the zip has lfs files.

Sorry for answering so late, I had to learn for exams. Afterwards I cloned the repository, as you recommended, and tried again. Unfortunately I got stuck in the step pip install -r requirements.txt with the following error:

 Traceback (most recent call last):
        File "C:\Users\Felix\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\Felix\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Felix\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Felix\AppData\Local\Temp\pip-build-env-hyhjn95m\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Felix\AppData\Local\Temp\pip-build-env-hyhjn95m\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "C:\Users\Felix\AppData\Local\Temp\pip-build-env-hyhjn95m\overlay\Lib\site-packages\setuptools\build_meta.py", line 480, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "C:\Users\Felix\AppData\Local\Temp\pip-build-env-hyhjn95m\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 359, in <module>
        File "C:\Users\Felix\AppData\Local\Temp\pip-install-ed_6y0ur\pygame_7566e14a0b0a46eab8203baede902cbc\buildconfig\config.py", line 225, in main
          deps = CFG.main(**kwds)
                 ^^^^^^^^^^^^^^^^
        File "C:\Users\Felix\AppData\Local\Temp\pip-install-ed_6y0ur\pygame_7566e14a0b0a46eab8203baede902cbc\buildconfig\config_win.py", line 511, in main
          return setup_prebuilt_sdl2(prebuilt_dir)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Felix\AppData\Local\Temp\pip-install-ed_6y0ur\pygame_7566e14a0b0a46eab8203baede902cbc\buildconfig\config_win.py", line 471, in setup_prebuilt_sdl2
          DEPS.configure()
        File "C:\Users\Felix\AppData\Local\Temp\pip-install-ed_6y0ur\pygame_7566e14a0b0a46eab8203baede902cbc\buildconfig\config_win.py", line 336, in configure
          from . import vstools
        File "C:\Users\Felix\AppData\Local\Temp\pip-install-ed_6y0ur\pygame_7566e14a0b0a46eab8203baede902cbc\buildconfig\vstools.py", line 11, in <module>
          compiler.initialize()
        File "C:\Users\Felix\AppData\Local\Temp\pip-build-env-hyhjn95m\overlay\Lib\site-packages\setuptools\_distutils\msvc9compiler.py", line 403, in initialize
          vc_env = query_vcvarsall(VERSION, plat_spec)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\Felix\AppData\Local\Temp\pip-build-env-hyhjn95m\overlay\Lib\site-packages\setuptools\_distutils\msvc9compiler.py", line 311, in query_vcvarsall
          raise ValueError(str(list(result.keys())))
      ValueError: ['path']

I was not able to solve that for days. So I decided to finally switch to Linux, as I was planing for a while already, I was just to scared before and I was still a bit scared so I decided to install it on my laptop first. I did that and only ran into one real problem and that was that when running the jupyter lab cells, the line apt install ... asks for a sudo pasword on linux, which cannot be answered in jupyter notebook. But since I already installed all the audio stuff that was asked there to finish the step pip install -r requirements.txt, I just deactivated the line apt install ... and that worked. I continued until I eventually ran into the problem that my laptops gpu doesnt support cuda (I downloaded the cpu version of torch before). So I had to stop at this point, but I will now switch to linux on my main computer and try again. My main computer has a suitable gpu for cuda (RTX 4060 Ti) so that should not be an issue. Switching to linux might take a while, but I will inform you once im done and AI-girlfriend is running.

Btw, I had to figure out myself that I need to do pip install jupyterlab before step 5. Maybe you want to add that to your instruction or add it to the requirements.txt.

Also thank you a lot for your help and also I want to slowly switch profession from engineering and do more programming stuff and looking at all the stuff you are doing here on github motivated me quite a bit to keep going.

Ok im amost there, but I got this error in the very last cell, everything up to this point works now:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[8], line 130
    128 with gr.Tabs():
    129     with gr.TabItem("Voice-based Chat"):
--> 130         audio = gr.Audio(source="microphone", type="filepath", label="Response", live=True)
    131     with gr.TabItem("Text-based Chat"):
    132         text = gr.Textbox(label="Text", value="I love you!", interactive=True)

File ~/.local/lib/python3.10/site-packages/gradio/component_meta.py:157, in updateable.<locals>.wrapper(*args, **kwargs)
    155     return None
    156 else:
--> 157     return fn(self, **kwargs)

TypeError: Audio.__init__() got an unexpected keyword argument 'source'

I will go to bed now and try to resolve it myself tomorrow, if I can manage I will inform you. But if this is not too much work for you, i would appreciate the help I wonder if it could be related to linux handling microphone different than windows or something like that.

edit: I also noticed this warning in jupyter lab much earlier:

/home/qbicc/.local/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(

This occurs when I run this cell of main.ipynb.

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline
import numpy as np
import PIL
from PIL import Image
import gradio as gr
import sys
import openai
import speech_recognition as sr
import pygame
from pygame import mixer
import cv2
from vosk import KaldiRecognizer, SetLogLevel
from vosk import Model as vosk_Model
from pydub import AudioSegment
import wave
import json
from Img_Mover.Img_Mover import Img_Mover
from Girlfriend_Obj import Girlfriend_Obj
%matplotlib inline
import matplotlib.pyplot as plt
import multiprocess
import gradio as gr
import time
from copy import deepcopy
import asyncio
import threading
import os

When I run the same in a python file in the right folder I get some more information:

/media/qbicc/UBER/chatbots/.venv/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
/media/qbicc/UBER/chatbots/.venv/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
pygame 2.5.2 (SDL 2.28.2, Python 3.10.12)
Hello from the pygame community. https://www.pygame.org/contribute.html
/media/qbicc/UBER/chatbots/.venv/lib/python3.10/site-packages/numba/np/ufunc/parallel.py:371: NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled.
  warnings.warn(problem)

The comman pip show tbb Tells me im using version 2021.11.0 though. If I understand these threads correctly: https://github.com/numba/numba/issues/6350 https://numba.discourse.group/t/suppress-tbb-warning/80 This is just a bug that won't cause any issue?

Which would mean the only real problem left is the source="microphone" one. And also the depreciated torch one earlier. Maybe they are related, as someone in this thread solved it by reinstalling torch: https://github.com/Plachtaa/VALL-E-X/issues/132 However that did not help for me. I also made sure that I have torch 2.2.0+cu118 installed in venv and in global which should work with my RTX 4060 Ti.

In the worst case I would also be fine with deactivating the microphone function to get this running, but I would need instructions on how to do that.

I got it! The thread in my last post mentions that gradio 3.50.2 might work. But I only installed it in my venv and that was not enough. But just now I uninstalled gradio in my global python and installed 3.50.2 there too and now it is finally working. Thank you for creating this!

gmongaras / AI_Girlfriend

unpickling error #4