MacOS Sonoma - Error while trying to update pip

[wolfspyre]

Hiya!

not really sure if I'm the only one running into this; but It happens every time I try...

SkwirrelBane:~ loiosh$ cd ../Shared/MLModels/LMStudio-ChromaDB/
SkwirrelBane:LMStudio-ChromaDB loiosh$ ls
latest      release
SkwirrelBane:LMStudio-ChromaDB loiosh$ cd release/
SkwirrelBane:release loiosh$ ls
v3.04           v3.04.zip       v3.3.1          v3.3.1.tar.gz       v3.3.2          v3.3.2.tar.gz       wipe_and_restore-3.3.2
SkwirrelBane:release loiosh$ more wipe_and_restore-3.3.2
#!/usr/bin/env bash
rm -rf v3.3.2/ ; tar xzf v3.3.2.tar.gz && mv ChromaDB-Plugin-for-LM-Studio-3.3.2/ v3.3.2
SkwirrelBane:release loiosh$ ./wipe_and_restore-3.3.2

SkwirrelBane:release loiosh$ cd ..
SkwirrelBane:LMStudio-ChromaDB loiosh$ ls 
lrwxrwxrwx   1 loiosh  2052    14 Jan 27 21:34 latest -> release/v3.3.2
drwxr-xr-x@  5 loiosh  2052    11 Jan 29 16:22 release
SkwirrelBane:LMStudio-ChromaDB loiosh$ cd latest/src/

so, I should be in a pristine extract of the 3.3.2 zipfile

SkwirrelBane:src loiosh$ python -m venv .
SkwirrelBane:src loiosh$ source bin/activate
(src) SkwirrelBane:src loiosh$

so far so good, right?

(src) SkwirrelBane:src loiosh$ python -m pip install --upgrade pip
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 980, in exec_module
  File "<frozen site>", line 616, in <module>
  File "<frozen site>", line 599, in main
  File "<frozen site>", line 531, in venv
  File "<frozen site>", line 384, in addsitepackages
  File "<frozen site>", line 226, in addsitedir
  File "<frozen site>", line 179, in addpackage
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte
(src) SkwirrelBane:src loiosh$

sigh

got any ideas?

[BBC-Esq]

I just updated a bunch of scripts and will be doing a release soon v3.4 that sets the encoding to utf-8 across all scripts, which should resolve these kinds of errors.

[wolfspyre]

Hm....

OSVer & arch

SkwirrelBane:LMStudio-ChromaDB  loiosh$ sysctl kern.osproductversion;   arch
kern.osproductversion: 14.4
arm64

Xcode:

SkwirrelBane:LMStudio-ChromaDB  loiosh$ xcode-select -p
/Applications/Xcode-beta.app/Contents/Developer
SkwirrelBane:git-repo loiosh$ cat /Applications/Xcode-beta.app/Contents/version.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>BuildVersion</key>
    <string>126</string>
    <key>CFBundleShortVersionString</key>
    <string>15.3</string>
    <key>CFBundleVersion</key>
    <string>22617</string>
    <key>ProductBuildVersion</key>
    <string>15E5178i</string>
    <key>ProjectName</key>
    <string>IDEApplication</string>
    <key>SourceVersion</key>
    <string>22617000000000000</string>
</dict>
</plist>

python

SkwirrelBane:LMStudio-ChromaDB loiosh$ python --version
Python 3.11.7

brew

SkwirrelBane:LMStudio-ChromaDB loiosh$ brew list
==> Formulae
abseil              gl2ps               libbluray           libxau              pango               snappy
aom             glew                libevent            libxcb              pcre                speex
aribb24             glib                libgit2             libxdmcp            pcre2               sqlite
assimp              glog                libgit2@1.6         libxext             pixman              srt
bash                gmp             libidn2             libxrender          pkg-config          suite-sparse
bash-completion         gnu-sed             libmagic            libyaml             portaudio           svt-av1
boost               gnutls              libmicrohttpd           little-cms2         protobuf            tailwindcss
brotli              go              libmng              lz4             pugixml             tart
c-ares              graphite2           libmpc              lzo             pv              tbb
ca-certificates         grpc                libnghttp2          mbedtls             pybind11            tesseract
cairo               harfbuzz            libogg              md4c                python-dateutil         the_silver_searcher
ceres-solver            hdf5                libomp              metis               python-distlib          theora
cjson               highway             libpng              molten-vk           python-filelock         unbound
cmake               hugo                librist             mpdecimal           python-jinja            utf8cpp
dav1d               hunspell            libsamplerate           mpfr                python-magic            webp
dbus                hwloc               libsndfile          mpg123              python-markupsafe       wget
double-conversion       icu4c               libsodium           ncnn                python-mpmath           x264
eigen               imath               libsoxr             netcdf              python-networkx         x265
flac                isl             libssh              nettle              python-platformdirs     xorgproto
fontconfig          jasper              libssh2             node                python-typing-extensions    xvid
freetype            jpeg-turbo          libtasn1            oniguruma           python@3.11         xz
frei0r              jpeg-xl             libtiff             openblas            pyyaml              z3
fribidi             jq              libunibreak         opencore-amr            qt              zeromq
gcc             jsoncpp             libunistring            openexr             rav1e               zimg
gdbm                lame                libuv               openjpeg            re2             zstd
gettext             leptonica           libvidstab          openssl@1.1         readline
gflags              libaec              libvmaf             openssl@3           rubberband
giflib              libarchive          libvorbis           opus                sdl2
git             libass              libvpx              p11-kit             six
git-lfs             libb2               libx11              pandoc              sleef

downloads:

SkwirrelBane:LMStudio-ChromaDB cd release; md5 v3.4.zip
MD5 (v3.4.zip) = dcd2fbcca97c0a587c70e8c7260090b1
SkwirrelBane:release loiosh$ md5 v3.4.tar.gz
MD5 (v3.4.tar.gz) = f72de29663be87003cb66e1ad768014c
SkwirrelBane:release loiosh$

extract

SkwirrelBane:release loiosh$  tar xzf v3.4.tar.gz && mv ChromaDB-Plugin-for-LM-Studio-3.4/ v3.4
SkwirrelBane:release loiosh$  cd .. ; ln -s release/v3.4 latest
SkwirrelBane:LMStudio-ChromaDB loiosh$ ls -l
total 3
drwxrwxrwx  5 loiosh  2052   11 Jan 30 14:05 git-repo
lrwxrwxrwx@ 1 loiosh  2052   12 Jan 29 23:39 latest -> release/v3.4
drwxr-xr-x@ 5 loiosh  2052   16 Jan 30 14:43 release

venv

SkwirrelBane:LMStudio-ChromaDB loiosh$ cd latest/src
SkwirrelBane:src loiosh$ ls
CSS                 document_processor.py           gui_tabs_settings_database_create.py    gui_tabs_vector_models.py       stop_sign.png
Tokenizer               download_model.py           gui_tabs_settings_database_query.py initialize.py               test.py
User_Manual             extract_metadata.py         gui_tabs_settings_models.py     loader_images.py            transcribe_module.py
bark_module.py              gui.py                  gui_tabs_settings_server.py     metrics_bar.py              utilities.py
check_gpu.py                gui_tabs.py             gui_tabs_settings_vision.py     replace_pdf.py              vision_cogvlm_module.py
choose_documents_and_vector_model.py    gui_tabs_databases.py           gui_tabs_settings_whisper.py        requirements.txt            vision_llava_module.py
config.yaml             gui_tabs_settings.py            gui_tabs_tools.py           select_model.py             vision_model_table.html
constants.py                gui_tabs_settings_bark.py       gui_tabs_tools_transcribe.py        server_connector.py         voice_recorder_module.py
database_interactions.py        gui_tabs_settings_database.py       gui_tabs_tools_vision.py        setup.py
SkwirrelBane:src loiosh$ python -m venv .
SkwirrelBane:src loiosh$ source bin/activate
(src) SkwirrelBane:src loiosh$

setup

(src) SkwirrelBane:src loiosh$ python -m pip install --upgrade pip
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 980, in exec_module
  File "<frozen site>", line 616, in <module>
  File "<frozen site>", line 599, in main
  File "<frozen site>", line 531, in venv
  File "<frozen site>", line 384, in addsitepackages
  File "<frozen site>", line 226, in addsitedir
  File "<frozen site>", line 179, in addpackage
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte
(src) SkwirrelBane:src loiosh$

cue sad trombone

Got any other ideas? or suggestions on wot to inspect?

[BBC-Esq]

I don't have macos nor linux...can you explain a little?

[wolfspyre]

sure! I'm Wolfspyre on the lms discord, may be easier to share context in chat

[wolfspyre]

I think this is because the working directory mountpoint was NFS-based, and there seems to be some wonkyness.

For anyone else running into issues like this, Try using a filesystem which doesn't require ._ files to be created (ie a filesystem which supports MacOS extended attributes natively ) as the working directory to see if that resolves your problem