search

h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/
http://h2o.ai
Apache License 2.0
11.42k stars 1.25k forks source link

Windows fatal exception: access violation after uploading a pdf file #1335

Open isaac-aburto opened 9 months ago

isaac-aburto commented 9 months ago

I am working on an EC2 instance (g4dn.xlarge)

The installation is going well. It works perfectly if I upload any other type of file (txt, csv, xml...), but when I try to upload a PDF file I get the error and the application stops.

Windows fatal exception: access violation

Thread 0x000002cc (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\concurrent\futures\thread.py", line 81 in _worker
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00001d34 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\concurrent\futures\thread.py", line 81 in _worker
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00000764 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 797 in _poll
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 444 in select
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 1871 in _run_once
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 603 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 321 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 636 in run_until_complete
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\runners.py", line 44 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\uvicorn\server.py", line 62 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\uvicorn\main.py", line 587 in run
  File "C:\Users\Administrator\h2ogpt\openai_server\server.py", line 251 in run_server
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Current thread 0x000015cc (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\ctypes\__init__.py", line 374 in __init__
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\magic\loader.py", line 44 in load_lib
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\magic\__init__.py", line 209 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\unstructured\file_utils\filetype.py", line 25 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\unstructured\partition\pdf.py", line 57 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\pdf.py", line 57 in _get_elements
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\unstructured.py", line 87 in load
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3212 in file_to_doc
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3464 in path_to_doc1
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3761 in <listcomp>
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3761 in path_to_docs
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 7369 in _update_user_db
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 7169 in update_user_db
  File "C:\Users\Administrator\h2ogpt\src\gradio_runner.py", line 6356 in update_user_db_gr
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\gradio\utils.py", line 661 in wrapper
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\anyio\_backends\_asyncio.py", line 807 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00001bac (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 797 in _poll
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 444 in select
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 1871 in _run_once
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 603 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 321 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 636 in run_until_complete
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\runners.py", line 44 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\uvicorn\server.py", line 62 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x000018d0 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 324 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 607 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\apscheduler\schedulers\blocking.py", line 30 in _main_loop
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00002208 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 324 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 607 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\tqdm\_monitor.py", line 60 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x000004e0 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 324 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\queue.py", line 180 in get
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\posthog\consumer.py", line 104 in next
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\posthog\consumer.py", line 73 in upload
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\posthog\consumer.py", line 62 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00001fc4 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\gradio\blocks.py", line 2361 in block_thread
  File "C:\Users\Administrator\h2ogpt\src\gradio_runner.py", line 6021 in go_gradio
  File "C:\Users\Administrator\h2ogpt\src\gen.py", line 2039 in main
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 475 in _Fire
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 141 in Fire
  File "C:\Users\Administrator\h2ogpt\src\utils.py", line 65 in H2O_Fire
  File "C:\Users\Administrator\h2ogpt\generate.py", line 12 in entrypoint_main
  File "C:\Users\Administrator\h2ogpt\generate.py", line 16 in <module>
Windows fatal exception: access violation

Thread 0x000002cc (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\concurrent\futures\thread.py", line 81 in _worker
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00001d34 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\concurrent\futures\thread.py", line 81 in _worker
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00000764 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 797 in _poll
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 444 in select
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 1871 in _run_once
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 603 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 321 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 636 in run_until_complete
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\runners.py", line 44 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\uvicorn\server.py", line 62 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\uvicorn\main.py", line 587 in run
  File "C:\Users\Administrator\h2ogpt\openai_server\server.py", line 251 in run_server
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Current thread 0x000015cc (most recent call first):
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\pdf.py", line 57 in _get_elements
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\unstructured.py", line 87 in load
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3212 in file_to_doc
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3464 in path_to_doc1
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3761 in <listcomp>
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3761 in path_to_docs
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 7369 in _update_user_db
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 7169 in update_user_db
  File "C:\Users\Administrator\h2ogpt\src\gradio_runner.py", line 6356 in update_user_db_gr
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\gradio\utils.py", line 661 in wrapper
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\anyio\_backends\_asyncio.py", line 807 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00001bac (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 797 in _poll
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 444 in select
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 1871 in _run_once
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 603 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\windows_events.py", line 321 in run_forever
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\base_events.py", line 636 in run_until_complete
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\asyncio\runners.py", line 44 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\uvicorn\server.py", line 62 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x000018d0 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 324 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 607 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\apscheduler\schedulers\blocking.py", line 30 in _main_loop
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 953 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00002208 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 324 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 607 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\tqdm\_monitor.py", line 60 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x000004e0 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 324 in wait
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\queue.py", line 180 in get
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\posthog\consumer.py", line 104 in next
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\posthog\consumer.py", line 73 in upload
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\posthog\consumer.py", line 62 in run
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 1016 in _bootstrap_inner
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\threading.py", line 973 in _bootstrap

Thread 0x00001fc4 (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\gradio\blocks.py", line 2361 in block_thread
  File "C:\Users\Administrator\h2ogpt\src\gradio_runner.py", line 6021 in go_gradio
  File "C:\Users\Administrator\h2ogpt\src\gen.py", line 2039 in main
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 475 in _Fire
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\fire\core.py", line 141 in Fire
  File "C:\Users\Administrator\h2ogpt\src\utils.py", line 65 in H2O_Fire
  File "C:\Users\Administrator\h2ogpt\generate.py", line 12 in entrypoint_main
  File "C:\Users\Administrator\h2ogpt\generate.py", line 16 in <module>
pseudotensor commented 9 months ago

Looks like unstructured barfs:

Current thread 0x000015cc (most recent call first):
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\pdf.py", line 57 in _get_elements
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\unstructured.py", line 87 in load

Can you pip install an older version of unstructured or see if any other changes help?

I also see:

Current thread 0x000015cc (most recent call first):
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\ctypes\__init__.py", line 374 in __init__
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\magic\loader.py", line 44 in load_lib
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\magic\__init__.py", line 209 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\unstructured\file_utils\filetype.py", line 25 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\unstructured\partition\pdf.py", line 57 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\pdf.py", line 57 in _get_elements
  File "C:\ProgramData\miniconda3\envs\h2ogpt\lib\site-packages\langchain_community\document_loaders\unstructured.py", line 87 in load
  File "C:\Users\Administrator\h2ogpt\src\gpt_langchain.py", line 3212 in file_to_doc

Maybe there is crash due to multiple threads trying to do some imports or access some libraries. Known python bugs. Maybe can move imports earlier to avoid such races.

pseudotensor commented 9 months ago

E.g. you can add these to top of gpt_langchain.py

import magic
from unstructured.partition.pdf import partition_pdf

Let me know if that helps, and I can move some imports outside local scopes.

isaac-aburto commented 9 months ago

I tried with an older version of unstructured, but doesn't work. Same when trying to change some imports in the code.

Thanks to your response I decided to review certain libraries, especially the magic.

The error Windows fatal exception: access violation, apparently happen in the file: C:\ProgramData\miniconda3\envs\h2ogpt\Lib\site-packages\magic\loader.py 

def _lib_candidates():

  yield find_library('magic')
  #print("sys.platform: ", sys.platform)

  if sys.platform == "darwin":

    paths = [
      '/opt/local/lib',
      '/usr/local/lib',
      '/opt/homebrew/lib',
    ] + glob.glob('/usr/local/Cellar/libmagic/*/lib')

    for i in paths:
      yield os.path.join(i, 'libmagic.dylib')

  elif sys.platform in ("win32", "cygwin"):

    #prefixes = ['msys-magic-1', 'libmagic', 'magic1', 'cygmagic-1', 'libmagic-1']
    prefixes = ['libmagic']

    for i in prefixes:
      # find_library searches in %PATH% but not the current directory,
      # so look for both
      yield './%s.dll' % (i,)
      yield find_library(i)

The code was trying to get into these dll files, but they did not exist in the folder. What I did was move the file located at: C:\ProgramData\miniconda3\envs\h2ogpt\Lib\site-packages\magic\libmagic\libmagic.dll to C:\ProgramData\miniconda3\envs\h2ogpt\Library\usr\bin and commented the list of files that could not be found.

I don't know if it's the best solution, but it's the only one that has helped me.

pseudotensor commented 9 months ago

Interesting, thanks. I'll see if I can understand.