McCloudS / subgen

Autogenerate subtitles using OpenAI Whisper Model via Jellyfin, Plex, Emby, Tautulli, or Bazarr
MIT License
570 stars 49 forks source link

Jellyfin HTTP/1.1" 500 Internal Server Error - ERROR: Exception in ASGI application #87

Closed shazwozza closed 4 months ago

shazwozza commented 5 months ago

Using Jellyfin portable on Windows 11.

Installed Webhooks - followed instructions. Made sure all correct boxes ticked, Request header: Key: Content-Type
Value: application/json. Webhook url is set to http://_subgenip_:9000/jellyfin

Installed subgen. I load http://_subgenip_:9000 to configure settings. JELLYFINSERVER is set to http://_jellyfinip_:8096 Now i wasn't sure if i should have USE_PATH_MAPPING: True or false. But i played with both settings, as well as setting PATH_MAPPING_FROM: and PATH_MAPPING_TO: to the media folder directory that I added on jellyfin.

I added the new media directory to Jellyfin and played some videos. In the subgen terminal i can see it triggers the webhook but get the following error (see below). I'm not sure why a connection is trying to be made to 192.168.1.111 ? That is neither the subgen or jellyfin ip address, nor any device on my network.

2024-05-04 22:36:09,502 DEBUG: itemid is: 86ba3fd6dec09361b583628baaa4758a
INFO:     192.168.1.3:12397 - "POST /jellyfin HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
    raise err
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
    conn.request(
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 400, in request
    self.endheaders()
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\http\client.py", line 1331, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\http\client.py", line 1091, in _send_output
    self.send(msg)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\http\client.py", line 1035, in send
    self.connect()
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 207, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x000002DE519C8D10>, 'Connection to 192.168.1.111 timed out. (connect timeout=None)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='192.168.1.111', port=8096): Max retries exceeded with url: /Users (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x000002DE519C8D10>, 'Connection to 192.168.1.111 timed out. (connect timeout=None)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\middleware\errors.py", line 186, in __call__
    raise exc
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\middleware\errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\middleware\exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 72, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\fastapi\routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\fastapi\routing.py", line 193, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\concurrency.py", line 42, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\anyio\_backends\_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\anyio\_backends\_asyncio.py", line 851, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\Downloads\jellyfin_portable\subgen-main\subgen.py", line 363, in receive_jellyfin_webhook
    fullpath = get_jellyfin_file_name(ItemId, jellyfinserver, jellyfintoken)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\Downloads\jellyfin_portable\subgen-main\subgen.py", line 715, in get_jellyfin_file_name
    users = json.loads(requests.get(f"{jellyfin_url}/Users", headers=headers).content)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py", line 507, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='192.168.1.111', port=8096): Max retries exceeded with url: /Users (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x000002DE519C8D10>, 'Connection to 192.168.1.111 timed out. (connect timeout=None)'))
McCloudS commented 5 months ago

Sorry, somehow gmail marked the issue notification as spam so I didn't see this. It sounds like you don't need path mapping as long as your subgen instance and Jellyfin are on the same machine.

http://192.168.1.111:8096 is the default setting in the script, so it's using that if it doesn't detect the environment variable.

The environment variables (that are saved from the webui @ http://subgenip:9000) may not be getting saved correctly (or in the wrong path somehow). Do you have a file named subgen.env inside your subgen directory?

You should be seeing something similar to: File subgen.env not found. Using default values. or File subgen.env not found. Environment variables not set. in your terminal if above was the case.

McCloudS commented 5 months ago

Are you clicking save at the bottom?

shazwozza commented 4 months ago

Yes both subgen and jellyfin are on the same machine. USE_PATH_MAPPING: False Yes subgenn.env is inside the subgen directory I made sure i clicked save at bottom.

EDIT: It looks like it's processing but still getting errors. I restarted pc. Started subgen via command using "python launcher.py -u -i -s" On subgen:9000 settings, i had to reenter the jellyfinip:port I started jellyfin portable. Now i created an API token from jellyfin. I then added that key into the subgen:9000 settings. (I think this is how i'm at least getting some processing logs. But still getting errors at the end). I played a video and now the subgen terminal is showing some activity as follows. It seems like it's processing but i get some error in the end.


2024-05-13 21:01:16,430 DEBUG: Jellyfin event detected is: PlaybackStart
2024-05-13 21:01:16,430 DEBUG: itemid is: 84df5256a7400a580288d6f33102d8ed
2024-05-13 21:01:16,513 DEBUG: Path of file: D:\Files\Video.mp4
2024-05-13 21:01:16,605 DEBUG: No subtitles in 'eng' language found in the video.
2024-05-13 21:01:16,606 INFO: Added Video.mp4 for transcription.
2024-05-13 21:01:16,606 INFO: Transcribing file: Video.mp4
2024-05-13 21:01:16,607 DEBUG: Model was purged, need to re-create
2024-05-13 21:01:16,621 INFO: Metadata refresh queued successfully.
2024-05-13 21:01:16,622 INFO: Metadata for item 84df5256a7400a580288d6f33102d8ed refreshed successfully.
INFO:     192.168.1.3:12380 - "POST /jellyfin HTTP/1.1" 200 OK
config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.26k/2.26k [00:00<?, ?B/s]
vocabulary.txt: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 460k/460k [00:01<00:00, 249kB/s]
tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.20M/2.20M [00:05<00:00, 415kB/s]
model.bin: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.53G/1.53G [22:40<00:00, 1.12MB/s]
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.███████████████████████████████████████████████████| 1.53G/1.53G [22:40<00:00, 1.19MB/s]
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
Traceback (most recent call last):
  File "C:\Users\shazwozza\Downloads\jellyfin_portable\subgen-main\launcher.py", line 168, in <module>
    main()
  File "C:\Users\shazwozza\Downloads\jellyfin_portable\subgen-main\launcher.py", line 163, in main
    subprocess.run([f'{python_cmd}', '-u', 'subgen.py'], check=True)
  File "C:\Users\shazwozza\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', '-u', 'subgen.py']' returned non-zero exit status 3.
McCloudS commented 4 months ago

Awesome, getting way further. The libiomp5md.dll error isn't a subgen problem, but is essentially highlighting you have two versions of libiomp5md.dll in your python path somewhere.

See: https://stackoverflow.com/questions/64209238/error-15-initializing-libiomp5md-dll-but-found-libiomp5md-dll-already-initial or https://community.anaconda.cloud/t/omp-error-15-initializing-libiomp5md-dll-but-found-libiomp5-already-initialized/48993

or a simple work around for now: in your command prompt before running subgen, type set KMP_DUPLICATE_LIB_OK=TRUE

Hopefully if you have that resolved, we can get further and see where the problem is.

McCloudS commented 4 months ago

I added os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" as a default to the script, so if you run python launcher.py -u it should get you past the error.

shazwozza commented 4 months ago

I added os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE" as a default to the script, so if you run python launcher.py -u it should get you past the error.

Thanks for your help! That worked a treat!

McCloudS commented 4 months ago

Great. Are you seeing any output yet (subs in Jellyfin)?

shazwozza commented 4 months ago

Great. Are you seeing any output yet (subs in Jellyfin)?

Yes it outputs an .srt file whenever I play a video.

I remembered reading on your reddit post that it was possible scan and transcribe any media file it sees.

"It depends on your configuration. If you want to transcribe as media is added, configure the media server webhooks."

What would I have to configure in webhooks plugin so that any new library i create, it will start generating subtitles, without having to play each video individually?

McCloudS commented 4 months ago

Use the batch endpoint at http://subgen:9000/docs or look at the watch folder variable in the readme.