Transcription Generation Does Not Work

MattyBoombalatty commented 1 month ago

Describe the current behavior

Using whisper-ctranslate2, it appears that videos do not get transcribed. Please note that this is a GPU accelerated server and the dependencies have been referenced correctly.

Error

{
  "payload": {
    "videoUUID": "750bd3d2-1f13-4a05-bd80-ef63b4917acb"
  },
  "err": {
    "stack": "Error: ENOENT: no such file or directory, open '/mnt/md0/peertube/tmp/transcription/1YvFhDvoyAZhnu99UQowEB/19a4c40d-37a3-499c-af72-32af862ac185-0-fragmented.json'",
    "message": "ENOENT: no such file or directory, open '/mnt/md0/peertube/tmp/transcription/1YvFhDvoyAZhnu99UQowEB/19a4c40d-37a3-499c-af72-32af862ac185-0-fragmented.json'",
    "errno": -2,
    "code": "ENOENT",
    "syscall": "open",
    "path": "/mnt/md0/peertube/tmp/transcription/1YvFhDvoyAZhnu99UQowEB/19a4c40d-37a3-499c-af72-32af862ac185-0-fragmented.json"
  }
}

production.yaml

video_transcription:
  # Enable automatic transcription of videos
  enabled: true

  # Choose engine for local transcription
  # Supported: 'openai-whisper' or 'whisper-ctranslate2'
  engine: 'whisper-ctranslate2'

  # You can set a custom engine path for local transcription
  # If not provided, PeerTube will try to automatically install it in the PeerTube bin directory
  engine_path: null #'/var/www/peertube/.local/bin/whisper-ctranslate2'

  # Choose engine model for local transcription
  # Available for 'openai-whisper' and 'whisper-ctranslate2': 'tiny', 'base', 'small', 'medium', 'large-v2' or 'large-v3'
  model: 'small'

  # Or specify the model path:
  #  * PyTorch model file path for 'openai-whisper'
  #  * CTranslate2 Whisper model directory path for 'whisper-ctranslate2'
  # If not provided, PeerTube will automatically download the model
  model_path: '/var/www/peertube/.local/lib/python3.10/site-packages/ctranslate2.libs' #/libcudnn-463fd6d5.so.8.9.7'

  # Enable remote runners to transcribe videos
  # If enabled, your instance won't transcribe the videos itself
  # At least 1 remote runner must be configured to transcribe your videos
  remote_runners:
    enabled: false

Steps to reproduce

1. 2. 3.

Describe the expected behavior

No response

Additional information

PeerTube instance:
- URL: https://nicecrew.tv
- Version: 6.3.0
- NodeJS version:
- Ffmpeg version:
Browser name, version and platforms on which you could reproduce the bug:
Link to browser console log if relevant:
Link to server log if relevant (journalctl or /var/www/peertube/storage/logs/):

Chocobozzz commented 1 month ago

Hi,

Please enable server debug logs, retry the transcription and paste them so we have more context :)

MattyBoombalatty commented 1 month ago

The first issue was that a model.bin was not found in /var/www/peertube/.local/lib/python3.10/site-packages/ctranslate2.libs, so I copied the model libcudnn-463fd6d5.so.8.9.7 file and renamed it to that, but then I got this error.

ExecaError: Command failed with exit code 1: /mnt/md0/peertube/bin/pip/bin/whisper-ctranslate2 /mnt/md0/peertube/web-videos/private/f160754b-4019-4751-bcf2-edebe56a3e56-1080.mp4 --model_directory /var/www/peertube/.local/lib/python3.10/site-packages/ctranslate2.libs --word_timestamps True --vad_filter true --vad_min_silence_duration_ms 5000 --output_format all --output_dir /mnt/md0/peertube/tmp/transcription/7nMJ1EdEitMkuosGKb1vyC

Traceback (most recent call last):
  File "/mnt/md0/peertube/bin/pip/bin/whisper-ctranslate2", line 8, in <module>
    sys.exit(main())
  File "/mnt/md0/peertube/bin/pip/src/whisper_ctranslate2/whisper_ctranslate2.py", line 199, in main
    transcribe = Transcribe(
  File "/mnt/md0/peertube/bin/pip/src/whisper_ctranslate2/transcribe.py", line 110, in __init__
    self.model = WhisperModel(
  File "/mnt/md0/peertube/bin/pip/faster_whisper/transcribe.py", line 145, in __init__
    self.model = ctranslate2.models.Whisper(
RuntimeError: Unsupported model binary version. This executable supports models with binary version v6 or below, but the model has binary version v1179403647. This usually means that the model was generated by a later version of CTranslate2. (Forward compatibility is not guaranteed.)

Detecting language using up to the first 30 seconds. Use `--language` to specify the language
    at getFinalError (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/return/final-error.js:6:9)
    at makeError (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/return/result.js:108:16)
    at getAsyncResult (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/methods/main-async.js:167:4)
    at handlePromise (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/methods/main-async.js:150:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Ctranslate2Transcriber.transcribe (file:///var/www/peertube/versions/peertube-v6.3.0/packages/transcription/dist/whisper/transcriber/ctranslate2-transcriber.js:18:9)
    at async file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/video-captions.js:83:36
    at async VideoPathManager.makeAvailableFactory (file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/video-path-manager.js:166:22)
    at async generateSubtitle (file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/video-captions.js:80:9)
    at async Object.wrapPromiseFun (file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/plugins/hooks.js:8:24)
    at async Worker.processJob (/var/www/peertube/versions/peertube-v6.3.0/node_modules/bullmq/dist/cjs/classes/worker.js:453:28)
    at async Worker.retryIfFailed (/var/www/peertube/versions/peertube-v6.3.0/node_modules/bullmq/dist/cjs/classes/worker.js:638:24)

Chocobozzz commented 1 month ago

Hi,

Please try to run whisper manually with your custom model first. Closing this issue since it doesn't seem to be a PeerTube issue (you can still use the forum to get community support).

MattyBoombalatty commented 1 month ago

Without any custom paths set in production.yaml, this is the error.

ExecaError: Command was killed with SIGABRT (Aborted): /mnt/md0/peertube/bin/pip/bin/whisper-ctranslate2 /mnt/md0/peertube/web-videos/private/f160754b-4019-4751-bcf2-edebe56a3e56-1080.mp4 --model small --word_timestamps True --vad_filter true --vad_min_silence_duration_ms 5000 --output_format all --output_dir /mnt/md0/peertube/tmp/transcription/wWV9buCYwRRrUFo67UbpsG

Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory
    at getFinalError (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/return/final-error.js:6:9)
    at makeError (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/return/result.js:108:16)
    at getAsyncResult (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/methods/main-async.js:167:4)
    at handlePromise (file:///var/www/peertube/versions/peertube-v6.3.0/node_modules/execa/lib/methods/main-async.js:150:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Ctranslate2Transcriber.transcribe (file:///var/www/peertube/versions/peertube-v6.3.0/packages/transcription/dist/whisper/transcriber/ctranslate2-transcriber.js:18:9)
    at async file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/video-captions.js:83:36
    at async VideoPathManager.makeAvailableFactory (file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/video-path-manager.js:166:22)
    at async generateSubtitle (file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/video-captions.js:80:9)
    at async Object.wrapPromiseFun (file:///var/www/peertube/versions/peertube-v6.3.0/dist/core/lib/plugins/hooks.js:8:24)
    at async Worker.processJob (/var/www/peertube/versions/peertube-v6.3.0/node_modules/bullmq/dist/cjs/classes/worker.js:453:28)
    at async Worker.retryIfFailed (/var/www/peertube/versions/peertube-v6.3.0/node_modules/bullmq/dist/cjs/classes/worker.js:638:24)

Chocobozzz commented 1 month ago

Can you try https://github.com/SYSTRAN/faster-whisper/issues/516#issuecomment-2201088567?

MattyBoombalatty commented 1 month ago

This works.

https://github.com/SYSTRAN/faster-whisper/issues/516#issuecomment-2353538157

Chocobozzz / PeerTube