Transcoding failed: Missing lock for job

[kontrollanten]

Describe the current behavior

{
    "err": "Error: EIO: i/o error, read",
    "level": "error",
    "message": "Error in controller.",
    "timestamp": "2024-05-08T18:12:09.366Z"
}
{
    "level": "error",
    "message": "Error in job worker video-transcoding.",
    "err": {
        "stack": "Error: Missing lock for job de73a010-53c6-4142-91fa-0b17894c0f59. failed\n    at Scripts.finishedErrors (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/scripts.js:275:24)\n    at Job.moveToFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/job.js:439:32)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async handleFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:380:21)\n    at async Worker.retryIfFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)",
        "message": "Missing lock for job de73a010-53c6-4142-91fa-0b17894c0f59. failed"
    },
    "timestamp": "2024-05-08T18:12:45.763Z"
}
{
    "level": "error",
    "message": "Cannot execute job de73a010-53c6-4142-91fa-0b17894c0f59 in queue video-transcoding.",
    "payload": {
        "type": "new-resolution-to-hls",
        "videoUUID": "02f4fe9f-763e-43bb-b087-ae7142253ed8",
        "resolution": 1080,
        "fps": 25,
        "copyCodecs": true,
        "isNewVideo": true,
        "deleteWebVideoFiles": true
    },
    "err": {
        "stack": "TypeError: Cannot read properties of undefined (reading 'getVideoOrStreamingPlaylist')\n    at handleHLSJob (file:///var/www/peertube/versions/peertube-v6.1.0-c1/dist/core/lib/job-queue/handlers/video-transcoding.js:71:57)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Object.processVideoTranscoding [as video-transcoding] (file:///var/www/peertube/versions/peertube-v6.1.0-c1/dist/core/lib/job-queue/handlers/video-transcoding.js:34:9)\n    at async Object.wrapPromiseFun (file:///var/www/peertube/versions/peertube-v6.1.0-c1/dist/core/lib/plugins/hooks.js:8:24)\n    at async Worker.processJob (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:395:28)\n    at async Worker.retryIfFailed (/var/www/peertube/versions/peertube-v6.1.0-c1/node_modules/bullmq/dist/cjs/classes/worker.js:584:24)",
        "message": "Cannot read properties of undefined (reading 'getVideoOrStreamingPlaylist')"
    },
    "timestamp": "2024-05-08T18:12:45.839Z"
}

I don't think the EIO error is related since we see a lot of them in the logs, but since it was close in time it may be related.

Steps to reproduce

1. 2. 3.

Describe the expected behavior

No response

Additional information

• PeerTube instance:

  • URL:
  • Version: 6.1.0
  • NodeJS version: 18.20.2
  • Ffmpeg version: 4.4.2
  • Redis version: 7.2.3

• Browser name, version and platforms on which you could reproduce the bug:

• Link to browser console log if relevant:

• Link to server log if relevant (journalctl or /var/www/peertube/storage/logs/):

[Chocobozzz]

Hi,

I think the Cannot read properties of undefined error is related to the broken job Missing lock for job.

Did you have an issue with Redis? (EIO: i/o error, read is also a clue)

[kontrollanten]

Not any known issues with redis. When I check the redis logs there's no errors and the container has been up for months.

[Chocobozzz]

Do you have many Missing lock for job errors in your logs?

[kontrollanten]

No, we had 10 occurrences during a few days. During the same days we had issues with connection to our S3 provider, but that shouldn't affect transcoding jobs?

[kontrollanten]

We haven't seen this in a while. I'll open a new issue if it happens again.