search

Chocobozzz / PeerTube

ActivityPub-federated video streaming platform using P2P directly in your web browser
https://joinpeertube.org/
GNU Affero General Public License v3.0
12.74k stars 1.46k forks source link

Create video transcoding journal #4320

Open ghost opened 2 years ago

ghost commented 2 years ago

Describe the current behavior

Infrequently but not rarely, I get reports of videos that don't survive transcoding. Often there's a truncation at one or more resolutions, or the m3u8 playlist is incorrect.

I can't provide a good way to reproduce this yet, but it's worrying that such a serious problem can occur (end-user data loss!) without having any record of the original as-uploaded video file, or any kind of process that would make it possible to track changes to peertube's storage in a consistent way through the process of uploading, identifying, transcoding, and archiving media.

ghost commented 2 years ago

I was looking through my filesystem snapshots to see if I could recover the video this time, and I did indeed find a complete-looking copy of the video in an earlier snapshot. It looks like the order of operation is something like

  1. HLS Video and torrent at 1080p are created
  2. several hours elapse
  3. 1080p video re-transcoded and truncated
  4. Other resolutions (360p) immediately start transcoding
  5. ...
Chocobozzz commented 2 years ago

What is the status of the transcoding jobs of these broken videos? Some of them failed? Do you have multiple jobs for the same video/resolution? How long took these jobs?

ghost commented 2 years ago

I don't think I could locate the specific job info after all this time. My best assessment at the time was that transcode jobs were succeeding and running normally but there might have been multiple transcodes running simultaneously with the same output file.

I don't have detailed info though, and part of my purpose in opening this issue is to say we need better tools to both understand and avoid issues like this. It shouldn't be possible for me to have unrecoverable corruption on a video, regardless of what else fails. I should have an audit trail showing the transcode steps taken for each videofile on the server, regardless of what may or may not still be in the main json logs. And all transcodes should have info both for debugging these failures and recovering from unforseen problems.

kontrollanten commented 2 years ago

@scanlime I'm thinking about making a PR for this. My initial thought was just add video id filter on the Admin > Jobs page, but that's not too user friendly though. Then I thought about add some section on the video watch page (just for admins), but I'm not sure adding admin info to the watch page is a good solution. Maybe this should be in a separate tab on the video update page?

Chocobozzz commented 2 years ago

Hi,

I think you can create a dedicated REST API endpoint to display video transcoding journal, and then display it in a modal. This modal could be shown on item click from the video actions dropdown for example (dropdown that is displayed in the watch page and admin videos overview page).