search

mifi / lossless-cut

The swiss army knife of lossless video/audio editing
https://losslesscut.app/
GNU General Public License v2.0
26k stars 1.25k forks source link

Previewing panoramic (VR) videos #1667

Open blurymind opened 1 year ago

blurymind commented 1 year ago

I have a lot of issues to go through, so in order to make it easier for me to help you, I ask that you please try these things first

Description

Typical case for cutting big videos without losing quality are VR panoramic videos. Lots of drones these days support recording this type of video and it is useful to be able to take out highlights from a flight for example. Lossless cut is fantastic at this compared to any other tool out there, but it has a small flaw - which is that the video player only supports standard type video projection.

One potentially very nice improvement of life feature would be to add support (or a button to toggle a video viewing mode for 360/180/standard type projection videos - to normalize the picture from being warped.

I am noticing that the way the player is implemented (and excuse me if thats not the case) is just by rapidly passing images to a canvas element's ctx. https://github.com/mifi/lossless-cut/blob/master/src/CanvasPlayer.js#L18

My guess is that this project would like to keep away from any new dependencies that would make the request of this ticket easily possible (such as for example adding a dependency on threejs).

So if there is interest in implementing this - would it need to be done with vanilla javascript only - where the canvas video player is implemented? If it's ok to add some new library for panoramic video projection - are there any favorites that would fit well with the project? (apart of the one i mentioned - threejs, babylon3d comes to mind, aframe can do this too i think)

The video playback area will need to take input events from the mouse - so click-dragging would change the view too

mifi commented 1 year ago

Hi! Are you saying that losslesscut is actually able to cut panoramic videos and preserve the projection metadata in the output files, but the only problem is that you cannot preview videos with the projection in losslesscut? I believe there are some issues where, when cutting+concatenating, this projection metadata will be lost: https://github.com/mifi/lossless-cut/issues/169#issuecomment-1032861180

LosslessCut has 2 ways of playing back videos:

  1. if it's format/codec combination supported by html5 video (chromium), the video will be played back using a simple <video> tag.
  2. If the file is not supported by <video>, you're right, we create a canvas element that will receive a stream of frames from ffmpeg.

So I think it's possible to implement something like this, but maybe it should be implemented in a way that works for both playback modes 1. and 2. or maybe we say that for pano previews, we only support the canvas-playback method.

blurymind commented 1 year ago

I was impressed by both the performance of playback and export when dealing with large videos. Lossless cut exports in seconds for files over 5-10+ gb. Playback didn't lag for me either - where some native video players and a standard NLE struggles. If you compare that to doing this with a standard video editor - exporting a segment from a big file takes much longer and you still may lose the metadata + your output is re-encoded and with slightly degraded quality.

So this really makes it very attractive to me and would be interested in adding something there to do panoramic projection specifically for previewing. The loss of metadata does not affect me so much, because you can still play the video even without it- and tell the player how to best do it.

Not being able to know how your cuts would look like seems more important to me, since that informs how I do the cuts.

Would adding a new dependency to the project on threejs be too much for this? It seems to be the most common way its done and is well documented https://threejs.org/examples/webgl_video_panorama_equirectangular.html

Would you be interested in me doing a pr or this is already been on your mind and have something specific in mind? :)

mifi commented 1 year ago

I think adding three.js should be fine. I'm open for a PR as long as it doesn't change the current playback code too much. Maybe before spending too much time, you could roughly describe how you would implement it using three.js. I think the Canvas player is a bit slow, so if possible, maybe somehow hooking into the html5 <video> player would be a better fit?