Native support for codecs/formats (TS,H265,mpeg2++)

mifi commented 6 years ago

Due to losslesscut using chromium's html5 video tag which only supports certain container formats and codecs, we currently need to "convert to supported format" when trying to open exotic formats or codecs. This could be improved with the help of ffmpeg.

Workarounds

Compile custom electron

https://github.com/electron/build-tools

Convert to supported format

Currently it's possible to cut those files by first using the File -> Convert to supported format (slow) function, however this is a re-encoding (although fast) of the whole file, just to be able to view it in the HTML5 video player (note that the original and cut versions will not be re-encoded and are still lossless.)

Improve current proxy/dummy code

Currently a <canvas> element is used, and ffmpeg will stream low-resolution jpeg images to that canvas.

Drawbacks:

Slow, laggy, low resolution, low fps
No audio

This could be improved:

Electron (Chromium) might never support h265

~~It seems like Chrome and most other browsers are not going to be supporting H265 as it stands now, and since we are using Chromium (Electron), this will be a problem. It might be possible to enable a flag to enable H265 decoding, but it requires some modding of electron.~~

Update: Chromium now supports h265 (hardware decoding only)

killaz47 commented 5 years ago

heres the solution from the chrnoium developers https://github.com/henrypp/chromium/blob/master/hevc_support.md https://github.com/henrypp/chromium/blob/master/hevc_support_new.md

killaz47 commented 5 years ago

Now that more and more cameras (including DJI Mavic 2) are starting to support H265, it would be nice to have support for this here as well. Currently it's possible to cut those files by first using the File -> Convert to friendly format (slow) function, however this is a re-encoding (although fast) of the whole file, just to be able to view it in the HTML5 video player (note that the original and cut versions will not be re-encoded and are still lossless.)

It seems like Chrome and other browsers are not going to be supporting H265 as it stands now, and since we are using Chromium (Electron), this will be a problem. It might be possible to enable a flag to enable H265 decoding, but it requires some modding of electron.

If anyone has some bright ideas, shout out!

https://stackoverflow.com/questions/36382568/h-265-hevc-web-browser-support

webtorrent/webtorrent-desktop#561

electron/libchromiumcontent#198

https://github.com/strukturag/libde265.js

heres the solution from the chrnoium developers https://github.com/henrypp/chromium/blob/master/hevc_support.md https://github.com/henrypp/chromium/blob/master/hevc_support_new.md

mifi commented 5 years ago

I'm not sure how to do this with electron. (LosslessCut uses Electron which uses Chromium.) Feel free to open an issue at electron to see if this can be integrated. I think it is not a trivial task, so I don't have time to look at this now.

TechManiacHD commented 5 years ago

Lately I swap my gear and see, that recording in HEVC format gives me better quality over H264 even with lower bitrate. I would love to switch to HEVC for all my projects although only Lossless Cut stops me from doing that - https://i.imgur.com/avu5AQE.png (after converting to friendly format).

Is there a chance this will work at some point or should I still sit on old format? Thanks.

esamattis commented 5 years ago

I'm seeing that too with GoPro 7 HEVC videos and fast convert. The slow one works.

TechManiacHD commented 5 years ago

You're right! It works with slow one (That quality xD). Enough for cutting no worries :) Thanks, that will do for time being.

mifi commented 5 years ago

Great! the reason for the low quality is to make it as fast as remotely possible, and only to be used as a preview.

antonioriva commented 5 years ago

As LossLess-cut is build in electron do you ever considered to use mpv as player component? I've just found that a project to do that it's already there... https://github.com/Kagami/mpv.js/ with also some samples implementation

mifi commented 5 years ago

Thanks for the tip! I didn't know about mpv.js, but if it's easy to integrate then that's definitely an option. https://github.com/Kagami/mpv.js/issues/27

mifi commented 5 years ago

In 1.5.0 I have implemented a hack that lets us cut H265 videos, but scrubbing and playing only works at a very low fps and currently without audio.

ThexXTURBOXx commented 4 years ago

grafik Getting this error when trying to merge 2 H.265 videos together. Is there something I can do about this?

peterhighgrove commented 4 years ago

Is it possible to add support for grabbing a frame at 4k/hevc to jpg even though the preview is not native?

mifi commented 4 years ago

Is it possible to add support for grabbing a frame at 4k/hevc to jpg even though the preview is not native?

I have implemented this now

peterhighgrove commented 4 years ago

Nice, thanks! Can't wait to test it in the next release!

mifi commented 4 years ago

Check out newest version now

peterhighgrove commented 4 years ago

Thanks, tested 3.17.9. It's working great:)

ThexXTURBOXx commented 4 years ago

@mifi Is there any news on supporting H265 natively?

mifi commented 4 years ago

No. I think it needs a lot of work because have to compile a custom electron with a custom chromium that uses a custom ffmpeg. Not something i will prioritize unless this work gets funded somehow, or if we can find the work of someone else who did this already.

desertpogona commented 3 years ago

This is still a problem in the current build and I was wondering if progress is being made on it as video standards move forward

mifi commented 3 years ago

AFAIK chromium/electron still doesn't support H265 and they refuse to support it due to expensive licensing: https://bugs.chromium.org/p/chromium/issues/detail?id=684382 So we still need to build a custom electron with a custom chromium to get this native support

gmcmicken commented 2 years ago

I came across this guide "it’s not as difficult as it might sound" Electron with custom Chromium build

and then the instructions for HEVC support in chromium

Also noticed this gem from the chromium forums that is currently being implemented Issue 1286132: Implement HEVC for video playback and video conferencing on Windows

Linnshet commented 2 years ago

Any updates on native HEVC implementation? The solution of making a dummy proxy is pretty brilliant, but it would be nice to see my videos in full res.

mifi commented 2 years ago

@gmcmicken nice finds. I've not taken on the task of building a custom chromium. However I do have an idea with the https://developer.mozilla.org/en-US/docs/Web/API/MediaSource to hopefully allow for more smooth streaming from ffmpeg to the player. It may also allow playing audio. Note that it still needs to encode the video while playing but I think h264 encoding can be done very fast these days. Could also set the duration which would remove the need for the html5ified dummy file.

romadd64 commented 2 years ago

There seems to be some good news: Sida Zhu has implemented support for hw decoding for HEVC in chromium v104.0.5084.0, disabled by default and enabled with the PlatformHEVCDecoderSupport feature. (https://github.com/StaZhu/enable-chromium-hevc-hardware-decoding) . chromium 104 needs electron nighty build 21 (https://github.com/electron/releases) . Mikael, could you build losslesscut updated to this electron version and with feature enabled? Thanks a lot!

mifi commented 2 years ago

that's fantastic! Then I think once electron 20 becomes stable we can upgrade to that. Note that upgrading losslesscut to newest electron is quite a big job. But I plan on doing it soon, now that electron>v8 slowness on macos has been fixed #714

romadd64 commented 2 years ago

You are right, also latest electron 20 beta has chromium 104, with the 8-weeks release cycle we should hopefully have a stable 20 soon. I understand that going from electon 8 to 20 could be a big jump, I'll donate for your effort. Thanks.

romadd64 commented 2 years ago

Hi Mikael, do you have some progress on this improvement?

mifi commented 2 years ago

Hi. Unfortunately not, as ive been very busy with other projects. But, assuming electron now supports natively, it will be solved once Llc upgrades electron, which is one of the first next big tasks

dellhem commented 2 years ago

Hi. Unfortunately not, as ive been very busy with other projects. But, assuming electron now supports natively, it will be solved once Llc upgrades electron, which is one of the first next big tasks

I will definitely buy the app store version to subsidise a couple of cups of coffee for your effort ;)

mifi commented 1 year ago

I'm testing with Electron v21 now and with app.commandLine.appendSwitch('enable-features','PlatformHEVCDecoderSupport') h265 seems to be working! so expect an alpha release of losslesscut soon

loogaru commented 1 year ago

OMG, It works wonderfully! I'm gonna cry now)) Quick, precise, responsive, original quality, no preview conversion... this makes my workflow much comfortable (I'm shooting long scripted 4k videos, and use LosslessCut to markdown them, cut good takes, and it really helps me save much storage space). Thank you, thank you, thank you, @mifi!

Spaarpott commented 1 year ago

Yep. In v3.48.2 HEVC works like a charm!

Saltiola7 commented 1 year ago

HEVC cutting

Works like a charm on my Fujifilm X-T4 10 bit HEVC!

Im going to cut and delete more than half of my 14 TB storage now and save a ton from storage space costs now and in the future. Will toss a coin to my @mifi

titobf commented 1 year ago

This is great. Is the app in Microsoft Store going to be updated also? Thanks

domportera commented 1 year ago

thank you so much for working on this app and integrating this feature :) this will improve my housekeeping tremendously

mifi commented 1 year ago

@titobf yes, but i want to iron out bugs first before releasing to stores. already a few bugs coming in.

samahmoo96 commented 1 year ago

I'm having an issue. I can't seem to convert h.265 to the player in 3.48.2. It just yells at me that the file is not natively supported, I try to convert, it does, and then it yells again. Here's the media info I pulled from VLC. I could send the file as well if you needed to reproduce the bug on your end.

Stream 0 Codec: MPEG-H Part2/HEVC (H.265) (hevc) Type: Video Video resolution: 1920x1080 Buffer dimensions: 1920x1088 Frame rate: 29.970030 Decoded format: Planar 4:2:0 YUV 10-bit LE Orientation: Top left

Stream 1 Codec: A52 Audio (aka AC3) (a52 ) Language: English Type: Audio Channels: Stereo Sample rate: 48000 Hz Bits per sample: 32

Stream 2 Codec: Text subtitles with various tags (subt) Language: English Description: English CC Type: Subtitle

EDIT: Nevermind, fixed it. New bug though. The program freaks out how I mentioned above if you're accessing a file off a network share. When I moved the file to the same PC, however, it was able to load up perfectly fine. So it has some issue with network shares (I'm currently using windows' built-in drive sharing if you wanted to replicate it yourself)

alex-ong commented 1 year ago

Just banding onto everyone else that is thanking you for HEVC support. My previous workflow was converting 12-13 hevc files to h264 using nvenc which took 25~ minutes before i could start any work.

This is excellent!

nosferatu500 commented 1 year ago

https://chromestatus.com/feature/5186511939567616

Enables support for decoding HEVC video on platforms where hardware (e.g., GPU, media accelerator, etc) for decoding HEVC is available (Android 5.0+, macOS 11+, with supported hardware on Windows 8+ and ChromeOS).

I wonder how it will work on macOS 10.15 for example? Regarding this commit

@mifi

JerriAtric commented 1 year ago

Issue: can't see HEVC video in the main window -- just see black

Environment: Mac, Mojave (10.14.6), LC 3.48.2. Footage shot on GoPro 11 in HEVC format.

Interestingly, if I choose "Show thumbnails" the thumbnails do render. Audio plays correctly, and I can perform cuts. I just don't see the video in the main window. H.264 video does display.

LCscreen

UPDATE 22.12.10 - Upgrading to Mac OS 11 (Monterey) solved the problem. Thanks nosferatu500.

nosferatu500 commented 1 year ago

@JerriAtric

HEVC is available (Android 5.0+, macOS 11+, with supported hardware on Windows 8+ and ChromeOS)

nosferatu500 commented 1 year ago

I think the simplest solution here is to split the array for supported and unsupported systems with respect to HEVC.

mifi commented 1 year ago

Oh I assumed it works everwhere. @JerriAtric do you see any error message in the developer tools? I wonder how we can detect if hardware hevc support is enabled from inside losslesscut. Maybe we just need to try it and check if the video tag gives an error? And if it gives ane rror then fallback to generate preview

edit: maybe we need to use this api: https://github.com/StaZhu/enable-chromium-hevc-hardware-decoding#mediacapabilities

nosferatu500 commented 1 year ago

Source

let obj = document.createElement('video');
console.log(obj.canPlayType('video/mp4;codecs="hev1.1.6.L120.90"'));

dpallen commented 1 year ago

Same issue as JerriAtric above, but on Win11 22621.963. Exact same behavior:

Issue: can't see HEVC video in the main window -- just see black Interestingly, if I choose "Show thumbnails" the thumbnails do render. Audio plays correctly, and I can perform cuts. I just don't see the video in the main window. H.264 video does display.

mifi commented 1 year ago

@dpallen are you running newest llc beta? you can work around this for now by either downgrading, or file -> convert to supported format

dpallen commented 1 year ago

Thanks for the quick reply, but I resolved my issue after some more testing. My footage used 4:4:4 chroma subsampling. Using 4:2:0 HEVC footage works perfectly in llc 3.48.2 beta.

mifi commented 1 year ago

So to summarize, in order to detect if the player/device supports a file, (for example HEVC) we need to:

const mediaConfig = {
  type: 'file',
  video: {
    contentType : 'video/mp4;codecs="hev1.1.6.L120.90"',
    width: 1920,
    height: 1080,
    bitrate: 10000, 
    framerate: 30,
  }
}

const { supported } = await navigator.mediaCapabilities.decodingInfo(mediaConfig);
// result.supported indicates whether or not the video with given profile, width, and height can played well on the browser

The problem is to find the contentType parameter. It is described here: https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter but it varies by format/codec. Need to find a library where someone implemented this logic into a library for all formats/codecs. I would think that the output from ffprobe could be transformed into this codec parameter.

Potential implementations for each format/codec:

AV1 spec implementation
MP4

mifi / lossless-cut