mpv-player / mpv

🎥 Command line video player
https://mpv.io
Other
28.73k stars 2.93k forks source link

Playback mode that automatically skips silent parts. #6797

Open LichenSymbiont opened 5 years ago

LichenSymbiont commented 5 years ago

Like this ffmpeg-based script that processes a file to remove silent parts with certain inputs for how silent it should be when cutting and such: github.com/carykh/jumpcutter

Another playback mode that would be useful is to detect silence from one of 2 audio channels, and playing mono.

There's probably a lot of things you can do with a "playback mode" feature.

ghost commented 5 years ago

It'd be pretty trivial to write a filter which does this for audio only. But if video is involved, it would be much more complex.

afcady commented 4 years ago

The audio filter should be disabled if there is any video stream. Is it possible to make an audio filter know whether there is video playing?

Zocker1999NET commented 4 years ago

I would want to use this filter in a video file too, so there should be an option or something like that, if that is even possible.

cankaratepe23 commented 4 years ago

I would like to see this implemented as well.

Would it be possible to have mpv run silencedetect on the file and store the output (internally or in a temp directory) to skip the detected sections maybe?

barolo commented 3 years ago

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

avih commented 3 years ago

What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...

barolo commented 3 years ago

I don't know how it works but it's not a total silence, and surprisingly it's useful with many that I watch, game streams, any educational content, longform talks, reviews, obviously not with music. I'd love to have it for podcasts though. It doesn't skip/cut the content, just speeds it up

cankaratepe23 commented 3 years ago

What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...

For me personally, it's recordings of online lectures. I'm sure this would be useful to many people as well, and it can also be used for similar events such as webinars etc.

barolo commented 3 years ago

Yes, of course. I'm using it for lectures too, together with a small speed up. It's a real godsend, wish I knew about it earlier. It's a huge timesaver Hopefully someone will be able to jank it out of NewPipe, together with scaletempo2 it would be a killer

avih commented 3 years ago

Can someone upload a sample video someplace, and list the first few timestamps+duration of silence which should be removed?

I wonder if ffmpeg's silencedetect can be used in a script in a similar way which cropdetect is used.

barolo commented 3 years ago

silencededetect been used that way for audio only things as per reddit, (outdated) The NewPipe one is pretty aggressive and it skips between words, you can see it happening in video but it's unnoticeable in audio and it's hard to timestamp, I'll look for something with longer pauses

barolo commented 3 years ago

For example, it turns this https://youtu.be/T7SWETadMn0 16 min video into a 9 min video and from listening just to the audio I wouldn't be able to tell, it just sounds like she's speaking without ever catching breath

AFAIK they use this from ExoPlayer

avih commented 3 years ago

Please don't refer to other players and what they do or don't do right. Just give a sample clip and describe the first few timestamps (seconds resolution would be enough) and rough duration where such silence should be removed.

barolo commented 3 years ago

That wasn't my intention, since they're open source I've linked directly to the code bit so it might be of use to someone. As I said it's hard to do because the implementation that I care about removes silence between words, so even shorter than a second, turning 2 hour podcasts into an hour one, without becoming exhausting to listen

afcady commented 3 years ago

Personally I want it for music. Because I would like to skip the 10 to 15 seconds of silence that for some reason initiates many recordings of classical music. I just want the music to start instantly when I start the player.

barolo commented 3 years ago

@afcady that one is achievable with lavfi skipsilence filter, it's pretty much what it's meant for.

pascal-mueller commented 3 years ago

I would love to see this feature. I think a lot of people would. I'd love to be able to do something like:

mpv -detectSilence video.mp4

which results in mpv searching the audio stream for silence (for some given threshold) and opening it. It then set's like a"breakpoints" where the given silent part begins and ends and automatically skips it. Now people say that's a bad solution because you might skip content that's important but most people use it for lectures and how many lectures did you watch where the lecturer showed something without saying anything about it? Rarely, if ever.

To accommodate that, one would need motion detection - but that is resource heavy, so for the sake of efficiency I'd just stick with the above approach.

To give it more user friendlieness, one could enable the auto skip only on the first run through. So you watch the video, you come to a part where the video is silent for let's say 2 seconds and mpv automatically skips it. You now see that you somehow got lost and skip back. Since you skipped back manually, mpv won't skip forward again but let's you watch the previously skipped part.

Would make the life of educational content consumer a lot easier.

rwmpelstilzchen commented 3 years ago

I really hope this feature will be implemented.

IMHO, the user should have two options: to skip silent segments whenever they occur or to skip them only at the beginning and the end of the file.

MinmoTech commented 3 years ago

I would like something similar to this browser extension: https://github.com/vantezzen/skip-silence

barolo commented 3 years ago

This script solves the issue for SOME, videos, some meaning ones with subs/captions as it fast-forwards when no text is visible. [text can be made invisible easily] In practice it should mean most of the lectures and YT videos, works surprisingly well.

Edit. just found that there's one for audio too

pironside44 commented 3 years ago

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

RIGHT? I need this for MPV and as a chrome extension!

barolo commented 3 years ago

T

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

RIGHT? I need this for MPV and as a chrome extension!

There is a chrome/firefox extension already called skip-silence mentioned above, it works fairly well If you want for it to work with your local videos just use something like Jellyfin as media provider, it will work with it too.

ishitatsuyuki commented 2 years ago

Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.

https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b

See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).

Enjoy!

barolo commented 2 years ago

Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.

https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b

See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).

Enjoy!

Thank you so much for this!. It works well. Finally, I can get back to watching lectures on the desktop.

notevenaperson commented 2 years ago

speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua

barolo commented 2 years ago

speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too. Auto-editor based script works best for now, just needs speedup instead of cutting for more comfortable viewing.

notevenaperson commented 2 years ago

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

You might just re-do the timings with an automatic retimer like alass.

I agree that speedup is preferable to cutting.

PriyanshuPriyadarshi commented 2 years ago

There is similar feature in NewPipe and it is very useful while watching lecature. Screenshot_2022-11-07-18-31-58-807_org schabi newpipe debug channeltabs_1_1

Topping1 commented 2 years ago

I have tweaked the script https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 (speed-transition-Audio.lua) and it works better now. Mainly the speed up to 5x during silences was too much and sometimes the upcoming audio would cut out. Now there is a gradual transition to 3x and works better.

ghost commented 1 year ago

Have you found a satisfying way to use ffmpeg's silencedetect / silenceremove in mpv?

I'm interested in experimenting with skipping over any silence in audio lasting more than a second or two.

AN3223 commented 1 year ago

I wrote a script here and there is also another script here. Mine requires hwdec to be off and it works by fast forwarding, so it may or may not be practical for your use case @07416.

EDIT: I'm now seeing bitingsock's script in this thread, it seems to work in a similar fashion.

ghost commented 1 year ago

I wrote a script here and there is also another script here. Mine requires hwdec to be off and it works by fast forwarding

@AN3223 Good recommendations. Do you happen to know an entirely automated script (all three set keybinds)? I plan creating a dedicated profile for speech-only playback and perhaps another for music, skipping silence between songs and discovering hidden tracks. For this workflow manual actions don't quite fit.

AN3223 commented 1 year ago

Hm, I'm not sure. Maybe someone can adapt one of the scripts for that purpose. I don't really use mine and I find it to be unreliable.

DonRichie commented 1 year ago

Hello, I have a proof-of-concept to show. The following mpv lua script: https://gist.github.com/DonRichie/23d8442e91ef8dc4917f9e6a9d6fe5c0 <- outdated, see next post

The idea:

  1. execute ffmpeg in a subprocess to determine silent parts beforehand
  2. Use the information to execute mpv's "seek" command and skip the silent parts There is also a variable edge_delta = 0.06 which can be adjusted to leave a bit of the silence for softening the jumpcuts.

Usage:

  1. save script in ~/.config/mpv/scripts
  2. play local video file (web urls do not work)
  3. Press F2 to toggle on/off , it will need around 1/60 of the video duration to execute ffmpeg, then the osd will show a message and the effect starts to happen.
  4. Use silence_threshold variable in the script to adjust the silence threshold. There is also the possibility to dynamically decrease it with F3 and increase it with F4, but since I only allow one ffmpeg process at once this feature is currently very annoying.

Problems:

Possible improvements:

DonRichie commented 1 year ago

I rewrote my previous proof-of-concept and created a new script based on the same idea. It is now able to stream the ffmpeg output line by line. This means huge video files can now start skipping silence, shortly after the functionality is activated.

See my comments in the file for further information: https://gist.github.com/DonRichie/1628527b41d5076712c97f8d6e76d4db

Have fun. If you find bugs, let me know. I would be happy to hear some opinions.

Main problems:

DonRichie commented 1 year ago

There is a new revision available under the link in my last post. The minimum duration of silence which ffmpeg detects is now configurable. Lower values heavily impact the time ffmpeg needs to scan the file.

Also if it is set too low, it can be that mpv can't skip fast enough This for example will effectively skip back a split second instead of forth: [my] skipping to 17.62 , saved_time=0.02

ferreum commented 1 year ago

Based on https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 I created a highly configurable script that works without external programs. I've published it here: https://github.com/ferreum/mpv-skipsilence I'm surprised that it was possible to make it work so well, very similar to the NewPipe feature when configured with aggressive speedup ramps. Unfortunately the speed changes very easily cause audio-video desynchronization, so I suppose it would still be nice to have something proper implemented in mpv itself (unless someone finds a solution to the desyncs).

alopatindev commented 1 year ago

speed-transition.lua script skips to the next subtitle rather than audio ... Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

If you combine it with OpenAI's whisper-based subtitles generator (specifically this yt-whisper fork) — you will likely get desired result. I use whisper specifically to cut pauses in vlog-recorder with very high precision, the same way yt-whisper generates subtitles.

Upd: something based on whisper.cpp, like whisper-subs, would be even better.

ferreum commented 11 months ago

Short update: With mpv 0.37, the audio/video desynchronization I mentioned above has been fixed. As a result, the skipsilence script works very well now.

barolo commented 10 months ago

speed-transition.lua script skips to the next subtitle rather than audio ... Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

If you combine it with OpenAI's whisper-based subtitles generator (specifically this yt-whisper fork) — you will likely get desired result. I use whisper specifically to cut pauses in vlog-recorder with very high precision, the same way yt-whisper generates subtitles.

Upd: something based on whisper.cpp, like whisper-subs, would be even better.

That's cool and all, there's one issue though, English isn't the only spoken language existing.

Short update: With mpv 0.37, the audio/video desynchronization I mentioned above has been fixed. As a result, the skipsilence script works very well now.

Amazing work, does everything that I wanted from such feature. It's a literal life-saver.

alopatindev commented 10 months ago

That's cool and all, there's one issue though, English isn't the only spoken language existing

There are multilingual models for whisper and whisper.cpp as well (for limited number of languages).