Open LichenSymbiont opened 5 years ago
It'd be pretty trivial to write a filter which does this for audio only. But if video is involved, it would be much more complex.
The audio filter should be disabled if there is any video stream. Is it possible to make an audio filter know whether there is video playing?
I would want to use this filter in a video file too, so there should be an option or something like that, if that is even possible.
I would like to see this implemented as well.
Would it be possible to have mpv run silencedetect on the file and store the output (internally or in a temp directory) to skip the detected sections maybe?
I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it
What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...
I don't know how it works but it's not a total silence, and surprisingly it's useful with many that I watch, game streams, any educational content, longform talks, reviews, obviously not with music. I'd love to have it for podcasts though. It doesn't skip/cut the content, just speeds it up
What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...
For me personally, it's recordings of online lectures. I'm sure this would be useful to many people as well, and it can also be used for similar events such as webinars etc.
Yes, of course. I'm using it for lectures too, together with a small speed up. It's a real godsend, wish I knew about it earlier. It's a huge timesaver Hopefully someone will be able to jank it out of NewPipe, together with scaletempo2 it would be a killer
Can someone upload a sample video someplace, and list the first few timestamps+duration of silence which should be removed?
I wonder if ffmpeg's silencedetect can be used in a script in a similar way which cropdetect is used.
silencededetect been used that way for audio only things as per reddit, (outdated) The NewPipe one is pretty aggressive and it skips between words, you can see it happening in video but it's unnoticeable in audio and it's hard to timestamp, I'll look for something with longer pauses
For example, it turns this https://youtu.be/T7SWETadMn0 16 min video into a 9 min video and from listening just to the audio I wouldn't be able to tell, it just sounds like she's speaking without ever catching breath
AFAIK they use this from ExoPlayer
Please don't refer to other players and what they do or don't do right. Just give a sample clip and describe the first few timestamps (seconds resolution would be enough) and rough duration where such silence should be removed.
That wasn't my intention, since they're open source I've linked directly to the code bit so it might be of use to someone. As I said it's hard to do because the implementation that I care about removes silence between words, so even shorter than a second, turning 2 hour podcasts into an hour one, without becoming exhausting to listen
Personally I want it for music. Because I would like to skip the 10 to 15 seconds of silence that for some reason initiates many recordings of classical music. I just want the music to start instantly when I start the player.
@afcady that one is achievable with lavfi skipsilence filter, it's pretty much what it's meant for.
I would love to see this feature. I think a lot of people would. I'd love to be able to do something like:
mpv -detectSilence video.mp4
which results in mpv searching the audio stream for silence (for some given threshold) and opening it. It then set's like a"breakpoints" where the given silent part begins and ends and automatically skips it. Now people say that's a bad solution because you might skip content that's important but most people use it for lectures and how many lectures did you watch where the lecturer showed something without saying anything about it? Rarely, if ever.
To accommodate that, one would need motion detection - but that is resource heavy, so for the sake of efficiency I'd just stick with the above approach.
To give it more user friendlieness, one could enable the auto skip only on the first run through. So you watch the video, you come to a part where the video is silent for let's say 2 seconds and mpv automatically skips it. You now see that you somehow got lost and skip back. Since you skipped back manually, mpv won't skip forward again but let's you watch the previously skipped part.
Would make the life of educational content consumer a lot easier.
I really hope this feature will be implemented.
IMHO, the user should have two options: to skip silent segments whenever they occur or to skip them only at the beginning and the end of the file.
I would like something similar to this browser extension: https://github.com/vantezzen/skip-silence
This script solves the issue for SOME, videos, some meaning ones with subs/captions as it fast-forwards when no text is visible. [text can be made invisible easily] In practice it should mean most of the lectures and YT videos, works surprisingly well.
Edit. just found that there's one for audio too
I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it
RIGHT? I need this for MPV and as a chrome extension!
T
I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it
RIGHT? I need this for MPV and as a chrome extension!
There is a chrome/firefox extension already called skip-silence mentioned above, it works fairly well If you want for it to work with your local videos just use something like Jellyfin as media provider, it will work with it too.
Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.
https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b
See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).
Enjoy!
Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.
https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b
See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).
Enjoy!
Thank you so much for this!. It works well. Finally, I can get back to watching lectures on the desktop.
speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua
speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua
Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too. Auto-editor based script works best for now, just needs speedup instead of cutting for more comfortable viewing.
Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.
You might just re-do the timings with an automatic retimer like alass.
I agree that speedup is preferable to cutting.
There is similar feature in NewPipe and it is very useful while watching lecature.
I have tweaked the script https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 (speed-transition-Audio.lua) and it works better now. Mainly the speed up to 5x during silences was too much and sometimes the upcoming audio would cut out. Now there is a gradual transition to 3x and works better.
Have you found a satisfying way to use ffmpeg's silencedetect / silenceremove in mpv?
I'm interested in experimenting with skipping over any silence in audio lasting more than a second or two.
I wrote a script here and there is also another script here. Mine requires hwdec to be off and it works by fast forwarding, so it may or may not be practical for your use case @07416.
EDIT: I'm now seeing bitingsock's script in this thread, it seems to work in a similar fashion.
I wrote a script here and there is also another script here. Mine requires hwdec to be off and it works by fast forwarding
@AN3223 Good recommendations. Do you happen to know an entirely automated script (all three set keybinds)? I plan creating a dedicated profile for speech-only playback and perhaps another for music, skipping silence between songs and discovering hidden tracks. For this workflow manual actions don't quite fit.
Hm, I'm not sure. Maybe someone can adapt one of the scripts for that purpose. I don't really use mine and I find it to be unreliable.
Hello, I have a proof-of-concept to show. The following mpv lua script: https://gist.github.com/DonRichie/23d8442e91ef8dc4917f9e6a9d6fe5c0 <- outdated, see next post
The idea:
Usage:
Problems:
Possible improvements:
I rewrote my previous proof-of-concept and created a new script based on the same idea. It is now able to stream the ffmpeg output line by line. This means huge video files can now start skipping silence, shortly after the functionality is activated.
See my comments in the file for further information: https://gist.github.com/DonRichie/1628527b41d5076712c97f8d6e76d4db
Have fun. If you find bugs, let me know. I would be happy to hear some opinions.
Main problems:
There is a new revision available under the link in my last post. The minimum duration of silence which ffmpeg detects is now configurable. Lower values heavily impact the time ffmpeg needs to scan the file.
Also if it is set too low, it can be that mpv can't skip fast enough This for example will effectively skip back a split second instead of forth: [my] skipping to 17.62 , saved_time=0.02
Based on https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 I created a highly configurable script that works without external programs. I've published it here: https://github.com/ferreum/mpv-skipsilence I'm surprised that it was possible to make it work so well, very similar to the NewPipe feature when configured with aggressive speedup ramps. Unfortunately the speed changes very easily cause audio-video desynchronization, so I suppose it would still be nice to have something proper implemented in mpv itself (unless someone finds a solution to the desyncs).
speed-transition.lua script skips to the next subtitle rather than audio ... Edit: Found another: sub-skip.lua
Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.
If you combine it with OpenAI's whisper-based subtitles generator (specifically this yt-whisper fork) — you will likely get desired result. I use whisper specifically to cut pauses in vlog-recorder with very high precision, the same way yt-whisper generates subtitles.
Upd: something based on whisper.cpp, like whisper-subs, would be even better.
Short update: With mpv 0.37, the audio/video desynchronization I mentioned above has been fixed. As a result, the skipsilence script works very well now.
speed-transition.lua script skips to the next subtitle rather than audio ... Edit: Found another: sub-skip.lua
Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.
If you combine it with OpenAI's whisper-based subtitles generator (specifically this yt-whisper fork) — you will likely get desired result. I use whisper specifically to cut pauses in vlog-recorder with very high precision, the same way yt-whisper generates subtitles.
Upd: something based on whisper.cpp, like whisper-subs, would be even better.
That's cool and all, there's one issue though, English isn't the only spoken language existing.
Short update: With mpv 0.37, the audio/video desynchronization I mentioned above has been fixed. As a result, the skipsilence script works very well now.
Amazing work, does everything that I wanted from such feature. It's a literal life-saver.
That's cool and all, there's one issue though, English isn't the only spoken language existing
There are multilingual models for whisper and whisper.cpp as well (for limited number of languages).
Like this ffmpeg-based script that processes a file to remove silent parts with certain inputs for how silent it should be when cutting and such: github.com/carykh/jumpcutter
Another playback mode that would be useful is to detect silence from one of 2 audio channels, and playing mono.
There's probably a lot of things you can do with a "playback mode" feature.