search

ManimCommunity / manim

A community-maintained Python framework for creating mathematical animations.
https://www.manim.community
MIT License
21.42k stars 1.57k forks source link

Post PyAV-Refactor: make sure pydub is only fed .wav or .raw files #3831

Closed behackl closed 1 month ago

behackl commented 3 months ago

If an pydub.AudioSegment is created from a file that is neither a .wav or .raw file, the library tries to interact with the ffmpeg CLI. Two possible resolutions:

jeertmans commented 3 months ago

If an pydub.AudioSegment is created from a file that is neither a .wav or .raw file, the library tries to interact with the ffmpeg CLI. Two possible resolutions:

  • Tell users they should have ffmpeg installed if they want support for non-.wav/.raw audio files 😢
  • ... or convert manually using pyav (as far as possible?)

This is possibly a major disadvantage of Pydub: it only handles raw data, and supports other formats using FFmpeg. I also encountered that with #3763 (which I did not have much time to work on lately) - and maybe a good (but complex) thing might be to only rely on PyAV. Pydub is actually a very small library, and rewriting the few blocks we only really need shouldn't be too hard.

behackl commented 3 months ago

I was thinking about this -- indeed, instead of trying to salvage the situation locally (which I've tried in #3832), it would probably be cleaner to reimplement the little functionality that we need from scratch. While this would be more work, it would also be a more flexible solution that we could tailor specifically to our needs (e.g., with audio in movie segments).

As far as I've looked into it, the only features we (currently) need from an AudioSegment are:

In fact, it would not be too hard to write a small wrapper class with some methods for working with raw audio data, plus appropriate coversion methods that use pyav to read the raw data from a given file. I'll think some more about it, but as of right now I feel like I might prefer this approach.

jeertmans commented 3 months ago

Fair point! Do you think I could try to ship that as part of #3763 (when I have some time, later this summer) - or if this should be part of another PR?

JasonGrace2282 commented 3 months ago

As a reviewer, I would find it easier to review if they were in separate PRs. Separate PRs also have the additional benefit that we can merge one earlier then the other, so that we have more time to test the changes. That being said, if it's easier to incorporate it into the PR you mentioned, feel free too (at least from my point of view).