The Future of MoviePy project

[B3QL]

I'm opening the issue in an attempt to provide a platform for discussion on the project's future.

The project is unmaintained and it looks like the original contributors lost interest in it. Currently, few fellows giving their best and taking care of reviewing PRs and fixing the issues related to their projects but it's not enough to keep the project healthy. The library is obsolete to the point that 3rd party open source software that depends on it is having problems with being included in the next Debian release.

If we want to save the project we need to build a community around it and/or seek some financial help from OSS foundations.

Nevertheless, we need YOUR help. If you have an idea of how to help the MoviePy project survive, please contribute in the comment.

[Zulko]

Regarding this:

I spent a few hours reading last night and ffmpeg at its native bin and wrapper levels all operate on filepaths and therefore cannot easily be handled in pure RAM

I might miss a point, but I think this is not true, ffmpeg is capable of both reading and writing from/to stdin/sdtout with pipes, that's in fact how moviepy communicate with ffmpeg.

• For the output: I think ffmpeg can pipe its output to RAM (looks like pyAV can), but not sure. It might depend on format: some formats are meant to be easy to stream, while some formats require to change the video's header after the last frame is written - these might not be streamable to the RAM.
• I believe MoviePy cannot write a video with sound in pure RAM. This is because you can't pipe both video and sound to a file at the same time, and so the sound needs to be in its own file.

[OsaAjani]

• I believe MoviePy cannot write a video with sound in pure RAM. This is because you can't pipe both video and sound to a file at the same time, and so the sound needs to be in its own file.

Yes, I can confirm that. Currently we could easily do pure ram audio, pure ram video, but not both (considering you choose an output format that is streamable). I think it might be possible to do both though, using something like named pipes.

Ideally we'd keep MoviePy beginner-friendly, which means "pip install and start making videos". I agree with the idea of making it beginner-friendly, which is why I would have liked to be able to make preview always available to the user. But I cannot find a way to do that. So the idea was to make the whole thing more coherent, by saying "we do not provide any binary, not ffmpeg, not ffplay, you must install them and provide path to them on your own, and if you dont, moviepy will not even start". What I like in that is, though it need one more step from the user, it remove any form of magical behavior. We need this bin, you must install this bin, you must spoke this bin path.

Maybe the solution should be the complete opposite, taking inspiration from imageio-ffmpeg, and make a small python package just providing binaries for ffplay and make moviepy dependent on it ?

What I really wish we could avoid is a situation where half the binaries are shipped, and half are not.

Talking about imageio, I've always find the readthedoc theme for documentation horrible and not readable, what do you think about moving to the same theme as imageio, https://pydata-sphinx-theme.readthedocs.io/en/stable/index.html ?

[tburrows13]

Talking about imageio, I've always find the readthedoc theme for documentation horrible and not readable, what do you think about moving to the same theme as imageio, https://pydata-sphinx-theme.readthedocs.io/en/stable/index.html ?

Here is my WIP docs rework from a couple of years ago: https://moviepy-tburrows13.readthedocs.io/en/improve-docs/

[OsaAjani]

Thanks @tburrows13 that's what I have found online and I'm using it as the base for the updated doc.

[davidbernat]

@OsaAjani @tburrows13 @Zulko: Caught up on comments. Broadly agree with everything.

1. "I think it might be interesting to start with a short tutorial..." "pip install and start making video"

I agree. Happy to code review, since it seems like @OsaAjani is off to the races on this.

2. FFMPEG io to pure RAM.

Seems conclusive that this is a no. As a general rule I am writing my code bases to only temporarily files and load into ByteIO as the trend toward in RAM is likely to continue in downstream applications. It may be that the filesystem has a filepath that handles this as proxy. Has anyone looked at those solutions?

3. Using ImageIO.

I notice that FFMPEG run through subprocess.run tends to be much faster than MoviePy in many cases. It seems that under the hood, ImageIO is a definitive wrapper for these sort of executions. If so, I will commit a few hours to research ImageIO to catch up. Please specify. Otherwise, I am noting here that numerous speed improvements can be made throughout.

Thanks for bearing the load and please cc/review request me.

[OsaAjani]

@davidbernat I will make sure to ping you for code review and feedbacks when I have something coherent, but for now it's mostly guerrilla mode, I attack on every front when I see an opportunity, and run away. I will need to run a lot of unit tests etc. Before being ready to publish anything good enough for code review.

What lead you to think ffmpeg cannot do pure ram ? 'Cause I'm 100% sure we send data to encode to ffmpeg as stdin, so pure ram. And I'm pretty sure it can output to a pipe as well, and 100% it can stream a file to something like rtp. Even if all of those would failed, I've never used it but it must exists tool that can maintain a file system in pure ram.

Ffmpeg in subprocess is absolutely faster than moviepy, but that's because moviepy as to send each frame one by one to ffmpeg subprocess, so it need or compute all of those before. That is currently the most important bottleneck.

[davidbernat]

@OsaAjani: If you need help, I will don the camouflage and try to dig in.

1. re: RAM. It was mentioned above. It is not clear to me that PIPE in/out is what we want either, though would seem to be a sufficient replacement, depending on security. I will investigate.

2. faster with FFMPEG Yes. Hunch is many moviepy routines can be replaced in this fashion. There are also other improvements that simply arise by tuning ffmpeg for use cases, e.g. without encoding.

[davidbernat]

@OsaAjani: If you need help, I will don the camouflage and try to dig in.

1. re: RAM. It was mentioned above. It is not clear to me that PIPE in/out is what we want either, though would seem to be a sufficient replacement, depending on security. I will investigate.

2. faster with FFMPEG Yes. Hunch is many moviepy routines can be replaced in this fashion. There are also other improvements that simply arise by tuning ffmpeg for use cases, e.g. without encoding.

[OsaAjani]

And OpenCV, Scikit and scipy are now all gone.

Both tools.tracking and tools.segmenting have been lost during those scuffles, so press F to pay respect !

More seriously, those seems like rather advanced task with only a few user case (more particularly segmenting) that would be more suitable to be left for the user to code using the solution he feels match his scenario best (its not the same thing to follow a moving car, a background landscape point and a face, and different algorithms probably give better results than one unified solution).

[davidbernat]

Thank you @OsaAjani. Well done. Really. Can you go be a favor and quickly outline here or in a new ticket what those six or seven tasks were of the tracking module? I am engineering other ML tasks on a side project and I could engineer up those solutions using modern tools, or at least sketch out the interfaces that a user could use. This may be for personal or historical uses.

[OsaAjani]

Well the best thing is to go look at the code of the old modules in https://github.com/Zulko/moviepy/tree/master/moviepy/video/tools, I cannot really explain it more than that ^^.

Mostly tracking offer a method for tracking based on pattern recognition with opencv. that does pattern recognition using cv2.matchTemplate(pat, pic, cv2.TM_CCOEFF_NORMED). That's the only method I think would be worth reintroduced in main project, if you was able to do it without using opencv or any additional lib.

For further tracking with more recent algorithms I think those things would be better as examples/tutorial or even blog posts.

Segmenting find different objects in a clip frame, using ndi.find_objects. really not sure their is that much of an usage for this one.

Both those function mostly works on a frame by frame basis, returning a x,y position for a time t. It would be easy enough to rewrite it for a user.

[davidbernat]

I suppose my question was more extensive: what are the suite of tools that a system should have, not limited by the simplicity of OpenCV. But, I take your point and agree with you more broadly. If you have an amending list, please do.

[OsaAjani]

Oh, well I guess autotracking based on match template would be the base, autodection and tracking of human body/faces would be nice, even more if you could distinguish between individuals to track them one by one (of course this is a slippery slope in term of bad practice, but well, it has legitimate usage and the bad ones already exists anyway). I think auto-detection of things like license plates could also be useful. Anything allowing for automatic detection and blurring of infos that could lead to identify a person is useful. I guess the same could be said for violent content, nudity, etc.

For segmenting I guess object counting and classification are not without interest. The combination of both object detection and tracking based on matching template could probably allow for interesting and generalist use cases.

For me, of all of those, faces auto extracting is really the most useful, because blurring faces automatically for anonymization is probably a very common requirement for a lot of process, from simple automatic video editing to generation of video dataset for machine learning.

[davidbernat]

@OsaAjani cc: @Zulko: I agree with your "advanced utility production" use case, which is beyond the scope of moviepy.

In my opinion, the order of importance is approximately:

1. identification of objects based on similar template (i.e., a make and model of a car)
2. license plate, face detection, address bounding box detection (turnkey and tracking)
3. tracking of objects which process through either of the two pipelines above.
4. body instance recognition (recognizing different bodies are different, and across frames) and their pose detection
5. the application of non-reversible blurring for anonymousness and privacy (including as a service)
6. object segmentation (which outlines objects, and segments into labeled components, in addition to bounding boxes)
7. 'child safe filters' (of which nudity, violent content filters are subsets)
8. online interfaces (c.f., notebooks, shareable scripts, etc)

What else does this list need? @OsaAjani do I have your permission to add your email address to an 'observer' of the LLC ML pipelines work. I am in the process of writing steering documents more broadly and already implementing core technologies of these types. Exciting to think of this reaching fruition robustly several months from now.

[ShakedDovrat]

If you are interested in a view point of a user - I don't think MoviePy should do any computer vision algorithms. I am a computer vision engineer and would never use MoviePy for my algorithms. I do use OpenCV directly and other frameworks such as PyTorch, TensorFlow, Scipy, etc. This field is saturated with many great Python libraries. There are no video editing libraries other than MoviePy. I encourage you to focus your efforts on video editing, where MoviePy is most needed. Thank you for all your efforts 🙏

[OsaAjani]

@OsaAjani do I have your permission to add your email address to an 'observer' of the LLC ML pipelines work.

Sure, not sure I could give it any time but no problem.

If you are interested in a view point of a user - I don't think MoviePy should do any computer vision algorithms.

I agree, with maybe the exception of simple function that would take zone of the screen and do tracking on it, as I guess tracking + blurring of an element is a quite common function for an editor, as well as a tool useful to many users that don't necessarily know how to make their own. But clearly not something essential right now.

[davidbernat]

@OsaAjani Great. Kindly email me your email address. I will add your suggestion to this list.

Advanced Utilities for Production to help monetize creators.

In my opinion, the order of importance is approximately:

1. identification of objects based on similar template (i.e., a make and model of a car)
2. license plate, face detection, address bounding box detection (turnkey and tracking)
3. tracking of objects which process through either of the two pipelines above.
4. tracking of a generic bounding frame throughout camera movements
5. body instance recognition (recognizing different bodies are different, and across frames) and their pose detection
6. the application of non-reversible blurring for anonymousness and privacy (including as a service)
7. object segmentation (which outlines objects, and segments into labeled components, in addition to bounding boxes)
8. 'child safe filters' (of which nudity, violent content filters are subsets)
9. online interfaces (c.f., notebooks, shareable scripts, etc)

=====

Not to worry @ShakedDovrat. We are discussing tools explicitly outside the scope of moviepy. If you are a talented computer vision engineer, we could use your hands-on input development, and suggestions. Consider emailing me from the email address in this spreadsheet to discuss what you already do.

=====

People interested in participating in the Future of MoviePy project 2023/2024 or simply curious about how they can keep abreast of changing capabilities should email the point of contacts in this spreadsheet. https://docs.google.com/spreadsheets/d/1FftaQDTjZi8pMmeLQthY4BNpaKMLB2TDssUnuQZBJQI/edit?usp=sharing

[OsaAjani]

@Zulko, @tburrows13 and everyone, do you think we should keep moviepy.video.io.sliders (for matplotlib) and moviepy.video.io.html_tools (for ipython), or should we consider them out of MoviePy scope and remove them ? Personally I would move for removing them as they are quite specific to both tools, only usefull for few users and seems like having really a lot more to do with either matplotlib or ipython than with moviepy.

[OsaAjani]

Also, I have one more question, concerning the Clip API, on which I would like the point of view of @tburrows13 and @mgaitan as I think they are the main author behind the idea/code, as well as @Zulko as he is (I think) the author behind the original idea in v1. Of course, everyone else POV is appreciated.

So, here is the context:

With the v2.dev, all special effect (aka everything in moviepy.video.fx and moviepy.audio.fx) are automatically added to VideoClip and AudioClip classes as methods at runtime (in main __init__.py, using the inspect module) . The idea being (from what I'd gathered peaking at PR, correct me if I'm wrong) to make it easier and cleaner to call effects and clips.

In v1.X, a similar system was implemented, where some special effects (not all I think), where manually added in editor.py, for the exact same reasons.

I'm sorry but I have to say it: I hate it... No offense @tburrows13 and @mgaitan, your implementation is really neat and I like the way you've done it, and I totally get the idea behind the original. But I find it to be very confusing, as it make things unclear on many levels and I think more difficult to explain to beginners:

• It introduce many way to do the same thing (.fx and direct method access), leading to inconsistencies in code and examples, and generally more risk for errors and misunderstanding.
• It will introduce different way to call custom user defined effects and MoviePy effects.
• It makes static analysis harder, which mean inferior code auto-completion, etc.
• It makes effects looks like VideoClip and AudioClip and methods when they are not, meaning it is harder for someone not familiar with the mechanic to find the source code and understand how things works, as well as tracing and debugging.
• It makes accurate documentation generation of the code more difficult.
• It introduce the risk to override a clip property by defining an effect with the same name.
• And probably a lot more I cannot think of right now.

In addition, in v2.dev, all functions modifying a clip and returning the modified version now starts with with_. This in order to make clear to the end user that those functions will not modify the original clip. I find this to be a good idea, it is a little verbose, but its actually quite practical when using auto-completion, it allow users to easily identify copying mechanism (not always an easy things to do, even more considering decorators and function inter-calling), which is a recurrent problem beginners seems to encounter when using MoviePy. It is also very simple to explain to user: if it start by with_, you have to store the result/reassign the original if you want to use the updated version. The auto-adding of effects to clips kind of broke that consistency.

Okay, writing .fx every-time may not be the sexiest thing, but it is totally transparent to the user, we know exactly what happen, and it is a clear distinction between clip modification and effects.

In my opinion, we should always go for the more straightforward line, and avoid introducing two way to do the same thing if we can. In that optics, I would like to propose that:

1. We remove the auto-addition of all effects as clip methods
2. We only document and promote applying effects through call of clip.fx, even though a user could actually call an effect by doing effect_name(clip, p1, p2, ...).

Opinions ?

[Zulko]

Regarding computer vision tools: the ones included in moviepy were for things like automated head bluring, so relevant to a video editing package. I don't think that found its public and so could be removed (and later provided by add-on library).

Regarding moviepy.video.io.sliders: not sure what these were for, they might even pre-date ipython widgets. Probably not essential for moviepy.

Regarding moviepy.video.io.html_tools: it provides ipython_display which is the way to display a clip or a snapshot of a clip in Jupyter notebooks. This is a very useful feature (probably one of the features I use the most) and it doesn't strictly add a dependency. It should probably be renamed display_in_notebook and added as a clip method clip.display_in_notebook() at some point.

Regarding the fx auto-loading, this is a balance act between sound coding practices and making it easy on users. While "resize" or "change speed" are technically effects, it's easier to have them as clip methods clip.resize() and clip.speedx(), it's more discoverable this way than clip.fx(vfx.resize, 0.5).

I understand that having a hackish way of loading these methods has it's inconveniences though. Maybe the most used effects could be added "manually" as clip methods in the code, and the more exotic ones would be only accessible from the effects modules?

One more comment is that the API design clip.fx(my_effect, **effect_kwargs) did not age well in my opinion, it's a bit of an akward way to call a function without explicitly calling it. Now that all notebooks and editors provide parameter suggestions and type analysis, there would be a big advantage to use classes for effects, like this:

my_new_clip = (
    my_clip
    .with_effect(afx.VolumeMultiplier(factor=0.5))
    .with_effect(fx.SpeedMultiplier(factor=2))
    .with_effect(fx.ColorsToBlackAndWhite())
    .with_effect(fx.LuminosityAndContrast(luminosity=0.8, constrast=0.4))
)

Effects could then even be chained in a single call:

my_new_clip = my_clip.with_effects([
    afx.VolumeMultiplier(factor=0.5))
    fx.SpeedMultiplier(factor=2))
    fx.ColorsToBlackAndWhite())
    fx.LuminosityAndContrast(luminosity=0.8, constrast=0.4)
])

[OsaAjani]

Regarding moviepy.video.io.html_tools: it provides ipython_display which is the way to display a clip or a snapshot of a clip in Jupyter notebooks. This is a very useful feature (probably one of the features I use the most) and it doesn't strictly add a dependency. It should probably be renamed display_in_notebook and added as a clip method clip.display_in_notebook() at some point.

Okay, so we keep that one and remove the others. I would tend to keep the ipython_display name though, as I think having ipython in the name make it extra easy to find for people looking for such a feature.

I understand that having a hackish way of loading these methods has it's inconveniences though. Maybe the most used effects could be added "manually" as clip methods in the code, and the more exotic ones would be only accessible from the effects modules?

In my opinion having only some effects available would make things even harder to understand. If we keep the effects as method of clip paradigm, then we should have all effects accessible. Else it become quite tedious for the user to lookup which one are available. Adding effect manually by doing VideoClip.resize = vfx.resize in __init__.py does not seems to improve the situation for something like vscode. Really I think the way it is done now is the best way, I just think we shouldn't do it...

One more comment is that the API design clip.fx(my_effect, **effect_kwargs) did not age well in my opinion, it's a bit of an akward way to call a function without explicitly calling it. Now that all notebooks and editors provide parameter suggestions and type analysis, there would be a big advantage to use classes for effects.

I agree it would be more readable to have class and constructors instead of function. Should we provide something like an Effect interface ? I kind of like how lightweight it is to write a custom effect though.

We cannot use with_effect as a name though, because the current fx does not copy the clip (the effect is in charge of that, which I think is not necessarily wise - but maybe necessary for some reason I've missed - as I fear users implementing their own effects will frequently forget to do so).

Maybe we should just go and keep effects as they are, and call that a problem for another day ?

[Zulko]

.I would tend to keep the ipython_display name though, as I think having ipython in the name make it extra easy to find for people looking for such a feature.

At the time (2015) notebooks were called ipython notebooks, today they are called jupyter notebooks or something else, so I think there should be notebook in the function name.

In my opinion having only some effects available would make things even harder to understand.

We'll disagree on that, if moviepy v2 removes things as useful and central as clip.resize() clip.crop() I'm not sure that will help people. My opinion is there should still be a resize effect, but also a shortcut method:

class VideoClip:
    def resize (self, factor)
        return self.with_effect(vfx.Resize(factor))

We cannot use with_effect as a name though, because the current fx does not copy the clip (the effect is in charge of that, which I think is not necessarily wise.

Exactly. When I wrote the MoviePy API I wasn't too rigorous about inplace vs outplace modifications and it ended up being unclear in some places (to my defense, it's not always obvious what to do, for instance when we copy a clip that reads from a video file, should we also copy the ffmpeg process that reads from the file? The answer depends on context).

Now I think it should always be out-place (I like the idea of "out-place by default"), and this should be made clear by using a name like with_effect. Something like this:

class VideoClip:
    def with_effect (self, effect)
        clip_copy = self.copy()
        effect.transform_clip_inplace(clip_copy)
        clip_copy.source_clip = self
        clip_copy.effect = effect
        return clip_copy

Just for the sake of discussion, a more rigorous way to implement clips with effects, would be to give them a class of their own. The following doesn't need to copy any clip, and doesn't have a long chain of get_frame() functions calling each other (but it's handling the simplest scenario):

class FrameEffectClip(VideoClip)
    def __init_(self, source_clip, effects):
         self.source_clip = source_clip
         self.effects = effects
         self.duration = source_clip.duration

    def get_frame(t):
        frame = self.source_clip.get_frame(t)
        for effect in self.effects:
            frame = effect.transform_frame(t)
        return frame

my_new_clip = FrameEffectClip(my_clip, [BlackAndWhite(), Crop(x1, y2), Rotate(90)])

I don't want this kind of considerations to block your work. I'd be fine with

• fx becomes with_effect and is outplace
• Ideally effects become classes with a transform_clip_inplace function, to avoid the akward fx(effect, **effect_kwargs)
• Ideally at least a shortlist clip.resize, clip.crop are available as shorthands, and added in the code, not through code introspection, if that can avoid confusion.

[OsaAjani]

We'll disagree on that, if moviepy v2 removes things as useful and central as clip.resize() clip.crop() I'm not sure that will help people. My opinion is there should still be a resize effect, but also a shortcut method:

Okay, I propose we just let things as they are right now. I'm not very happy with the design, but it is honestly not a big thing, and in no way does is prevent anyone from using the programming style he feel like is the more natural. I've already written the doc with example mixing both style and made clear when and how effects are added to clips class as methods.

to my defense, it's not always obvious what to do, for instance when we copy a clip that reads from a video file, should we also copy the ffmpeg process that reads from the file? The answer depends on context

For your defense, this is a nightmare of a question, and the kind of behavior we are actually very rarely interacting with (at least on purpose) on a day to day basis. Furthermore, Python is quite a strange language when it comes to arguments and by reference/by value passing.

Now I think it should always be out-place (I like the idea of "out-place by default"), and this should be made clear by using a name like with_effect.

I'm 100% for it. Out-place by default is clean and simple, easily understandable by any programmer no matter background or expertise.

It create a new problem though... If we say that it is up to with_effect to do the clip outplace, then what happen when we call an effect directly from the clip (something like clip.resize()) ?

Today all effects seems to outplace clip, mainly because they all use transformation functions, and those do out-placing at one point or another by in turn calling out-placing methods (I feel like this have saved us many time just by luck ^^). So, we could ignore the problem for now by just making it as an internal rule to always make sure in our effects to not modify the original clip. The problem does not exists for user defined We would always add a "useless" duplication when call with_effect though. Probably not a big thing.

• fx becomes with_effect and is outplace
• Ideally effects become classes with a transform_clip_inplace function, to avoid the akward fx(effect, **effect_kwargs)
• Ideally at least a shortlist clip.resize, clip.crop are available as shorthands, and added in the code, not through code introspection, if that can avoid confusion.

I will do 1&2, for 3 I will just leave things as they are. Instead of transform_clip_inplace, maybe we can just go for apply (I don't like inplace because most effects use transform, who is not inplace) ?

As for FrameEffectClip I've been thinking of a similar mechanic, but maybe not with a dedicated class, more like an evolution of existing clip, by adding a new effects property, and modifying the original getframe function. Not sure also how it may affect something like chaining with effect modifying clip properties, or how it would impact order with other modification with with* methods. Anyway, this is a problem for a future day, so we still have time to test and see ^^.

Thanks again for your thoughts and opinions, know that they are highly appreciated and esteemed :heart::heart::heart:!

[OsaAjani]

PS : I will also rename ipython to notebook as suggested

[Zulko]

If we say that it is up to with_effect to do the clip outplace, then what happen when we call an effect directly from the clip (something like clip.resize()) ?

To be clear these should be outplace too (everything is outplace) since they are equivalent .fx(resize, factor). These methods should b=rpobably be called clip.resized() and clip.cropped() etc if they are outplace.

an evolution of existing clip, by adding a new effects property, and modifying the original get_frame function

This is pretty close. Where it becomes tricky is that these effects can change a clip's fundamental properties like width, height, duration, it's a bit cleaner when it's clear that width weight and duration are the values post-effects.

[keikoro]

:wave: I've left this issue in my inbox so I can keep an eye on it, but work/RL keep getting into the way of my replying – especially since this kind of discussion requires more time/energy/thought than the average issue. Reason why you may have seen me tag issues, leave the occasional (semi-canned) reply and trigger workflows on PRs (although I go longer between batches now) while "ignoring" this more important issue.

@B3QL I'm sorry, I've been meaning to get back to you on this for several months now:

Btw @keikoro you seem to be one of the most active contributors around here. Can you help me with creating the proper structure for the project to enable more people?

Since what I wanted to weigh in on has less to do with technical details anyway, I hope you'll forgive me for not having caught up with the most recent comments – which I'm not even sure are best left here since they seem to go into implementation specifics, whereas I understood this issue to be asking a broader question, namely "Where do we/where does MoviePy go from here?".

So, IMO/IME, a project like this – a project of this size, with regard to both code base and (would-be) user base, as well as its (apparent) place within other ecosystems, e.g. Linux distros, or as dependency in other projects – needs covering of several different areas. These areas aren't self-contained but overlap, or play into one another, so can/could well be done by the same people. What's lacking is a good amount of people willing to handle these different areas longer term. And maybe it wouldn't actually hurt to have people specifically volunteer to help with specific areas besides programming.

These areas include:

• code maintenance: fixing bugs, refactorings for consistency, fixing outdated examples
• developing the code/project further: new features, major rewrites, refactorings for e.g. upcoming dependency upgrades
• "devops" work: workflows, releases/publishing, dev tooling in general
• ticket management: keeping on top of incoming issues & PRs, grouping related tickets together (labels), closing duplicate/invalid/stale/superseded tickets
• user support: answering user questions or at least directing them to other places where they might find answers, incl. older tickets or WIP PRs
• documentation: fixing outdated documentation, improving existing documentation, preparing updates to the docs for upcoming changes to the code base, taking care of any other user-facing info on various platforms (e.g. READMEs, CONTRIBUTING files)
• actual project management: creating issues for specific tasks/TODOs, coming up with milestones to serve as umbrellas for smaller TODOs, keeping track of said tickets'/milestones' progress, checking in with people assigned to tasks

I'm aware MoviePy is neither a business nor a paid-for open source effort (: I'm not saying there need to be X number of people dedicated to each individual area. Or even that every one of them needs to be covered. But a project like MoviePy needs more than just people interested in working on the code (and the code alone). Even though everyone willing to spend time on improving the code is obviously appreciated as well.

Three other/related things the project is currently missing:

1. a way for maintainers to quickly, easily and reliably communicate with one another about project-related stuff,
2. better organisational infrastructure, i.e. move to an organization account for better permission management (as also suggested somewhere upthread, I believe)
3. clearer communication/transparency about who is actively involved in the project, and in which capacity.

Regarding 1.: there is a Gitter for the project, but the separation I always wanted between the channels – one for user questions, one for intra-project communication, never happened, and I don't really use Gitter anymore. I don't think it's much of a thing anymore in general. Whatever the new means of communication ends up being, it should be something which everyone (still) somewhat actively involved in the project as well as anyone demonstrating willingness to become actively involved is fine with using. E.g. I saw a Google doc to collect [ideas/suggestions/I don't quite remember] linked somewhere above – which is a tool I personally don't use to collaborate with random people on the internet, which makes it something I can't contribute to. Generally, it probably shouldn't be something which any one person has to actively maintain. Which I'd guess would rule out e.g. Discord? (?)

Point 3. is the result of me seeing users occasionally tag people who are listed as maintainers in the README to get their attention – as well as (former) maintainers reacting unnecessarily angrily to being tagged. A quick fix could be to update the maintainers section in the README so it only lists all (more or less) active maintainers + their (last) responsibilities, and to put all major former contributors we want to thank somewhere else, so it's clear there's a separation between the two.

[OsaAjani]

a way for maintainers to quickly, easily and reliably communicate with one another about project-related stuff,

I already hate myself for proposing that but... mailing list ? I mean, I dont really like mailing list, but it seems to be enough for Linux, so it should be enough for us.

@zulko

My opinion is there should still be a resize effect, but also a shortcut method:...

How many of those shortcut would you consider worth adding, could you give me a list of the one that you find necessary ?

[davidbernat]

@keikoro @OsaAjani @tburrows13 @Zulko: just catching up myself.

There is a lot of discussion here so I will be ultra-brief:

1. I am already working on distributed editor / computer vision tools. I.e., what will become face blurring v1.1.
2. I profoundly disagree with removing crop, clip, etc. These are Clip Manipulation tools; not Content Manipulation tools. Essentials v. Production or Publishing tools.
3. No opinion on the .fx package, except to say that the lambda-ization of its code: clip.fx(effect_func, args) is never utilized for processing speed; so I found its use fragile. This became a code restructuring question; and I will not engage in that to help you all move things along.
4. I personally favor a mailing list for discussion prior to issues. But I want Zulko to drive as much as possible at this stage out of respect.

This is where I have been talking about Steering and MoviePy 2024. Please let me add you to the emails. I already intend to drive this forward in a computer vision and machine learning monetization strategy, and would hate to see you not benefit. Please email me at the email address I have in the spreadsheet or on my GitHub. Good morning :) I am quite fond of you folks.

What's lacking is a good amount of people willing to handle these different areas longer term. And maybe it wouldn't actually hurt to have people specifically volunteer to help with specific areas besides programming.

These areas include:

• code maintenance: fixing bugs, refactorings for consistency, fixing outdated examples
• developing the code/project further: new features, major rewrites, refactorings for e.g. upcoming dependency upgrades "devops" work: workflows, releases/publishing, dev tooling in general
• ticket management: keeping on top of incoming issues & PRs, grouping related tickets together (labels), closing duplicate/invalid/stale/superseded tickets
• user support: answering user questions or at least directing them to other places where they might find answers, incl. older tickets or WIP PRs
• documentation: fixing outdated documentation, improving existing documentation, preparing updates to the docs for upcoming changes to the code base, taking care of any other user-facing info on various platforms (e.g. READMEs, CONTRIBUTING files)
• actual project management: creating issues for specific tasks/TODOs, coming up with milestones to serve as umbrellas for smaller TODOs, keeping track of said tickets'/milestones' progress, checking in with people assigned to tasks

===== People interested in participating in the Future of MoviePy project 2023/2024 or simply curious about how they can keep abreast of changing capabilities should email the point of contacts in this spreadsheet. https://docs.google.com/spreadsheets/d/1FftaQDTjZi8pMmeLQthY4BNpaKMLB2TDssUnuQZBJQI/edit?usp=sharing

[keikoro]

Having read back on the comments when @OsaAjani joined the conversation (except for the most recent ones that go into tech details):

I really, really like your suggestions here, @OsaAjani! Also: thank you for tackling the docs!

I already hate myself for proposing that but... mailing list ? I mean, I dont really like mailing list, but it seems to be enough for Linux, so it should be enough for us.

I don't get around to reading/writing personal email much these days, but agree that this would be the easiest/simplest solution. I wonder if the Python Software Foundation would let us make use of their infrastructure? They maintain mailman lists for various Python user groups: https://mail.python.org/mailman/listinfo This would take organisational load off of our shoulders and simultaneously make it more official. Not saying the latter is super relevant at this point, and maybe we could even ask for a private list (for maintainers only, or for the time being), but still.

@davidbernat I don't really know what to make of your repeated requests/proposals to let you/your LLC take over MoviePy/for you to head any organisational rejuvenating/restructuring/... efforts. So far, no-one seems to have reacted overenthusiastically to these suggestions, nor has anyone else been added to the Google doc you created, which signals a lack of interest to me.

[keikoro]

I wonder if the Python Software Foundation would let us make use of their infrastructure? They maintain mailman lists for various Python user groups: https://mail.python.org/mailman/listinfo

Oh, just realised there are way more lists: https://mail.python.org/mailman3/lists/

[davidbernat]

@keikoro cc: @Zulko: I bring funding and a new kind of expertise. As for, the longevity of this project, which is from @Zulko, he has yet to even provide maintainer controls to the people making improvements to the code, so no reasonable person would expect any enthusiastic addition of new directions at this time, but making the conversation happen has been functioning. With total respect to @Zulko, for doing what has been very, very difficult, moviepy is out of date, implemented from a very talented one-person-view with significant deficits to its competitors, and has no recurring maintainers or contributors. You do not think now is the time for the world to be glomming on to my Google Doc (and there is one other person)? I will be happy to provide you my contact information to continue this conversation offline and add you to the document when you feel the time for what I am building has been achieved. Thank you for your insightful contribution.

[keikoro]

@davidbernat I'm not passing judgement on your bigger plans or ideas – I couldn't, really, because there's not enough for me to go on: a quick Google search for your name didn't turn up anything useful and your GitHub profile only tells me we've interacted a couple of times via tickets, which I criticised for being complete. (;

What I'm saying is that everyone subscribed to this issue will have received your message. Further repetition won't help drive your point home but only adds noise at this point. What you are talking about is basically a business proposal; any decision on that kind of thing is ultimately up to @Zulko, and discussion of it should be moved to more appropriate channels.

[davidbernat]

@keikoro You are welcome to find my LinkedIn. As for the tickets, I posted solutions to several of those tickets, too. :) TTYS.

[tburrows13]

It seems to me that there are 2 competing discussions in this thread:

1. A push to get v2.0 out the door, by @OsaAjani. This involves further API cleanup, better docs, removal of no-longer-needed dependencies. All very good things and I'm enjoying reading the progress that has been made. In theory I'd like to contribute to this, but realistically that probably won't happen in any significant way. If there's specific questions or PRs to review, feel free to tag me.
2. Talks about cloud, LLC, monetization (?) from @davidbernat. None of what I've seen here particularly interests me, and I've not seen anyone else offer enthusiasm either. I could believe that there is a need in this space, but I don't see how moviepy could sensibly evolve into any of this. My primary suggestion is to please open a new issue for discussing it so that it doesn't get confused with the rest of it, since it seems to be a complex topic and a large departure from anything that has happened in moviepy before now. If you find that no one is interested in discussing it there, then that shows that there is no enthusiasm from any current maintainers and you should consider making it an add-on project, complete rework from scratch, or fork.

[davidbernat]

I agree with @tburrows13. The issue was included in this ticket at a time when the scope of this ticket was broader. Are maintainers being created or are we will sticking with documentation only?

[tburrows13]

Are maintainers being created or are we will sticking with documentation only?

What do you mean?

[davidbernat]

The conversation one week ago, before you and @Zulko entered, was a limitation that only documentation could be updated, because PRs need to be approved, and those approvals were not being handed out. Conversation does seem to have shifted to specific code changes, but I did not see any direct mention of changing the code yet, only the documentation. From this point forward on this ticket I am simply observing and peer-reviewing on documentation changes, FYI. Thank you, @Zulko et al.

[tburrows13]

Ah right, well I'm all for adding more maintainers then, yes. That's up to @Zulko then, I don't have such permissions. I'd also support moving to a GH organisation, which would lessen the burden on @Zulko, since others could have the permissions.

Also the gitter chat does still seem to work, I don't have any objections to encouraging its use again.

[davidbernat]

I support that as well, the move to an organization. Whatever offline chat functionality is used I will do what I can to join. Not slack, not discord are where I vote. Happy to join the maintainers crew if permissions are granted by @Zulko. Especially keen to code review.

[keikoro]

Are maintainers being created or are we will sticking with documentation only?

The conversation one week ago, before you and @Zulko entered, was a limitation that only documentation could be updated, because PRs need to be approved, and those approvals were not being handed out.

This sounds like PRs are being blocked on purpose, which isn't the case. The argumentation also falls a bit short because updates to the docs equally need to be submitted in the form of PRs.

All of us (active) maintainers can run the automatic checks on submitted PRs and approve PRs, AFAIK. The primary blockers are code reviews + being able to make informed decisions on if/how submitted code fits into the larger v2 rewrite.

I personally cannot review code-heavy PRs for the most part because I've lost track of where we are at resp. where we want to go with v2, and I don't have the time to catch up. Where catching up means: knowing what the code currently does, what it's supposed to do in v2 – or, lacking that, which transformations it should/could undergo to fit in with the rest of v2 – and, ideally, previous discussions, GH issues,.. on the particular problem the code is trying to solve. Hence: informed decision. I currently also don't have a machine available on which I can mess around, run experimental, potentially unstable builds etc.

If we had a general dev branch next to a main branch, I'd perhaps feel more comfortable merging PRs which may later turn out to be in conflict with what's still planned for v2 – provided these accumulated changes got thoroughly tested before finally being merged into the productive/live branch. Lack of time is generally an issue, though.

Happy to join the maintainers crew if permissions are granted by @Zulko. Especially keen to code review.

So far, new maintainers were never added based on their stating their wish to contribute to the project IIRC, but based on demonstrable active inolvement as regular users. Whether via contributions to code or docs, reviews, improvements of workflows/tooling, helping out with tickets/user support, productive input in discussions like these etc.

I haven't caught up with our tickets yet, so dunno if @OsaAjani has submitted anything yet, but based on the docs-related comments earlier, @OsaAjani might be a candidate (depending on what comes of it ^^ – side note: it might make sense to split the docs update across multiple PRs instead of one large monster PR, for which a reviewer has to set a larger amount of time aside). The way @mgaitan got re-involved (if you ignore the fact he's still a maintainer for a moment) is another good example.

[keikoro]

Addendum: @davidbernat, I didn't mean to suggest you are not already actively contributing in the ways outlined above, btw., it's more that your handle/avatar didn't ring a bell for me now beyond issue comments/the issues referenced earlier. + As said, I'm not caught up on new activity on issues or PRs.

[OsaAjani]

I haven't caught up with our tickets yet, so dunno if @OsaAjani has submitted anything yet, but based on the docs-related comments earlier, @OsaAjani might be a candidate (depending what comes of it ^^ – side note: it might make sense to split the docs update across multiple PRs instead of one large monster PR, for which a reviewer has to set a larger amount of time aside). The way @mgaitan got re-involved (if you ignore the fact he's still a maintainer for a moment) is another good example.

Hi, I haven't submitted anything yet, still a work in progress. Multiple PR would be easier to review, but things change kind of everywhere at once, not only in the doc side, but also on the code, and its just too difficult to make things coherent in atomic PR (to be clear, it could probably be done, it's just that I feel like it's already a lot of work without having to think of short-term coherency, I dont want to invest the extra time. Also it does not necessarily match very well with way I am working on it).

[davidbernat]

Feel free to let me know when your docs are ready for eyes, @OsaAjani, and I'll step in as we discussed.

@keikoro: It was an accurate review of the ticket at the time I was summarized. Thank you for your input and continued leadership of maintaining the code. And many of us who spend the last year depending on this package can attest, we would not be here today without you having written what you have written and knocking out those open tickets one at a time. If you find that the maintainers require more manpower to keep up with the demand you can find me right where I am. Cheers. xo

[OsaAjani]

Following your proposition of using class for effect @zulko, I got something like that.

Current version

def scroll(
    clip, w=None, h=None, x_speed=0, y_speed=0, x_start=0, y_start=0, apply_to="mask"
):
    if h is None:
        h = clip.h
    if w is None:
        w = clip.w

    x_max = w - 1
    y_max = h - 1

    def filter(get_frame, t):
        x = int(max(0, min(x_max, x_start + round(x_speed * t))))
        y = int(max(0, min(y_max, y_start + round(y_speed * t))))
        return get_frame(t)[y : y + h, x : x + w]

    return clip.transform(filter, apply_to=apply_to)

Called with :

from moviepy.editor import *

clip = VideoFileClip('./example.mp4')
clip = clip.fx(vfx.scroll, y_speed=1.5)
clip.write_videofile('./scroll.mp4')

New version :

from moviepy.Effect import Effect

class Scroll(Effect) :

    def __init__ (self, w=None, h=None, x_speed=0, y_speed=0, x_start=0, y_start=0, apply_to="mask") :
        self.w = w
        self.h = h
        self.x_speed = x_speed
        self.y_speed = y_speed
        self.x_start = x_start
        self.y_start = y_start
        self.apply_to = apply_to

    def apply(self, clip) :
        if self.h is None:
            self.h = clip.h

        if self.w is None :
            self.w = clip.w

        x_max = self.w - 1
        y_max = self.h - 1

        def filter(get_frame, t):
            x = int(max(0, min(x_max, self.x_start + round(self.x_speed * t))))
            y = int(max(0, min(y_max, self.y_start + round(self.y_speed * t))))
            return get_frame(t)[y : y + self.h, x : x + self.w]

        return clip.transform(filter, apply_to=self.apply_to)

Note that Effect is a new interface sitting in moviepy.Effect ensuring all effects will implement the apply method, with an instance of Clip as first parameter).

Called with:

from moviepy import *

clip = VideoFileClip('./example.mp4')
clip = clip.with_effect(vfx.Scroll(y_speed=1.5))
clip.write_videofile('./scroll.mp4')

I also added the support for list of effects, which is less verbose than chaining. So, we could do:

from moviepy import *

clip = VideoFileClip('./example.mp4')
clip = clip.with_effect([vfx.Resize(0.5), vfx.Scroll(y_speed=1.5)])
clip.write_videofile('./scroll.mp4')

Honestly, I'm not sure how I feel about the change. I like that:

• we now have a working auto-completion when writing the effect
• it makes ultra easy the creation of effect combination, meaning a user can easily create and reuse a complex effect made of simpler one (I could imagine an OldMovie effect made of a list of simpler effects, like BlackAndWhite, Grainy and Flickering).
• with_effect is longer than fx, but a lot more understandable
• I feel like the fact effects are objects make it more intuitive for someone coming from other languages that any element of the object is accessible from object method (probably not so obvious when working with function only)

I dont like that:

• its more code to write an effect
• We have to do the boilerplate assignment in init. Though I think in some case it could be a nice place to do things like logic on default values to simplify things in the main code, in practice we frequently need clip info for that and dont have access to clip yet. Maybe this could be improved with a decorator though. Making effects dataclass could also probably be a solution.

@Zulko, do you still think this is the way to go ? If so I will do the change for all effects (I will not improve/refactor them though, just do the plumbing to turn them into class and make things work). And find a way to also add them as clip methods.

I think that will be the last evolution I will do though, I still have a lot of work to do on the doc, and probably a lot of rewriting of unit/actual tests, as well as miscellaneous polishing to do, and I dont have a lot of time left before I have to get back to real payed work ^^.

[keikoro]

@OsaAjani @Zulko Could these discussions about code/docs be moved to a different issue?

[davidbernat]

Following your proposition of using class for effect @Zulko, I got something like that. New version :

from moviepy.Effect import Effect

class Scroll(Effect) :
```python

Short chime in here @OsaAjani:

1. I really like this new change. Clear. Understandable. The effects themselves are encapsulated with classes, yet easily written in clear user code. Chaining effects works. I think chaining makes more sense than using a list input; as, I think chaining is always elegant. (It requires backend software to be written more restrictive). I like that a lot, and the extra few characters are nothing compared to code reability.
2. What do you mean 'not you have to save'? Because a viewer was removed?
3. I agree @keikoro we should move that previous post of yours to a new issue. Mazel Tov!

[OsaAjani]

Well, I'm not against a new issue just for talking about code, but for now this issue is kind of the best place to have every relevant person opinion, so maybe it ain't such a bad place (at the end of the day, better be in the bad place with the good people than in the good place with nobody ^^) ?

I was also wondering about moving transitions into fx and turning them into efffects? We have only 4 of thoses, and they really are nothing more than effects and effects on masks. We could easily consider them as simple effects (I suspect it's already more or less how everybody does transitions right now, by applying effect to masks) and avoid maintaining two different API. Thoughts ?

[davidbernat]

Well said. Give me the day to look into the transitions code. I presume you mean the fade-in/fade-out? I noticed the API was buggy with lots of fiddling to get what was supposed to work work, and I want to dive in to the moviepy core, and ffmpeg core to see how it is executed. At the risk of expanding scope of this conversation, is there no direct API in ffmpeg to simply turn a video into a numpy? I’ll implement something at some point I’m sure.

[keikoro]

@OsaAjani I'm not reading any of the tech back-and-forth myself – I simply don't have the time, and it means I'll miss comments relevant to what this issue was actually about if they are embedded in tech-heavy talk. Which surely can't be the intention.

This issue really has gone off-topic and will also lose other people interested in the original topic. If this were a different repo, I'd already have unsubscribed from the issue. If this were a different issue within this repo, I would have closed it. Anyone interested in/able to spend time on implementation will follow a new issue with equal interest if you drop a link in here.

[davidbernat]

LOL. @OsaAjani Create a new ticket; good people follow to good tickets. :) Or you can email me offline like you already did. I recommend building a ticket for the public discussion. Feel free to copy my comments over as you do. Link your ticket directly please. @Zulko happy to be added to the maintainers list whenever you feel your team is ready to update PRs.