Moving Core Renderer to a new Project

[naveen521kk]

This was discussed in yesterday's meeting that the Manim Core should be small and easily understandable, and shouldn't contain complicated code. As in it should only contain Renderer, Scene, Camera, File Writer and basic Mobject structures.

My thoughts on this:

• Create a new project, call it manim-core and move all the Renderers, Camera, Scene and anything we consider important. Let it have the same module name, manim and I have seen that this works. This repository will have a dependency on the other and published as a separate PyPI package.
• manim-core would only have as much as it is useless without other parts of the library, with no other extra dependencies like ManimPango, NetworkX.
• Make this project, manim a collection of curated plugins by the community, while this doesn't affect the core library.
• Making parts of library optional to install, for example, one could be able to just get geometric Mobjects + Core renderer.

[GameDungeon]

While I can see this becoming a topic of hot debate, I for one support this idea.

[GameDungeon]

Also, I commonly see: "Manim is not an X library, it's an animation library. With this setup, it can be anything without bogging down the whole package.

[leotrs]

I find this idea interesting. If others agree, we would just have to determine very clearly what goes in the core and what goes in this repository. Other plugins can be more free-for-all.

@naveen521kk what is your vision for this repository then? If much of it will be partitioned into different plugins, what goes into this "main" plugin? Are you thinking this will mainly be a library of common mobjects and perhaps other utility functions? If so we can call it manim-common, as it will probably be required by most other plugins.

This will also allow for more granularity when it comes to required reviews. For example, manim-core can have 2 required reviews per PR, while other plugins may be more lenient.

[GameDungeon]

Along with this, we can create an Offical Plugin library that has high standards and checks, and then the rest are free for all.

[WampyCakes]

I'm generally opposed to this for a few reasons. If these can be satisfactorily taken care of, I'm a bit more open to this direction.

1. This seems to make documentation difficult. You're going to basically be breaking the library into many plugins. Are these plugins going to all be in the same documentation or their own? If their own, not only does this make it difficult to traverse, but the standards of what constitutes good documentation will be wide and varying.
2. What practical use is manim-core to the end user? I understand if you want the core of the library to be simple and only what's necessary for the developers, but if it provides no practical benefit to the end user (and perhaps more confusion and grief with using manim), it's only going to increase the number of questions received such as:
   • What's the difference between manim and manim-core?
   • What should I install?
   • What can I do with manim-core?
   • Why can't I do XYZ? (To which they'll be told they're missing whatever plugin has that feature)

3. It seems I'm hearing two contradictory things here.

   Make this project, manim a collection of curated plugins by the community, while this doesn't affect the core library.

   This makes it sound like manim would basically just be everything that currently exists (like graphs and barcharts and such) while manim-core would be just what's necessary to render, in which case my 2nd point above comes into play. The other thing is that if these are plugins "by the community" we'll have a lot of problems that will make manim less enjoyable and easy to use (somewhat outlined in point 4 below). For example, someone could make a plugin that would be in this curated assortment and then abandon it with no trace. In this case, we now have a feature that only one person really contributed to without anyone else to maintain it. I understand that some parts of manim are currently like this (such as pango and opengl), but I think it still makes sense to support text and opengl for their own reasons that I won't go into because it's out of the scope of this issue. The way manim is currently structured helps ensure that even if someone gets hit by the bus, their contribution likely won't wither to death as the library moves farther into a direction that breaks functionality.

   Making parts of library optional to install, for example, one could be able to just get geometric Mobjects + Core renderer.

   This makes it sound like two things are going to happen. One is that it will make the installation process more tedious. The second thing is that this seems to contradict your other point. I don't see how manim is a curated assortment of plugins while also allowing you to install individually what you want.

4. This point is where my strongest disagreement with this direction comes in. Should manim go this direction of making everything a plugin, I'm very worried that it will no longer receive the high amount of maintenance that manim enjoys currently. As the author of one of the manim plugins that currently exists, I must say I do not give it the attention it deserves. It doesn't even have my latest additions pushed to it or released. Nor is my documentation consistent with manim's. And it also has no tests to know if it's compatible with new versions of manim. One of the ongoing projects with manim is tests and code coverage. If it all turns to plugins, we'll lose good testing, docs, and maintenance. It's more difficult to create a bunch of tests for something that doesn't even exist in the same codebase.

I think the way we currently do it is relatively smart. If something is worth being available as part of the library (either because it's simple, widely needed, would be widely used, etc.), then it goes in the library. If it's something that others could use but doesn't really fit the scope of manim's core library (like the rubikscube), then it belongs as a plugin. This also helps solve the problem I raise in point 3a. I could actually make a case that manim-onlinetex and manim-fonts belong in the core library because they serve a different type of purpose than manim-rubikscube.

These are not my exhaustive thoughts on the topic, but rather what's come to mind at the moment. I must say, I do not foresee this being a particularly wise or practical change in the long run. And as others have said before regarding manim refactors, time spent on this is time that could have been spent making manim better for the end users, which I don't think this will accomplish.

[GameDungeon]

1: If we do an official plugin library we can have a mini section in the main docs for each plugin.

2: If we have pip install manim grab all the main plugins this should not be an issue, but I see your point

3: I think the above could solve this issue.

4: Maybe the main plugin library could fix this?

You do make some good points, and the above are just some suggested solutions.

[WampyCakes]

@GameDungeon Those are highly dependent on people actually following what's prescribed in the docs, which we've seen not be particularly effective in the past. Currently, additions and changes to manim work because you have to get your PRs reviewed, approved, and merged.

To your 2, if that's what pip installing manim does, then there's little to no point in doing this anyway.

I'd also say that I'm not sure if this suggestion is coming out of what's actually best for manim or whether it's coming from a conflict between having manimpango and opengl support.

[leotrs]

I do agree that in general this will require more maintenance, which is not our strong suit.

[GameDungeon]

ok, I think this is likely to be rejected. But before we do, let's look at some other projects structured like this, and see how they work so well.

[naveen521kk]

I find this idea interesting. If others agree, we would just have to determine very clearly what goes in the core and what goes in this repository. Other plugins can be more free-for-all.

So, the core part of Manim should contain:

• Renderers
• Scene
• Camera
• base Animation class
• Utils that is used in the above-mentioned things.

@naveen521kk what is your vision for this repository then? If much of it will be partitioned into different plugins, what goes into this "main" plugin? Are you thinking this will mainly be a library of common Mobjects and perhaps other utility functions? If so we can call it manim-common, as it will probably be required by most other plugins.

Essentially this isn't about splitting everything. The core idea behind this issue was to just bring out the "Core Manim" (things mentioned above) into a separate project and maintain it there, while this repository will have every other thing other than the "Core Manim", including geometry mobject, different Animations so on and so forth. This would help in making, "Core Manim" small and easily understandable.

This seems to make documentation difficult. You're going to basically be breaking the library into many plugins. Are these plugins going to all be in the same documentation or their own? If their own, not only does this make it difficult to traverse, but the standards of what constitutes good documentation will be wide and varying.

As I already told you, this isn't about splitting everything into plugins, it seems fine to have "Core Manim" documentation separately which users won't need IMO.

What practical use is manim-core to the end user? I understand if you want the core of the library to be simple and only what's necessary for the developers, but if it provides no practical benefit to the end user (and perhaps more confusion and grief with using manim), it's only going to increase the number of questions received such as:

Yes, this is just a developer intention of having "Core Manim" simple, as a user you won't need to care about this and as usual you can do pip install manim to install. You can't do anything worthy with manim-core other than creating your own Mobjects and animating it, for everything else it requires full install of Manim.

Making parts of library optional to install, for example, one could be able to just get geometric Mobjects + Core renderer.

This makes it sound like two things are going to happen. One is that it will make the installation process more tedious. The second thing is that this seems to contradict your other point. I don't see how manim is a curated assortment of plugins while also allowing you to install individually what you want.

Sorry, for not being clear, this issue is essentially about moving the "Core Manim" to a new repository as a new project. There shouldn't be any installation issue because Manim should be depending on manim-core, so that won't be a problem.

[leotrs]

Thanks for clarifying, I misunderstood at first.

Some libraries have a module like docutils.core and everything else is in other submodules like docutils.utils etc. Is this something similar to what you are saying? Would your plan also work by putting the "core" stuff into a submodule rather than a different repository/package?

[GameDungeon]

While that could work, I don't think that is what they were saying. I do like that idea though.

[naveen521kk]

Some libraries have a module like docutils.core and everything else is in other submodules like docutils.utils etc. Is this something similar to what you are saying? Would your plan also work by putting the "core" stuff into a submodule rather than a different repository/package?

What I mean was to have "Core Manim" as a different project and different repository with more strict rules on reviews, it should have the module name as manim though.

For example, have a look at jaraco.envs and jaraco.home, both have the same module name jaraco and each adding a file to the same module. Essentially, we would move manim.renderer like it to the other repo and some parts of other parts to other repo. It's as simple as just splitting things for convenience and when installed pip will put everything together for manim to be usable.

[leotrs]

I understood what you meant, I was just trying to offer an alternative that may cause less problems :)

[WampyCakes]

Assuming I am not misunderstanding still about what all you meant with plugins and such. If manim would basically stay the same as it is now and just compartmentalized a bit more, then I prefer @leotrs's idea of modules to new repositories and packages and such. I think multiple repositories would make contributing more difficult.

[naveen521kk]

That should be fine, but I still like the idea of moving it to a new repository and maintaining it with more attention, maybe how about doing both. We can first move the core things into a submodule, eg. manim.core and then move the submodule into another repo?

[GameDungeon]

Too much work, for little reward

[eulertour]

The separation should be the other way around; a manim should contain things that everyone will use and e.g. manim-full will contain that with more plugins that people may want. I don't see any problem with keeping the plugin code this repository initially in order to ease the transition, but they shouldn't be installed by default.

[GameDungeon]

Honestly, the way things are right now is fine. I think we should add many of the future features into plugins. Right now manim is a good core animation library and has become so much more than an "Animation engine for explanatory math videos." I feel like a carefully curated library of plugins should be created. These plugins could extend the core function of manim to do anything. Want to make a physics animation. Just use manim with a plugin. Graphing, Same. We should start treating manim like animation and rendering library for any purpose.

[eulertour]

I don't think I disagree with any of that past the first two sentences. But I do think things like pango bindings, jupyter support, sound, webgl rendering, and gradually all of the cairo-specific code should be moved into plugins so we can trim down all the excess code we've been building up.

I still don't think they need to necessarily be moved into a new repo as a first step, just separated from the essential code.

[GameDungeon]

I think this should be closed and a bunch of new issues opened. These issues being, moving each of those unessential pieces of code into plugins. I think it is pretty clearly decided that we should keep this repository, and then trim the fat into plugins.

[eulertour]

We can create new issues and mark this one as depending on those, so this one can act as a tracker for the subtasks.

[GameDungeon]

Makes sense. I guess I'll go make those.

[GameDungeon]

Anything else that should be made into a plugin? I think sound should stay.

[GameDungeon]

I also think jupyter support should stay.

[eulertour]

I think the list from before is a good place to start.

The plugins shouldn't be difficult to install or anything, and don't have to go into a different repo. Those features just won't be used by the vast majority of people who use manim.

[naveen521kk]

The plugins shouldn't be difficult to install or anything, and don't have to go into a different repo.

I think we were discussing moving things into a new repo, right? Can you elaborate on what you meant by that?

[eulertour]

They should go into a different repo eventually, but I think it'd cut down on friction if we don't focus on that from the start and instead just focus on isolating the code in a plugins directory at first. It's only a suggestion though, I don't have a really strong opinion either way.

[GameDungeon]

How would that work with our current plugin setup?

[GameDungeon]

Due to the turbulent-ness of the repository, we decided to put this on hold until OpenGL move is completed.

[eulertour]

We should be specific in what we mean by completed. Does this mean that it should be used by default?

[GameDungeon]

Until the render code is less volatile. A quote from @behackl "it is currently [not] the best moment to pursue these changes, given that there might be a lot of interface changes from the ongoing migration to OpenGL"

[eulertour]

That's not a measurable goal since "less volatile" is subjective.

If we can pin it down to switching the default or a certain level of test coverage there won't be any ambiguity as to when we should restart this.

[GameDungeon]

I don't really have an objective goal. For now, we can call it, "When OpenGL is switched to the default."

[markromanmiller]

I think the move to more OpenGL should be done with this refactoring in mind. I'm starting to touch the OpenGL code (I'm basically not going to make my next video without it) and it has brought up a bunch of questions for me, starting with "What is a Camera?" and "What is a Renderer?" and then moving on to Mobject, Animation, Scene, etc. If those are well-defined as we move to OpenGL, it will make the move to plugins a lot easier.

[RickyC0626]

ok, I think this is likely to be rejected. But before we do, let's look at some other projects structured like this, and see how they work so well.

Discourse currently uses plugin architecture to provide additional optional features, although they have been experimenting with it for about 4 years now. They have a default set of plugins that are shipped with the software, but hundreds of community-supported ones (with some members forming companies around maintaining these plugins).

Some libraries have a module like docutils.core and everything else is in other submodules like docutils.utils etc. Is this something similar to what you are saying? Would your plan also work by putting the "core" stuff into a submodule rather than a different repository/package?

What I mean was to have "Core Manim" as a different project and different repository with more strict rules on reviews, it should have the module name as manim though.

For example, have a look at jaraco.envs and jaraco.home, both have the same module name jaraco and each adding a file to the same module. Essentially, we would move manim.renderer like it to the other repo and some parts of other parts to other repo. It's as simple as just splitting things for convenience and when installed pip will put everything together for manim to be usable.

That's a good idea. The more sensitive sections of manim could be split up into their own repo so that they can have their own workflow and review process. It also filters out higher priority and lower priority PRs as the community version gains popularity. Meanwhile, we can adopt the plugin architecture internally for the rest of manim so that developers have an easier time navigating and testing specific sections/plugins for specific features.

What I'm envisioning for manim is that certain features will expand beyond just math animations and form their own niches over time. Eventually we'll have a medium-sized collection of libraries related to physics, engineering, game development, music visualization (@jsonvillanueva), linguistics, and their sub-fields. Even within mathematics, there will be users only interested in proofs, while other users are only interested in graphing and modeling. If we allow these communities to form naturally over time, but have a stable plugin architecture by the time they form, then it would be a good idea to start experimenting now before OpenGL becomes a larger headache.

[jsonvillanueva]

And as others have said before regarding manim refactors, time spent on this is time that could have been spent making manim better for the end users, which I don't think this will accomplish. -@WampyCakes

Sounds like something I said and I still believe this. But to address your earlier concerns (and a few others since I read through this as I wrote this):

1. This is an issue. The best I can think to do is to provide a better factory template for manim-plugintemplate for people to fork from. As for indexing, this would be much more manageable once pypa/warehouse allows searching PyPI with a filter for package metadata (i.e. the manim.plugins entry point metadata). PEP 643 which was approved December last year seems relevant.

2. This switch wouldn't serve the end-user... but I also don't think it would cause much confusion. The installation process would be the same for users: pip install manim. This would install manim-core (the directory core stuff would tentatively move to) as well as the stuff that's actually useful for basic constructs (Circle, Graph, Angle, Tex, Text, etc.) which should remain in this repo. I still don't think this is the best usage of our time atm until the manim plugins command has more features we need.

   For example, someone could make a plugin that would be in this curated assortment and then abandon it with no trace. -@naveen521kk

3. (a) I think the plugins structure needs to be strengthened before this can happen. Whether this is done in house or with pluggy, improvements are needed for plugins. Also, we have Click now so subcommand/options for manim plugins are slightly easier to create.

The thing I'd like to avoid most is wasted potential/effort. This is much easier to avoid if this repo stays as a community maintained effort with good CI/workflows, but Manim absolutely needs to be extensible for the many custom Mobjects/Containers that exist in our own personal projects. Some plugins -- if they become widely useful -- should be able to be moved under the organization. See how click falls under the main pallets org, but then has a bunch of click extensions under a separate organization, click-contrib.

3. (b) I genuinely don't think we should split the library into many optional parts to install. It should just be manim and manim-extras, or manim-full as @eulertour described later in this thread. The actual library (outside of WebGL/Jupyter) isn't big enough for this to be necessary.

4. I think improving manim-plugintemplate to be more in line with manim would be beneficial for the future of plugins -- it should more be a more full-fledged example as far as having a very short sphinx/docs page, simple testing, links to the current documentation/testing guidelines in the README, etc.

A lot needs to happen here (and some things at PyPI/warehouse) before this becomes a reality. In the meantime, I think as @leotrs and @eulertour mentioned in some manner, it'd make sense to start putting things in their appropriate modules/directories (e.g. manim.core, manim.plugins <-- that already exists btw) for organization purposes mainly.

Right now manim is a good core animation library and has become so much more than an "Animation engine for explanatory math videos." -@GameDungeon

I agree with you. Manim shouldn't advertise itself with the slogan for explanatory math videos. because it can be used for more. In fact, as @RickyC0626 has alluded to how I intend to use Manim, the user's case isn't always math-oriented. For instance, @kilacoda has created multiple Chemistry videos with their plugin chanim (which still needs to be plugin-ified for Manim btw), but because it has been popularized from Grant's usage in Math initially, we're sort of stuck with the idea of Mobject, and Manim -- but that's just a small branding issue nonetheless.

So, the core part of Manim should contain:

* Renderers

* Scene

* Camera

* base Animation class

* Utils that is used in the above-mentioned things.

-@naveen521kk

This needs more discussion but I think that discussion should happen following @eulertour's talk on OpenGL (if that's still happening) -- or at least until a few more contributors/devs are familiar with OpenGL + Manim. Some things to consider in the switch to OpenGL, how does the core change if at all? Is the hacky Camera class still needed? I agree with @MrMallIronmaker that this switch to OpenGL should account for this refactor.

In looking at Blender's modules (another open source OpenGL based application), I've found our Camera is both strange and lacking. It'd make more sense to think deeply about how OpenGL and Manim should interact; but whiles we're at it, why not take inspiration from Blender as far as what modules they include in what categories (of course, we should not aim to recreate Blender -- we must serve our own niche in animation). For instance, their Camera isn't under Core, it's under Rendering https://docs.blender.org/manual/en/2.93/render/cameras.html. In fact most things we'd consider Core in Manim aren't Core in Blender.

Not to side-track this discussion but from click-contrib, there's a few useful extensions for Manim that would be helpful to look into:

• Code/tab completion for CLI
• Config
• Plugins
• Sphinx-Click

[RickyC0626]

In looking at Blender's modules (another open source OpenGL based application), I've found our Camera is both strange and lacking. It'd make more sense to think deeply about how OpenGL and Manim should interact; but whiles we're at it, why not take inspiration from Blender as far as what modules they include in what categories (of course, we should not aim to recreate Blender -- we must serve our own niche in animation). For instance, their Camera isn't under Core, it's under Rendering https://docs.blender.org/manual/en/2.93/render/cameras.html. In fact most things we'd consider Core in Manim aren't Core in Blender.

The direction that the current manim structure seems to be heading is closely resembling a game engine, like Bevy, and other animation software. So, I would agree that Camera and rendering could be placed in manim.render, Scene could be placed in manim.scene, and text, UI, audio, math utils, etc could be placed in their respective modules.