search

ManimCommunity / manim

A community-maintained Python framework for creating mathematical animations.
https://www.manim.community
MIT License
25.19k stars 1.76k forks source link

Manim crashes with `ValueError: Cell is empty`. Weird dependency on dead code. #2572

Open Volker-Weissmann opened 2 years ago

Volker-Weissmann commented 2 years ago

Description of bug / unexpected behavior

Manim crashes with ValueError: Cell is empty if the right code is executed. There is a weird dependency on dead code: Removing

if False:
    print(weird_var)

makes the problem go away.

Expected behavior

No crash.

How to reproduce the issue

Code for reproducing the problem ```py class Bug(Scene): def construct(self): curve = VGroup() def get_curve(): if False: print(weird_var) return curve self.add(always_redraw(get_curve)) self.wait() weird_var = 19 ```

Additional media files

No media files are produced

Logs

Terminal output ``` Manim Community v0.14.0 [02/24/22 18:34:11] DEBUG Animation with empty animation.py:165 mobject DEBUG Hashing ... hashing.py:344 ╭───────────────── Traceback (most recent call last) ─────────────────╮ │ │ │ /usr/lib/python3.10/site-packages/manim/cli/render/commands.py:139 │ │ in render │ │ │ │ 136 │ │ for SceneClass in scene_classes_from_file(file): │ │ 137 │ │ │ try: │ │ 138 │ │ │ │ scene = SceneClass() │ │ ❱ 139 │ │ │ │ scene.render() │ │ 140 │ │ │ except Exception: │ │ 141 │ │ │ │ error_console.print_exception() │ │ 142 │ │ │ │ sys.exit(1) │ │ /usr/lib/python3.10/site-packages/manim/scene/scene.py:219 in │ │ render │ │ │ │ 216 │ │ """ │ │ 217 │ │ self.setup() │ │ 218 │ │ try: │ │ ❱ 219 │ │ │ self.construct() │ │ 220 │ │ except EndSceneEarlyException: │ │ 221 │ │ │ pass │ │ 222 │ │ except RerunSceneException as e: │ │ │ │ /home/volker/Sync/itp2/masterarbeit-praesi/bug.py:16 in construct │ │ │ │ 13 │ │ │ return curve │ │ 14 │ │ │ │ 15 │ │ self.add(always_redraw(get_curve)) │ │ ❱ 16 │ │ self.wait() │ │ 17 │ │ │ │ 18 │ │ weird_var = 19 │ │ 19 │ │ │ │ /usr/lib/python3.10/site-packages/manim/scene/scene.py:953 in wait │ │ │ │ 950 │ │ │ ) │ │ 951 │ │ │ 952 │ def wait(self, duration=DEFAULT_WAIT_TIME, stop_conditio │ │ ❱ 953 │ │ self.play(Wait(run_time=duration, stop_condition=sto │ │ 954 │ │ │ 955 │ def wait_until(self, stop_condition, max_time=60): │ │ 956 │ │ """ │ │ │ │ /usr/lib/python3.10/site-packages/manim/scene/scene.py:937 in play │ │ │ │ 934 │ │ │ │ 935 │ │ """ │ │ 936 │ │ start_time = self.renderer.time │ │ ❱ 937 │ │ self.renderer.play(self, *args, **kwargs) │ │ 938 │ │ run_time = self.renderer.time - start_time │ │ 939 │ │ if subcaption: │ │ 940 │ │ │ if subcaption_duration is None: │ │ │ │ /usr/lib/python3.10/site-packages/manim/renderer/cairo_renderer.py: │ │ 67 in play │ │ │ │ 64 │ │ │ │ logger.info("Caching disabled.") │ │ 65 │ │ │ │ hash_current_animation = f"uncached_{self.num │ │ 66 │ │ │ else: │ │ ❱ 67 │ │ │ │ hash_current_animation = get_hash_from_play_c │ │ 68 │ │ │ │ │ scene, │ │ 69 │ │ │ │ │ self.camera, │ │ 70 │ │ │ │ │ scene.animations, │ │ │ │ /usr/lib/python3.10/site-packages/manim/utils/hashing.py:349 in │ │ get_hash_from_play_call │ │ │ │ 346 │ _Memoizer.mark_as_processed(scene_object) │ │ 347 │ camera_json = get_json(camera_object) │ │ 348 │ animations_list_json = [get_json(x) for x in sorted(anima │ │ ❱ 349 │ current_mobjects_list_json = [get_json(x) for x in curren │ │ 350 │ hash_camera, hash_animations, hash_current_mobjects = ( │ │ 351 │ │ zlib.crc32(repr(json_val).encode()) │ │ 352 │ │ for json_val in [camera_json, animations_list_json, c │ │ │ │ /usr/lib/python3.10/site-packages/manim/utils/hashing.py:349 in │ │ │ │ │ │ 346 │ _Memoizer.mark_as_processed(scene_object) │ │ 347 │ camera_json = get_json(camera_object) │ │ 348 │ animations_list_json = [get_json(x) for x in sorted(anima │ │ ❱ 349 │ current_mobjects_list_json = [get_json(x) for x in curren │ │ 350 │ hash_camera, hash_animations, hash_current_mobjects = ( │ │ 351 │ │ zlib.crc32(repr(json_val).encode()) │ │ 352 │ │ for json_val in [camera_json, animations_list_json, c │ │ │ │ /usr/lib/python3.10/site-packages/manim/utils/hashing.py:314 in │ │ get_json │ │ │ │ 311 │ :class:`str` │ │ 312 │ │ The flattened object │ │ 313 │ """ │ │ ❱ 314 │ return json.dumps(obj, cls=_CustomEncoder) │ │ 315 │ │ 316 │ │ 317 def get_hash_from_play_call( │ │ │ │ /usr/lib/python3.10/json/__init__.py:238 in dumps │ │ │ │ 235 │ │ skipkeys=skipkeys, ensure_ascii=ensure_ascii, │ │ 236 │ │ check_circular=check_circular, allow_nan=allow_nan, i │ │ 237 │ │ separators=separators, default=default, sort_keys=sor │ │ ❱ 238 │ │ **kw).encode(obj) │ │ 239 │ │ 240 │ │ 241 _default_decoder = JSONDecoder(object_hook=None, object_pairs │ │ │ │ /usr/lib/python3.10/site-packages/manim/utils/hashing.py:298 in │ │ encode │ │ │ │ 295 │ │ _Memoizer.mark_as_processed(obj) │ │ 296 │ │ if isinstance(obj, (dict, list, tuple)): │ │ 297 │ │ │ return super().encode(self._cleaned_iterable(obj) │ │ ❱ 298 │ │ return super().encode(obj) │ │ 299 │ │ 300 │ │ 301 def get_json(obj): │ │ │ │ /usr/lib/python3.10/json/encoder.py:199 in encode │ │ │ │ 196 │ │ # This doesn't pass the iterator directly to ''.join( │ │ 197 │ │ # exceptions aren't as detailed. The list call shoul │ │ 198 │ │ # equivalent to the PySequence_Fast that ''.join() wo │ │ ❱ 199 │ │ chunks = self.iterencode(o, _one_shot=True) │ │ 200 │ │ if not isinstance(chunks, (list, tuple)): │ │ 201 │ │ │ chunks = list(chunks) │ │ 202 │ │ return ''.join(chunks) │ │ │ │ /usr/lib/python3.10/json/encoder.py:257 in iterencode │ │ │ │ 254 │ │ │ │ markers, self.default, _encoder, self.indent, │ │ 255 │ │ │ │ self.key_separator, self.item_separator, self │ │ 256 │ │ │ │ self.skipkeys, _one_shot) │ │ ❱ 257 │ │ return _iterencode(o, 0) │ │ 258 │ │ 259 def _make_iterencode(markers, _default, _encoder, _indent, _f │ │ 260 │ │ _key_separator, _item_separator, _sort_keys, _skipkey │ │ │ │ /usr/lib/python3.10/site-packages/manim/utils/hashing.py:195 in │ │ default │ │ │ │ 192 │ │ │ obj, │ │ 193 │ │ │ (MethodType, FunctionType), │ │ 194 │ │ ): │ │ ❱ 195 │ │ │ cvars = inspect.getclosurevars(obj) │ │ 196 │ │ │ cvardict = {**copy.copy(cvars.globals), **copy.co │ │ 197 │ │ │ for i in list(cvardict): │ │ 198 │ │ │ │ # NOTE : All module types objects are removed │ │ │ │ /usr/lib/python3.10/inspect.py:1575 in getclosurevars │ │ │ │ 1572 │ if func.__closure__ is None: │ │ 1573 │ │ nonlocal_vars = {} │ │ 1574 │ else: │ │ ❱ 1575 │ │ nonlocal_vars = { │ │ 1576 │ │ │ var : cell.cell_contents │ │ 1577 │ │ │ for var, cell in zip(code.co_freevars, func.__cl │ │ 1578 │ } │ │ │ │ /usr/lib/python3.10/inspect.py:1576 in │ │ │ │ 1573 │ │ nonlocal_vars = {} │ │ 1574 │ else: │ │ 1575 │ │ nonlocal_vars = { │ │ ❱ 1576 │ │ │ var : cell.cell_contents │ │ 1577 │ │ │ for var, cell in zip(code.co_freevars, func.__cl │ │ 1578 │ } │ │ 1579 │ ╰─────────────────────────────────────────────────────────────────────╯ ValueError: Cell is empty ```

System specifications

System Details - OS (with version, e.g., Windows 10 v2004 or macOS 10.15 (Catalina)): ArchLinux, fully upgraded - RAM: 8GB - Python version (`python/py/python3 --version`): Python 3.10.2 - Installed modules (provide output from `pip list`): ``` Package Version ----------------------------- ----------------- absl-py 1.0.0 alabaster 0.7.12 anytree 2.8.0 appdirs 1.4.4 argh 0.26.2 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 asn1crypto 1.4.0 astor 0.8.1 astroid 2.9.3 asttokens 2.0.5 astunparse 1.6.3 async-generator 1.10 attrs 21.4.0 Babel 2.9.1 backcall 0.2.0 Beaker 1.11.0 beautifulsoup4 4.10.0 black 21.12b0 bleach 4.1.0 bottle 0.12.19 breezy 3.2.1 Brlapi 0.8.3 btrfsutil 5.16.2 CacheControl 0.12.6 cachetools 5.0.0 ceph 1.0.0 ceph-volume 1.0.0 cephfs 2.0.0 cephfs-shell 0.0.1 certifi 2021.10.8 cffi 1.15.0 chardet 4.0.0 click 8.0.3 click-default-group 1.2.2 cloup 0.13.0 colorama 0.4.4 colour 0.1.5 commonmark 0.9.1 configobj 5.1.0.dev0 contextlib2 0.6.0.post1 cryptography 36.0.1 cssselect 1.1.0 cupshelpers 1.0 cvxopt 1.2.8 cycler 0.10.0 cypari2 2.1.2 cysignals 1.11.2 Cython 3.0a0 debugpy 1.5.1 decorator 5.1.1 defusedxml 0.7.1 deprecation 2.1.0 distlib 0.3.4 distro 1.6.0 dnspython 2.2.0 docutils 0.16 dulwich 0.20.32 entrypoints 0.4 evdev 1.4.0 executing 0.8.2 fasteners 0.16.3 fastimport 0.9.14 feedparser 6.0.2 flatbuffers 2.0.6 flowblade 2.8.0.3 fonttools 4.29.1 fpylll 0.5.6 future 0.18.2 gast 0.3.3 glcontext 2.3.4 gmpy2 2.1.2 google-api-core 2.5.0 google-auth 1.34.0 google-auth-oauthlib 0.4.4 google-pasta 0.2.0 googleapis-common-protos 1.53.0 gpg 1.17.0 grpcio 1.43.2 grpcio-tools 1.43.0 h5py 3.6.0 html5lib 1.1 idna 3.3 imagesize 1.3.0 importlib-metadata 4.8.1 ipykernel 6.9.1 ipython 8.0.1 ipython-genutils 0.2.0 ipywidgets 7.6.5 isort 5.10.1 isosurfaces 0.1.0 jedi 0.18.1 Jinja2 3.0.3 jsonschema 3.2.0 jupyter-client 7.1.2 jupyter-console 6.4.0 jupyter-core 4.9.1 jupyterlab-pygments 0.1.2 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.2 kiwisolver 1.3.2 lazy-object-proxy 1.7.1 lensfun 0.3.95 libfdt 1.6.1 listparser 0.18 lit 13.0.1.dev0 louis 3.20.0 lrcalc 2.0.0 lutris 0.5.9.1 lxml 4.7.1 Mako 1.1.6 mallard-ducktype 1.0.2 manim 0.14.0 ManimPango 0.4.0.post2 mapbox-earcut 1.0.0 Markdown 3.3.6 MarkupSafe 2.0.1 matplotlib 3.5.1 matplotlib-inline 0.1.3 mccabe 0.7.0 meld 3.20.4 memory-allocator 0.1.2 mercurial 6.0.3 meson 0.61.2 mistune 0.8.4 moderngl 5.7.0 moderngl-window 2.4.1 monotonic 1.6 more-itertools 8.10.0 mpi4py 3.1.3 mpmath 1.2.1 msgpack 1.0.3 multipledispatch 0.6.0 mypy 0.931 mypy-extensions 0.4.3 nbclient 0.5.11 nbconvert 6.4.2 nbformat 5.1.3 nest-asyncio 1.5.4 netsnmp-python 1.0a1 networkx 2.6.3 notebook 6.4.8 numpy 1.22.2 oauthlib 3.1.1 opt-einsum 3.3.0 ordered-set 4.0.2 packaging 20.9 pandas 1.4.1 pandocfilters 1.5.0 parso 0.8.2 patch-ng 1.17.4 pathspec 0.9.0 pathtools 0.1.2 patiencediff 0.2.2 pep517 0.12.0 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.0.1 pip 20.3.4 platformdirs 2.4.1 pluginbase 1.0.0 ply 3.11 powerline-status 2.8.2 pplpy 0.8.7 primecountpy 0.1.0 progress 1.6 prometheus-client 0.12.0 prompt-toolkit 3.0.28 protobuf 3.19.4 psutil 5.9.0 ptyprocess 0.7.0 pure-eval 0.2.2 pwquality 1.4.4 pyasn1 0.4.8 pyasn1-modules 0.2.8 pybind11 2.9.1 pycairo 1.20.1 pycodestyle 2.8.0 pycparser 2.21 pycups 2.0.1 pycurl 7.44.1 pydub 0.25.1 pyflakes 2.4.0 pyglet 1.5.21 Pygments 2.11.2 PyGObject 3.42.0 PyJWT 2.2.0 pylint 2.12.2 pyOpenSSL 21.0.0 pyparsing 3.0.0 PyQt5 5.15.6 PyQt5-sip 12.9.1 pyrr 0.10.3 pyrsistent 0.18.1 PySocks 1.7.1 python-dateutil 2.8.2 python-distutils-extra 2.39 python-xlib 0.31 pytoml 0.1.21 pytools 2022.1 pytz 2021.3 pytz-deprecation-shim 0.1.0.post0 pyxdg 0.27 PyYAML 6.0 pyzmq 22.3.0 rados 2.0.0 rbd 2.0.0 readability-lxml 0.8.1 Reflector 2021.11.20.2.41.3 requests 2.27.1 requests-oauthlib 1.3.1 resolvelib 0.5.5 retrying 1.3.3 rgw 2.0.0 rich 11.2.0 rpy2 3.4.5 rsa 4.8 scipy 1.8.0 screeninfo 0.7 Send2Trash 1.8.0 setproctitle 1.2.2 setuptools 59.3.0 setuptools-scm 6.4.2 sgmllib3k 1.0.0 simplejson 3.17.6 six 1.16.0 skia-pathops 0.7.2 snowballstemmer 2.2.0 soupsieve 2.3.1 Sphinx 4.4.0 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 srt 3.5.0 stack-data 0.2.0 sympy 1.9 TBB 0.2 team 1.0 tensorboard 2.8.0 tensorboard-data-server 0.7.0a0 tensorboard-plugin-wit 1.8.1 tensorflow-estimator 2.7.0 termcolor 1.1.0 terminado 0.13.1 testpath 0.5.0 toml 0.10.2 tomli 2.0.0 torbrowser-launcher 0.3.5 tornado 6.1 tqdm 4.62.3 traitlets 5.1.1 trash-cli 0.21.10.24 typed-ast 1.5.2 typing-extensions 4.1.0 tzdata 2021.5 tzlocal 4.0.1 urllib3 1.26.8 validate 5.1.0.dev0 watchdog 0.10.7 wcwidth 0.2.5 webencodings 0.5.1 Werkzeug 2.0.2 wheel 0.37.1 widgetsnbextension 3.5.2 wrapt 1.13.2 xmltodict 0.12.0 xonsh 0.11.0 zipp 3.7.0 ```
FFMPEG Output of `ffmpeg -version`: ``` ffmpeg version n5.0 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 11.2.0 (GCC) configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3 libavutil 57. 17.100 / 57. 17.100 libavcodec 59. 18.100 / 59. 18.100 libavformat 59. 16.100 / 59. 16.100 libavdevice 59. 4.100 / 59. 4.100 libavfilter 8. 24.100 / 8. 24.100 libswscale 6. 4.100 / 6. 4.100 libswresample 4. 3.100 / 4. 3.100 libpostproc 56. 3.100 / 56. 3.100 ```

Additional comments

behackl commented 2 years ago

Thank you for your report! I can at least offer an explanation; not sure how easy it would be to actually fix this.

Manim's caching system inspects your scene code before and between play calls (which self.wait effectively also is). In the block before the first animation, weird_var occurs, but does not have an assigned value, which is a problem: Manim wants to hash all variables and values in order for the caching system to decide whether a already rendered animation can be re-used or has to be rendered again.

Two workarounds to avoid this: either disable the caching system by rendering the scene with --disable_caching, or make sure variables have a value assigned before they are referenced otherwise in the code (setting weird_var = None at the beginning of construct also resolves the problem).

Volker-Weissmann commented 2 years ago
  1. I can confirm that --disable_caching fixes it, both in the toy code I posted and my real code.
  2. weird_var = None fixes it in the toy code. In a few hours, I will report if it fixes it in my real code.
  3. Slightly OT: You said that manim caches the python sourcecode/bytecode. I want something similar for an unrelated project of mine. Can you give me some hints where to start reading if I want to build a similar caching system. Or are you aware of any 3rd party libs that do that?
Volker-Weissmann commented 2 years ago

Update: A few var=None in the beginning fixes it in my real code. But I also found that in some cases, referencing a variable that will only be defined later does not break it. In fact, I found a case, where adding/removing self.add(Text("Hello")) made the difference between the code working or the code crashing with Cell is empty.

behackl commented 2 years ago

Can you give me some hints where to start reading if I want to build a similar caching system. Or are you aware of any 3rd party libs that do that?

Not sure about 3rd party libraries -- for Manim, the main part of the caching system is implemented here: https://github.com/ManimCommunity/manim/blob/main/manim/utils/hashing.py