Moonraker-Timelapse integration

[420treeenthusiast]

im trying to ssh the moonraker timelapse plugin for mainsail and i keep getting -sh: make: not found Seems im missing "make" any idea how to get/install it?

[consp]

-sh: make: not found

Make is missing because the system is not capable of compiling anything and thus does not need make. If you want it you'd need to alter the buildroot configuration in one of the variants.

You can run the install script manually but it assumes systemd is there which is not installed (since it is not needed). Unfortunately the script also looks to rely heavilly on bash, which is also not availble.

It will have to be installed manualy. (copy the timelapse.py to the correct dir, add the timelapse.cfg to be included in printer.cfg)

It will then fail because it needs ffmpeg which is also not installed (and needs to be cross compiled anyway, you might see a pattern here).

I'll have another go at it next week to see if ffmpeg, in this very specific case, does not eat all the available memory. No promisses though as ffmpeg has the tendency to eat all the memory it gets and then some and there is only 128mb available.

[xblax]

I'll have another go at it next week to see if ffmpeg, in this very specific case, does not eat all the available memory. No promisses though as ffmpeg has the tendency to eat all the memory it gets and then some and there is only 128mb available.

I believe this could work. I looked at the plugin source code a few days ago. The only thing that ffmpeg does is to create a h264 encoded mp4 file from the saved JPEG frames. If h264 encoding needs to much memory, it should be possible to modify the plugin to use ffmpeg to just create an MJPEG video of the saved frames without re-encoding.

That encoding happens after the print.

[consp]

That encoding happens after the print.

That;'s what I figured from it. ffmpeg isn't too much of a problem to get working, just hogs memory but if the printer is idle it should be fine. h264 depends a bit on the settings, can be quite lean (just takes a while)

[consp]

Added ffmpeg and libx264/265/h264 to give people some options. See feature/ffmpeg branch.

/root/printer_data/timelapse/images/ is used for images, /root/printer_data/timelapse for timelapses.

timelapse plugin is collected from git and added during post_build, timelapse.cfg is included at all times as it auto-disables if the moonraker config is missing.

This should also solve #84 #74.

Things of note, tested with x264:

• I initially used /tmp for images. This is a bad idea with larger images and larger prints. You will run out of memory. Default is now printer_data/timelapse/images/ but this makes rendering insanely slow.
• ffmpeg is memory and cpu hungry with the default settings, it will eat everything available. DO NOT attempt while printing.
• I've had it render anywhere from a few seconds to about 15 minutes (yes that long). It all depends on how much swap is needed, 15min was 450 frames.
• I have not tested it with klipperscreen only lite, I doubt it will cooperate due to memory constraints.

smem during long render with 456 frames from disk (about halfway during rendering):

4093 root     /usr/bin/ffmpeg -r 30 -i /r    32604    41280    41378    41764

I think the render settings need tweaking to get something properly usable. Any suggestions? Any clue how flashforge stock does it?

[xblax]

@consp Great work, I will give it a try and experiment a bit with the ffmpeg settings! Maybe we can just put the individual images into an MJPEG stream inside an MP4 container without re-encoding. But this would likely make the final video size a bit bigger.

[consp]

But this would likely make the final video size a bit bigger.

The 450 frame x264 one was ~19MB not huge, if people want to get a smaller size re-encoding is also an option outside of the printer. At about 130-150KB/frame (jpg) the mjpeg would be 60-70MB. Maybe we can tweak the snapshot's as well to reduce the size a bit if needed.

If it's not re-encoding the memory use will also drop significantly is my guess but I'm not too well versed in ffmpeg.

[xblax]

Ok, I can confirm that it's painly slow to render the video with default settings. Probably because lots of other processes need to be swapped. And it does sometimes trigger Klipper shutdown due to timing error, even though no print is running.

I printed a calibration cube, that gives 99 frames.

This is encoding with the default settings:

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 2 -g 5 -crf 23 -vcodec libx264 -pix_fmt yuv420p -an test01.mp4
140.57user 18.34system 3:41.57elapsed 71%CPU (0avgtext+0avgdata 60840maxresident)k
0inputs+0outputs (29580major+387393minor)pagefaults 0swaps

Limiting to one thread, already a bit faster because we are memory constrained, not CPU limited.

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 5 -crf 23 -vcodec libx264 -pix_fmt yuv420p -an test02.mp4
123.29user 3.47system 2:10.08elapsed 97%CPU (0avgtext+0avgdata 61508maxresident)k
0inputs+0outputs (308major+76434minor)pagefaults 0swaps

Using ultrafast preset, already much better. This avoid b-frames and therefore needs less memory.

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 5 -crf 23 -preset ultrafast -vcodec libx264 -pix_fmt yuv420p -an test03.mp4
24.60user 0.46system 0:28.62elapsed 87%CPU (0avgtext+0avgdata 38944maxresident)k
0inputs+0outputs (0major+8680minor)pagefaults 0swaps

Increasing CRF to 28. Makes the file smaller, but not much advantage otherwise.

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 5 -crf 28 -preset ultrafast -vcodec libx264 -pix_fmt yuv420p -an test04.mp4
22.80user 0.27system 0:25.02elapsed 92%CPU (0avgtext+0avgdata 38708maxresident)k
0inputs+0outputs (27major+10182minor)pagefaults 0swaps

Increasing the group size (less key frames). Only slight reduction in final file size:

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 30 -crf 28 -preset ultrafast -vcodec libx264 -pix_fmt yuv420p -an test05.mp4
23.38user 0.25system 0:28.55elapsed 82%CPU (0avgtext+0avgdata 38932maxresident)k
0inputs+0outputs (116major+8202minor)pagefaults 0swaps

Using constant bitrate instead of constant quality. No improvement.

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 30 -b:v 8000k -preset ultrafast -vcodec libx264 -pix_fmt yuv420p -an test06.mp4
23.50user 0.35system 0:26.66elapsed 89%CPU (0avgtext+0avgdata 39056maxresident)k
0inputs+0outputs (42major+10877minor)pagefaults 0swaps

No encoding, just mux MJEPG into MP4. That is very fast, but still takes some memory.

0.08user 0.38system 0:02.86elapsed 16%CPU (0avgtext+0avgdata 18640maxresident)k
0inputs+0outputs (3major+3108minor)pagefaults 0swaps
/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -c copy -an test07.mp4

Rencoding the JPEG files. Output still looks good, but takes surprisingly much memory for the task.

/usr/bin/time /usr/bin/ffmpeg -r 30 -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -c:v mjpeg -q:v 10 -an test08.mp4
14.68user 0.29system 0:17.57elapsed 85%CPU (0avgtext+0avgdata 31396maxresident)k
0inputs+0outputs (2major+5527minor)pagefaults 0swaps

These are the resulting files:

-rw-r--r--    1 root     root        3.2M Apr  8 22:06 test01.mp4
-rw-r--r--    1 root     root        3.2M Apr  8 22:10 test02.mp4
-rw-r--r--    1 root     root        6.3M Apr  8 22:20 test03.mp4
-rw-r--r--    1 root     root        3.5M Apr  8 22:17 test04.mp4
-rw-r--r--    1 root     root        3.0M Apr  8 22:24 test05.mp4
-rw-r--r--    1 root     root        3.4M Apr  8 22:36 test06.mp4
-rw-r--r--    1 root     root       18.2M Apr  8 23:12 test07.mp4
-rw-r--r--    1 root     root        7.2M Apr  8 23:22 test08.mp4

Based on these tests I would either recommend to not reencode the JEPGs or to use optimized H264 settings: /usr/bin/time /usr/bin/ffmpeg -r 30 -c mjpeg -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 30 -crf 23 -preset ultrafast -vcodec libx264 -pix_fmt yuv420p -an test.mp4

I attached the video. And yes, my LEDs are a bit overpowered :smile: I think that was at 80% LED intensity and flickers a bit due to PWM.

https://github.com/xblax/flashforge_ad5m_klipper_mod/assets/9802544/95424b20-1cbc-479e-85d7-a4f6392d1f8c

[xblax]

There are more things to look at:

• timelapse.cfg is missing, probably should also come from git
• trying other codes. I think vp8, vp9, av1, webp could be worth trying
• stripping down the ffmpeg build to the bare minimum after finding optimal settings - that could help a bit since the shared libraries are quite big (libavcodec is 10M)

[xblax]

I renamed the ticket to make more sense. Other points:

• default configuration of the plugin (should probably be disabled in the options by default, can be enabled in Mainsail/Fluidd)
• make sure it makes no issues when no camera is attached for non-pro users without camera mod

The time util that I used to benchmark ffmpeg was not included, I added it on master.

[420treeenthusiast]

Ok so I’m new to all this so be patient, I don’t think this is the Timelapse I was thinking of. What I was wanting to do was the Timelapse thru Mainsail/fluid like a beagle cam can do. Where basically it parks the print head at the same spot after every layer, takes a snapshot then resumes printing the next layer and so on.

[420treeenthusiast]

It seems to be a pretty common mainsail/fluid “plugin”

[xblax]

Ok so I’m new to all this so be patient, I don’t think this is the Timelapse I was thinking of. What I was wanting to do was the Timelapse thru Mainsail/fluid like a beagle cam can do. Where basically it parks the print head at the same spot after every layer, takes a snapshot then resumes printing the next layer and so on.

This is exactly what it is but I disabled print head parking in the options (could be enabled easily). I wanted to test if I observe any slowdowns due to the timelapse plugin, fortunately not but also did not watch it closely.

I consider parking the print head at each layer impractical for actual prints that are not just for show, since it can produce print artifacts.

But the timelapse feature is also useful for long unattended prints, to see where the print started fail.

[420treeenthusiast]

Perfect! Yea I agree, and I’m sure it takes 4x as long lol. Was mainly looking to use it for instagram content, won’t be a regular occurrence.

[consp]

• timelapse.cfg is missing

Forgot to push it ... it's in printer_config now

stripping down the ffmpeg build to the bare minimum after finding optimal settings - that could help a bit since the shared libraries are quite big (libavcodec is 10M)

Probably a good idea, after it's settled what to use.

[xblax]

Ok, after lot of testing, I think I found the optimal settings for timelapse encoding.

/usr/bin/time /usr/bin/ffmpeg -r 30 -c mjpeg -i /root/printer_data/timelapse/images/frame%6d.jpg -threads 1 -g 1 -crf 23 -preset veryfast -tune zerolatency -vcodec libx264 -pix_fmt yuv420p -an test_h264.mp4

gives lowest memory usage I could achieve and it's also a good tradeoff between speed and final file size. I thinks it's running quite decent. This example is for 99 frames (calibration cube at 0.2 layer height):

/usr/bin/time benchmark

37.28user 0.38system 0:39.66elapsed 94%CPU (0avgtext+0avgdata 23980maxresident)k
0inputs+0outputs (0major+5744minor)pagefaults 0swaps

smem during rendering:

PID User     Command                         Swap      USS      PSS      RSS
2631 root     /usr/bin/ffmpeg -r 30 -c mj        0    18316    18473    19804

Memory usage should be independed from the video length, and full print of 220mm should be rendered in less then 10min.

To achieve this I also had to strip down ffmpeg to the bare minimum. That alone reduces memory usage by 10M! I committed the optimized buildroot config to feature branch: https://github.com/xblax/flashforge_ad5m_klipper_mod/commit/d1f79422adb84f93206b30573650ea54f6c56931

Basically all other encoder options are worse than x264. Just muxing the JPEGs without encoding does not work, because Mainsail and Fluidd can't display the file in the browser, although it works for proper players like VLC.

The optimized settings for AD5M must be patched into timelapse.py

[consp]

Ok, after lot of testing, I think I found the optimal settings for timelapse encoding.

Oof, that must have been quite some work. Good to see the memory reduced, should even be usable with klipperscreen.

Memory usage should be independed from the video length, and full print of 220mm should be rendered in less then 10min.

We should definetly mention that so people do not complain it takes too long or report "bugs".

The optimized settings for AD5M must be patched into timelapse.py

Might be better to just staticly include it then and update if needed. It's not like it's a high-change product.

[consp]

Was just digging around in the original firmware (to check the changes in the update).

 _%03d.jpg -mf w=460:h=560:fps=10:type=jpeg -ovc lavc -o

Settings used, it uses mplayer's mencoder.

The optimized settings for AD5M must be patched into timelapse.py

I'll edit it in and add the timelapse.py as a pre-installed file in the post buildroot script to be copied. It's a simple script, shouldn't be too much of an issue to include it as static instread of pulling it from github.

[xblax]

Isn't mplayer mencoder just using ffmpeg internally, or is that really a separate thing? I agree on just including patched timelapse.py rather then pulling it from Git dynamically.

[consp]

Isn't mplayer mencoder just using ffmpeg internally

Did grep -i ffmpeg mencoder and it spewed out a lot so that seems accurate so we can ignore it. Just the settings for reference then.

[consp]

Pushed to branch, most changes are in the config, some in the file. Added file to prebuilt/moonraker-plugins/timelapse. 3671f9fbed76aa989f4437eeff6cb86df24377fc

Can be closed if merged I guess.

[xblax]

It's looking good in it's current state. Render was successful even with klipperscreen running: https://github.com/xblax/flashforge_ad5m_klipper_mod/assets/9802544/26434d98-6de2-4a9d-8a86-b4451ba420ef (at the beginning you see the PWM flickering of my LED)

The feature is on master now. We can close this issue.