OctoLapse not taking stabilised videos since last update.

[Ruhel786]

Since that latest OctoLapse update, it will no longer take stabilised videos. I have tried all troubleshooting steps including some fixes mentioned in #149 as this appears to be a similar issue. I have also reinstalled OctoLapse and set the configuration for Prusa MK3S defaults. I have made no changes between the 2 videos posted below other than updating OctoLapse.

Version of Octolapse

Octolapse Version: v0.4.2

Version of OctoPrint

OctoPrint Version: V1.8.7

When you ran into the problem, did you have diagnostic logging enabled?

Diagnostic Logging was Enabled: YES

What were you doing when the problem occurred

1. Print as normal with OctoLapse
2. Normal Timelapse in OctoPrint is switched off
3. .

What should have happened?

OctoLapse should have made a perfectly stabilised timelapse.

What happened instead?

OctoLapse made a jittery unstabilised timelapse.

Operating System running OctoPrint and Octolapse

OS Name: OctoPi Os Version: v1.0.0

Printer model & used firmware incl. version

Printer Model: Prusa i3 MK3S Printer Firmware Version: v3.11.0

Link to the gcode file you were printing when the problem occurred

Link to Gcode File: https://gist.github.com/Ruhel786/2e3c7d6c31c5e5a3d081739d9359acf8

Link to settings.json

Link to settings.json with all passwords removed: https://gist.github.com/Ruhel786/31dbed9bbcab3353bc59bf66fabd979c

Link to plugin_octolapse.log

Link to plugin_octolapse.log: https://gist.github.com/Ruhel786/97d05abd9eef10c6ddcbb6595bdf19d4

Link to octoprint.log

Link to octoprint.log: https://gist.github.com/Ruhel786/3a49fcfe8d0650e6e74fc5bbe6201c46

Screenshots and/or videos of the problem:

Screenshot/Video Links: Normal behaviour: https://www.youtube.com/watch?v=Lv5dd3YZh0s New jittery behavior: https://www.youtube.com/watch?v=OLKKtJmSm1o

Please consider becoming a patron

If you like this project, please support my work by becoming a patron, and consider adding a 'star' to the repository. It takes a lot of time and effort to maintain the project and respond to issues. The cost of test prints, software, cameras, printer parts, etc. can quickly add up, so every bit helps.

You can find various videos and tutorials by subscribing to my Youtube channel. You can also follow me on Twitter.

[FormerLurker]

Can you switch your logging profile in Octolapse to 'Log Everything', then click on the 'edit' pen in 'manage log files' and click 'Clear All Logs'. Then print 4 or 5 layers (enabling test mode is fine if you don't want to actually print anything), edit the logging profile again, and download the log via the 'download log' button?

Can you also record the Python version for me (lower left hand corner, probably 3.9.2 given that you are running octopi 1.0.0, but worth checking)?

I already reviewed the settings, and things look good. Just need to figure out what exactly is going on here.

[Ruhel786]

Can you switch your logging profile in Octolapse to 'Log Everything', then click on the 'edit' pen in 'manage log files' and click 'Clear All Logs'. Then print 4 or 5 layers (enabling test mode is fine if you don't want to actually print anything), edit the logging profile again, and download the log via the 'download log' button?

Can you also record the Python version for me (lower left hand corner, probably 3.9.2 given that you are running octopi 1.0.0, but worth checking)?

I already reviewed the settings, and things look good. Just need to figure out what exactly is going on here.

Hi, The Python is 3.9.2. I initially thought I may have overloaded the Pi but the resource monitor shows its fine.

\https://gist.github.com/Ruhel786/8e4008e6e21d97c0545b23e09541d645

plugin_octolapse (1).log Thank you.

[FormerLurker]

Something must have not worked with the 'Log Everything ' profile. Let me test it locally first, then I will get back to you.

[Ruhel786]

I've tried again multiple times but Log Everything only makes a 3kb file. All the stability options and trigger options have the same jittery effect.

[FormerLurker]

Ok, I fixed the logging. I have been pulling python 2 support to save a bit of my sanity, and missed a change necessary to get a singleton class to work (add metaclass to class definition instead of setting it manually). The configurator was basically wiping everything out and starting over each time a logger was requested. Simple fix, but bad bug, thanks for helping me find/fix it!

So, can you update Octolapse from this link using the plugin manager (use get more, then paste link into the '...from url' box): https://github.com/FormerLurker/Octolapse/archive/refs/heads/devel.zip

Then please redo the steps above, making sure you clear the logs right before you print. Thank you!!!!

[Ruhel786]

@FormerLurker Hi. Here's the file. https://gist.github.com/Ruhel786/185db5e9f0ca103924603db91bcbaa7b Thanks

[FormerLurker]

Well, I reviewed a significant chunk of your log, and couldn't see anything obvious. The first thing I checked was to make sure the stabilization coordinates were reasonable. It looks like you were using the 'Animated - Printing' stabilization, as the Y coordinate remained constant, and the X coordinate moved back and forth. Then I imported your settings and printed your sample gcode. My timelapse came out stabilized.

Typically when there are stabilization issues like this, it's a problem with the 3D printer's handling of the M400 command (wait for moves to finish), so I looked signs of this within the logs. Again, nothing obvious exactly (there is around a 1-3 second delay between sending M400 and receiving a response from the next command), though the delay seems a bit low to me depending on the queue size of your firmware. I'm running another test of your gcode and settings after pushing the new logging code to my instance and will try to find obvious differences between the time it takes for the M400 command to run.

The one thing I cannot check is any potential delay in the camera frame. For instance, if i request a snapshot from your camera, will I always get the latest frame, or could I be getting a cached frame somehow (say from 1 second ago).

Will let you know what I find regarding the m400 delays, but I have an Mk2.5, not an Mk3. In the meanwhile, can you do the following:

1. What firmware are you running on your Mk3? Did you recently update it?
2. Can you post your octopi.txt file and give me some info about your camera? Is it possible that something has changed with your camera setup at all?

I'm kind of grasping at straws here, I'll admit, but will continue to investigate.

[Ruhel786]

1. What firmware are you running on your Mk3? Did you recently update it?
2. Can you post your octopi.txt file and give me some info about your camera? Is it possible that something has changed with your camera setup at all?

Hi. Thanks for getting back to me. I am running MK3S FW v3.11.0 (the latest FW has a bug which causes a thermal anomaly error if the fan has been changed from the original and I'm running a Sunon fan.) Have not updated it recently and I'm only one FW behind the latest release I think.

I did recently change the camera cable to a longer one but the prints were working fine with it. Last night I went back to the original cable when I did my test mode print for the Log but the TimeLapse still was jittery. The only change I can think of is the OctoLapse update. It's a Raspberry Pi Camera V3.

My OctoPi.txt: https://gist.github.com/Ruhel786/b480830e114e13b80421d5bd4a670cd7

[FormerLurker]

I compared the logs between the first three frames of your timelapse and mine, and the M400 delays are within around 100ms, so I don't think that's the issue. Nothing in the log looks amiss at all!?

Would you mind disabling your other plugins and see if the problem persists? I still definitely want to know about your firmware and camera setup.

[FormerLurker]

There is some missing config in your OctoPi.txt, specifically the webcam setup. I know it's difficult to get it all copied out. The easiest way I know how to do it is to use an SFTP client (like FileZilla or something) and just download the whole file.

[Ruhel786]

octopi.txt Ive used FileZilla but the contents look the same. This OctoPi version was made by Foosal to get the new RasPi V3 camera working. So that might be why the settings are not there? Ive tried to get rid of all variences, Ive stopped using the Alpha Prusa Slicer to see it that was causing an issue.

[FormerLurker]

OK, I reviewed it again, and noticed this message:

### IMPORTANT: Looking for the camera settings? Those are now located
### in the camera-streamer directory! libcamera.conf to configure the Raspberry Pi
### camera and usb-default.conf to configure USB cameras. Read more about it here:
### https://faq.octoprint.org/camera-streamer-config

This version looks very new (the entry in the FAQ is 7 days old!), though we are both running octopi 1.0.0. I just reflashed prior to the last Octolapse (like 2 weeks ago) release, and your octopi.txt is a lot different than mine. Did you completely reflash your SD or something recently? I'm pretty confused, since it looks like you are using a different camera setup entirely. If this is true, is that 'smooth' timelapse above from before a reflash or update or something? Also, were can I get the octopi version you are using (with a refactored octopi.txt file)?

I'm just super confused.

FYI, this has nothing to do with your slicer. I printed the file you posted above, and it worked just fine.

[Ruhel786]

Sorry mate. Yes it says OctoPi 1.0.0 but I'm sure its a DEV build by Foosel as she was trying out some new camera stacks. I downloaded the version in Jan and haven't updated it since. I have another memory card spare so might try and do a clean install. https://github.com/OctoPrint/OctoPi-UpToDate/issues/2

[Ruhel786]

Just seen this comment in the OctoPi issues pages. Im doing a full wipe and install. Will let you know how it goes.

[FormerLurker]

You could try adding the --camera-force_active flag and see if that helps first. See this issue: https://github.com/ayufan/camera-streamer/issues/44

I hope that is the correct repository!!

[Ruhel786]

You could try adding the --camera-force_active flag and see if that helps first. See this issue: ayufan/camera-streamer#44

I hope that is the correct repository!!

Where would I add this flag?? Thanks

[FormerLurker]

Check out this link: https://community.octoprint.org/t/camera-streamer-configuration-on-octopi-1-0-0/49950

I can't answer more than that at the moment, since I've not messed with this yet. I'm currently rebuilding my local network (my switch died), but I'll try to take a look tomorrow. If you figure it out before then, please let me know what you did :)

[Ruhel786]

So I did a completely fresh install but I did restore from my Octoprint backup. The stuttering issue was present straight away. I turned off all the plugins with no effect on the issue. (only saw your plugin comment this morning) I then changed the delay to >1000ms in the camera settings and a quick test mode print seems to be stabilised. https://youtu.be/W9504NrecJA plugin.octolapse.log I'll test it out properly when I print something later and close this issue if it's fixed.

[Ruhel786]

@FormerLurker This is the best I could get from increasing the delay but the picture quality is very poor. https://youtu.be/SuNnNf234PM

If this is true, is that 'smooth' timelapse above from before a reflash or update or something?

The only difference between the smooth and jittery video is I ran apt update & apt upgrade then there was a message to update Octolapse. No other software changes were made. I'm going to reflash with an old version of OctoPi (Dev Build) that I was originally using, just have to find it. If that works I'm not gonna update and just leave it as is.

[FormerLurker]

Which version of OctoPi did you flash, the current release v1.0.0? Also, did you do the apt update and apt upgrade before your first print? I'm going to try that out here in any case.

I believe the issue must be that that the 'snapshot' action has changed (if your last install was using mjpg-streamer), or that the new camera-streamer is not waiting for the next frame to deliver a snapshot. It's also possible that there is some buffering going on. You could maybe try to increase the framerate and see if that reduces the necessary delay (1000ms seems quite extreme, and will lead to oozing). It may require a pull request to the camera-streamer repo to fix, unfortunately.

[Ruhel786]

Which version of OctoPi did you flash, the current release v1.0.0? Also, did you do the apt update and apt upgrade before your first print? I'm going to try that out here in any case.

I believe the issue must be that that the 'snapshot' action has changed (if your last install was using mjpg-streamer), or that the new camera-streamer is not waiting for the next frame to deliver a snapshot. It's also possible that there is some buffering going on. You could maybe try to increase the framerate and see if that reduces the necessary delay (1000ms seems quite extreme, and will lead to oozing). It may require a pull request to the camera-streamer repo to fix, unfortunately.

I used the pre-release with an updated camera stack as V3 Pi Cam doesn't seem to work with the stable release. I think you are correct as after updating the issue surfaced. Im going to do a fresh install with an old DEV build of OctoPi from Jan 2023 and see how it goes. How do I increase the framerate? And should I reduce the delay if I increase the frame rate? I'm lucky my last print only had 1 oozing issue. Hopefully with the build from Jan the issue doesn't surface and I'll just keep using that. Will keep you updated.

[FormerLurker]

I'm not sure how to change the settings with the new camera stack, but will let you know what I find during my testing. Keep me informed if you find anything out, and I'll do the same.

[FormerLurker]

I just flashed the latest pre-release octopi image, and there is definitely an increase in the delay even before updating. I'm running the update now, and will let you know how that goes. I posted a feature request to the camera-streamer repo asking if we can get an endpoint (or an endpoint option) for returning the next available frame. I looked through the source, and that might be a tall order. I don't understand the code currently, but I do know that there is a buffer, and it might not be easy to find out when the next frame is available.

I'll let you know what I find out after the update completes and I get a chance to make a timelapse in test mode. Shouldn't be long.

[FormerLurker]

So, this may be only a raspicam (libcamera.conf) issue, which is what I am assuming you have. My logitech seems to be working well after a slight increase in the camera delay (+100ms, which was only needed for the very first frame for some strange reason). Can you try this:

SSH into the pi. navigate to /boot/camera-streamer Edit libcamera.conf and set the OPTIONS like so:

OPTIONS='--camera-options="AfMode=2" --camera-options="AfRange=2" --camera-force_active'

See if that helps at all with a lower camera delay (say 250ms)

[FormerLurker]

So, results appear to be inconsistent on a usb camera. I did two videos, and the second had quite a bit of jitter (though a LOT less than you showed). I'm going to try adjusting fps and camera delay and see what happens.

[FormerLurker]

Last thing today hopefully: The print isn't finished yet, but so far doubling the FPS from 15 to 30 decreased the required delay enough that the jitter vanished without increasing the delay. My current guess as to what is happening (at least with the usb camera) is that when we request a snapshot, we are getting an image that is several frames behind. I would guess that this depends on the resolution/framerate/buffering, etc, so it's probably more complicated than I'm making it out.

I am going to assume that the most recent image you can acquire is the last image taken (as opposed to the NEXT frame). The default Octolapse camera profile has 125MS set for the camera delay, which was very conservative (I set mine way lower usually, sometimes to 0). The default framerate for camera-streamer in the new OctoPrint branch is 15 FPS. This means that you will get jitter if you are only 2 frames behind (66 .67MS 2 = 133.33MS, which is greater than the default delay of 125MS). At 30FPS, you can get away being up-to 3 frames behind with the default delay (33.33 3 = approx 100MS delay). Bumping the delay up to 134MS means you could be up to 4 frames behind. Note: This is a simplified version of what's happening, but I think it captures the essence of the issue. This shows it's a manageable, but would be SO much better if we could get the NEXT frame instead of something from several frames in the past. I will, without a doubt, get lots of issues similar to yours, and it will just add another confusing barrier for new users.

Now, for the raspicam, I think we need to figure out why exactly it's so far behind compared to the USB version. I am excited to see if the --camera-force_active flag has any impact. When I am done testing the usb camera, I'll go back to the raspicam, and then a dual camera setup. I fear adding more cameras might compound this problem, but the only way to tell is to test.

Ok, have a great weekend!

[Ruhel786]

Now, for the raspicam, I think we need to figure out why exactly it's so far behind compared to the USB version. I am excited to see if the --camera-force_active flag has any impact. When I am done testing the usb camera, I'll go back to the raspicam, and then a dual camera setup. I fear adding more cameras might compound this problem, but the only way to tell is to test.

Ok, have a great weekend!

I have good news. I made a thin tall cylinder stl to trigger the snapshot as much as possible. Im delay is as 250ms and I added the line in the .conf file. https://www.youtube.com/watch?v=xewJz0FVwtk I cant seem to see any jitter in the video with Raspicam V3. Thanks

EDIT: Im going to try with an actual print that moves the bed around a bit to see if it jitters.

[FormerLurker]

Fantastic!! There is already some movement in the camera-streamer issue I submitted, so this issue might even go away completely soon! I will keep you posted.

[Ruhel786]

So I have just finished a print and it appears to be fully stabilised back to its previous behaviour. Thanks, for all the help!! Stabilised Video