Not always taking photo with external DSLR and not giving any information about the error

[MineThingIssues]

Version of Octolapse

Octolapse Version:0.3.4

Version of OctoPrint

OctoPrint Version: 1.3.10

When you ran into the problem, did you have diagnostic logging enabled?

Diagnostic Logging was Enabled: Yes

What were you doing when the problem occurred

1. Configure Octolapse with External DSLR w/script
2. Connect printer and Canon 1300D camera to USB Hub that connects to the only free USB port on the raspberry pi model B
3. run a test print with full diagnostics

What should have happened?

The camera should have taken a picture every layer or at least thrown a readable error if not done correctly to help with debugging

What happened instead?

Camera worked for a few frames but every now and again it wouldn't take the picture and the error that came up on screen had no information. It simply read "Octolapse - Camera Error". It would either fix itself straight away and take a picture the next layer, or throw the same error again for the next few frames. The issue was more common at the start of prints and was less likely to happen later on. It would happen multiple times per print

Operating System running OctoPrint and Octolapse

OS Name: OctoPi Os Version: 0.16.0

Printer model & used firmware incl. version

Printer Model: Ender3 Printer Firmware Version: Marlin V1; Sprinter/grbl mashup for gen6

Browser and version of browser, operating system running browser

Browser: Google Chrome Browser OS: Windows

Link to the gcode file you were printing when the problem occurred

Link to Gcode File: Deleted -Issue also occurred on 7 other gcode files

Link to settings.json

Link to settings.json with all passwords removed: Forbidden to download.

Link to plugin_octolapse.log

Link to plugin_octolapse.log: https://gist.github.com/MineThingIssues/ff2ca8bdc3c0d0666a599f9eec41cb8f

Link to octoprint.log

Link to octoprint.log: https://gist.github.com/MineThingIssues/c2a7579969c54b6bd2e9c13a34d53872

Aditional Information: gphoto2 version: 2.5.20 libgphoto2 version: 2.5.22

screenshots:

[FormerLurker]

There's a lot going on here. First question: have you gotten Octolapse to work without the DSLR? If not, this is definitely step one since adding the DSLR only increases the amount of things that can go wrong.

Then the next thing to figure out is why you can't download your settings.json file. There is no good reason for this, as the file is never locked except while writing new settings. There are no errors in the log indicating an issue either. I did see an error regarding downloading the log file, but that problem is OctoPrint related unfortunately.

Also, I see lots of other non-octolapse related errors in your octoprint.log file. These look like they're related to the tracking plugin, so you should probably disable that.

Next, I see lots of errors related to a webcam, which probably either has an invalid URL in the camera profile, or maybe you don't have a webcam plugged in? Here is the error

HTTPConnectionPool(host='127.0.0.1', port=8080): Max retries exceeded with url: /?action=snapshot (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xaff24750>: Failed to establish a new connection: [Errno 111] Connection refused',))

In any case, either disable the webcam profile (uncheck it in the ocotolapse tab), or fix the webcam URL. You can test the webcam URL in the camera profile if you are actually using one.

Now the problem of missing snapshots depends a lot on what settings you are using (Octolapse and Slicer), so please do attach a small gcode file as a sample and see if you can get a copy of settings.json. Maybe try rebooting the pi and downloading the settings immediately? If not you should be able to grab it from the pi at the following location: /home/pi/.octoprint/data/octolapse/settings.json

Also, since this could be a DSLR issue, please let me know what your camera scripts looks like. The routine that generates errors relies on the script itself returning a readable error message. I'm working on this, but it turns out that it's quite tricky to execute an external process with a timeout and to reliably grab errors from the process's error stream (if the process even delivers error information at all).

Regarding this bit of info you provided 'Connect printer and Canon 1300D camera to USB Hub that connects to the only free USB port on the raspberry pi model B': So you are running your Cannon into a hub that then goes into the Pi? Also, when you say Pi model B, do you mean a Pi1B, Pi2b or Pi3b? I would remove the hub if possible from the equation at least for debug. Also, what's in the other USB ports besides your printer?

Once I have a gcode file, your settings.json file, and your camera scripts I should be able to debug. There are a few things you could do in the meanwhile. If you happen to be using 'layer trigger - high quality mode' please switch to 'layer trigger - compatibility'. The high quality mode will skip layers if it can't find a low impact place to take a snapshot, and that is by design. (Update: I see in your update you are using the compatibility version, that is good.) You also might want to turn your camera timeout up to 15 seconds just in case your DSLR is taking longer than five seconds to acquire and (maybe) download an image. If you are taking very high resolution images, this is definitely possible. It would be good to test at a lower res, so consider turning it down until things are tuned.

FYI, there are ways to GREATLY reduce the dslr snapshot time, and anything over a few seconds probably isn't acceptable from a quality perspective. Once things are working I can point you in that direction.

Thanks for your post, and I look forward to solving your problems.

[MineThingIssues]

Thanks for the quick reply :D As for the Pi and the USB hub, the Pi only has 2 USB ports, one of which is taken by a wifi USB dongle, the Pi doesn't have inbuilt wifi. On the Pi itself, it only says Pi1 so It may not even be a B (sorry for the mixup of information, I was told that it was a B but now I am unsure). As I am trying to print something now (and it has taken 5 hours so far so I don't want to cancel it just yet) I will have to wait until tomorrow to remove the USB hub. I will also try it without the DSLR. I have had the issue 10 times so far on this print: CE3_stagroot-kijaidesign-01.zip The camera script is: https://gist.github.com/MineThingIssues/f7b55284bc67167ff9ddbd12fdfeb812

Just ran into a weird issue (most likely the same issue) where it would take the picture but would continue straight after, not waiting 10-15 seconds for it to download like usual. This also (sometimes) throws an error like before, but I can see that the image preview for the timelapse updates with the new image most of the time. Edit: see my other comments as I now have the settings.json

as for the settings, the best I can do right now (as I don't want to cancel the print) is take screenshots (sorry for the added hassle) Both these camera profiles are the same just with a different description. I don't have a webcam set up and I only have one of the DSLR cameras selected for the octolapse timelapse.

[MineThingIssues]

After cancelling the 6-hour print, and restarting the server, I have managed to download the settings.json https://gist.github.com/MineThingIssues/878aa870ad3e71465e43553973548dac

[MineThingIssues]

Even with the timeout at 15 seconds, the picture is not taken (Although the camera screen goes black like it does when it takes a picture but for a shorter time) and it carries on straight away as if it thinks that it has taken and downloaded the image or the camera wasn't there at all. Quality has been reduced to the lowest (8MP)

[FormerLurker]

Oh no! I hope you didn't have to cancel because of Octolapse. There is a 'stop timelapse' button that will stop Octolapse without cancelling the print if you have problems with it in the future.

Now, getting to your error. I'd bet that your camera is timing out for several reasons:

1. Raspberry Pi1s aren't recommended for use with Octoprint and a streaming webcam. Even Pi0, has substantially more power, and it's not recommended either. Octolapse takes even more juice on top of that. For comparison, the PI1 has a single core at 750Mhz, 256MB of ram while the newer 3B+ has 4 cores at 1.2GHZ and 1GB of ram and 300 Mbps IO bandwidth (shared across all devices). I couldn't find the IO bandwidth of the PI1, but it's MUCH LOWER, probably way less than one third.
2. The IO bandwidth is seriously limited even on newer Pis (3b+ and the newer 3a), so your camera is transferring the image extra slowly via the USB.
3. Your timeout is 5S in your camera profile. I recommend cranking it up given your hardware.
4. On top of that all you are using a wifi dongle, which will further degrade performance.

Now, all that being said you still might be able to make it work. Here's what I would try:

1. Migrate from your current 'snapshot + download' script to a 'shapshot and exit immediately' script.
2. Either download the images and process the timelapse yourself, or use the 'pre render' script to download your images all at once at the end of your print.
3. Disable MJPEGSTREAMER and remove your webcam. You should do this even if you decide not to use Octolapse because your pi will cause print quality issues and maybe instability (possibly even without mjpegstreamer)
4. I recommend switching to Ethernet ASAP.

Check out this guide to prevent image download completely if you want to render your timelapse yourself, or the somewhat more complicated guide here if you would like to defer download until after your print is finished, but still have a timelapse automatically rendered. WARNING: That second option currently requires you to delete all images from your camera's SD card before starting. Also, not all cameras support the 'trigger-capture' gphoto2 flag, and some cameras return from trigger-capture before an image is captured. If you try either guide, let me know how it goes. They are in beta still and not in the wiki index yet.

[FormerLurker]

I saw in your your recent post that you already increased the timeout. leave it there while testing. You could go into bash and execute the 'take-snapshot' script over and over to see if it stalls or locks up. I would try both with a few minutes delay between calls, and some rapid fire requests.

Also, could your camera be going to sleep? Anyway if you can get either of the 'trigger capture' scripts running that I posted last time I think you might be able to make it work even on a pi1 (but keep in mind I have no evidence of that, lol!)

[MineThingIssues]

Thanks for the reply! I have read through the Deferred image download and near the end at step 5 should it be "Before render script"? Just checking because I know that the instructions are not finished and there are bound to be mistakes

[FormerLurker]

You are correct, nice catch! I will update that doc straightaway.

[FormerLurker]

The guide is now fixed, any other feedback you have would be appreciated!

[MineThingIssues]

Thanks for updating it :D I was wondering (step 4) if there are any parameters I should be aware of? I will not be trying it until tomorrow so you don't have to bother fixing it right away if there is

[MineThingIssues]

Raspberry Pi just failed to boot. I will reflash the SD card and install everything again and get back to you when I have everything sorted. If the Pi has died, Another one is on the way tomorrow (The Pi 3 B+) and I will get that one up and running as soon as possible

[MineThingIssues]

finally back to where I started lol

[FormerLurker]

Hmm, one of my comments never appeared. I thought I replied to this:

Thanks for updating it :D I was wondering (step 4) if there are any parameters I should be aware of? I will not be trying it until tomorrow so you don't have to bother fixing it right away

Yes, there are parameters! I won't have time today to figure them out, but just skip that for now. I recommend doing a very short print to test instead (feel free to use test mode as well to save time). I like this thinwall test. It's actually one of the most difficult things for Octolapse to handle too, so it's a good test of your settings if you do a live print.

[MineThingIssues]

Thanks, I will give that a go when I get time. my print settings in general are not very good at the moment so it will be interesting to see how well it prints anyway

[MineThingIssues]

I am currently 21 hours into a print and It will be about 15 hours more to finish but when I do I will set up the Pi 3 B+ and try out the deferred image download and tell you how it goes. Thanks for helping out with my issues!

[FormerLurker]

Sounds great, and it's my pleasure. I think you'll be blown away by the Pi3 compared to the Pi1! I'm considering trying the new Pi3A. The latest OctoPi has full support.

[MineThingIssues]

I think that one of the main reasons it is printing so slowly at 60mm/s 0.15mm layer height is that the Pi1 just can't handle it lol. I just installed octopi on the Pi3 B+ and wow it is fast. I noticed the heat (by accidentally slightly burning myself on the CPU) and I hope that I don't have to buy a heat sink for it to stop it from melting lol. It must just be that extra power. Just installing gphoto2 and libgphoto2 was so much quicker (30 mins rather than 3 hours). I can't wait for the print to finish in 11 hours from now and test it out with a new print and octolapse.

[FormerLurker]

Definitely get a heatsink (3 for pi3), and print a case first thing, lest your new hardware be rendered useless :)

[MineThingIssues]

Pi 3 A and A+ looks like a good option. quite similar performance to the B and B+ but more affordable (Although I need the extra functionality of the Pi 3 B+ for some other projects I will be working on soon)

[MineThingIssues]

I will buy some heatsinks asap lol

[yet-another-average-joe]

Hello, I had a similar problem. Same error message, but I could hear the DSLR (Nikon D5100) shooting pictures. Then, the printer stopped after 2 hours. I restarted from ground, and again, it stopped, but after 8 hours (12 hours print). With no error messages. Unfortunately, I was on a hurry, and did'nt pay attention to the terminal : I just disabled timelapses, as I needed the prototype for further development... I also completely crashed OctoLapse with a double timelapse : GoPro script + D5100 elabled (had to uninstall/relinstall OctoLapse : server error, icon not showing in menu bar). I always had problems with my RasPi 3 B+ and TouchUI (7" LCD Touch, HDMI, and all USB ports on a powered hub) : undervoltage. I tested with 2 RasPi, no differences. I'm considering feeding the Pi from a lab PSU directly to the GPIO +5V and GND pins... TouchUI seems to be a current hog ! I tested 2 different 3 amps PSUs.

(maybe I'm off topic, sorry... I read this thread many times, and not sure I understood each and every detail). It could also be related to #324

What could I do in order to give more accurate information ? Soon, I will print again a series of 12 hours designs, so it could be time to experiment. I've been thinking of experimenting with an Arduino Mega flashed with my printer firmware (Tevo Tornado, Marlin 1.3.9 or bugfix 2.0.x), with cold printing enabled, but there's a problem : homing. Do'nt know how to emulate homing ; maybe some push buttons ? I've only one printer, but lots of Arduino's and electronics components...

[FormerLurker]

@yet-another-average-joe, I've gotten reports about touch-ui, and there appears to be some kind of conflict. Also one of the progress plugins (can't remember which one). Try disabling ALL plugins and try with only Octolapse. If the issues go away it's likely a conflict. If I can figure out which plugin is causing the problem maybe I can figure out why and fix it.

Also turn off 'display real snapshot time' from the octolapse main settings too if you are having any problems.

If you are running underpowered it's just a matter of time until you have some kind of lockup or crash. I use this 40W adjustable PSU to run my pi and all other ~12V~ 5V equipment (lights, sensors, smoke detector, relays, etc...) and have never had a problem. However, I'm sure it's possible to build a program that will cause poor power situations to occur.

Have you had thermal throttling? If so definitely get some heatsinks, as they will help a lot. Also, what are your USB ports doing? Those can draw substantial power.

[MineThingIssues]

ordered some heatsinks and done some research. The Pi 3 B+ should have good enough cooling for a print that is only a few hours but after a longer time it can overheat without a heat sink or cooling fan. Printing something now with it and I am impressed! thanks for convincing me to upgrade. It can print about 5 times faster now. I will try octolapse after this print and tell you if I have the issue again. I will also try out the deferred image download. Here is a link to my first octolapse timelapse (35 hours on the Pi 1): https://drive.google.com/file/d/16wV6AO4eaaNV9FgW-lJZGpJ-_LjlHECc/view?usp=sharing

[FormerLurker]

@MineThingIssues, that is a GREAT looking video! The only thing I can see you might want to work on is your exposure. It looks like it might be on auto? If you turn auto exposure off you need to make sure your light levels don't change, which is a bummer, but it's better than the flickering. I think (but am not sure) that I can spot some LED flicker too (I have this problem big time with my strip lights), which can be caused by PWM operating at a lowish frequency (can't be seen with the naked eye at all). You can dampen that by putting small halogen spot close by, by reducing exposure, or by finding very high frequency LEDs, or LEDs that don't use pulse width modulation. Another bummer because PWM is awesome!

Also, good job on the quality! Almost no stringing and limited artifacting (yes, I just turned artifact into a gerund. I'm a rebel like that, lol). I wonder how faster printing will affect this, but my guess is that it should get even better. I can't believe you achieved that on a Pi1.. wow, really nice work!

[yet-another-average-joe]

@FormerLurker

OK for TouchUI, I will disable it. About thermal throtling : I had to put heatsinks as soon as I installed TouchUI ! It will be a pain : I'm a TouchUI fanboy (standalone and wth VNC)...

The plugins I use :

• Action Command Prompt
• Bed Visualizer (I did'nt even remember it was here...)
• Custom Control editor
• Fan Speed Control (it's just a button in config.yaml)
• Firmware Updater
• Gcode Editor
• Octolapse
• Preheat Button (a button in config.yaml)
• PrintTimeGenius Plugin (a must have, but it recalculates real time pinting duration)
• TouchUI (I'm aware that some users reported huge CPU usage, but I don't know how to visualize CPU usage)

I also installed Pixel. But Pixel is ran only for maintenance : I stop touchui service, and then sudo startx. So, I think it does not interfere . I never launch it while printing.

I will re-stress-test, ASAP, with no plugins except OctoLapse (within 1 or 2 days). But I really did'nt understand debug mode. Where's the log file ? And where are the OctoLapse JPG temp files ? (timelapse/tmp ?)

[FormerLurker]

PrintTimeGenius Plugin

This has known issues with OctoPrint in general (at least a recent version did). It's a REALLY cool plugin, but might need a bit of performance work since a lot of stuff is happening in the OnGcodeQueueing hook, which is also where Octolapse does the majority of its work. Definitely disable that one for testing + touch UI. I don't think any of the others should cause problems.

Also, I don't mean to say that either of those two plugins are bad or broken at all! Just disable for testing ;) In fact, I'm jealous of PrintTimeGenius using an actual marlin simulator. I'd LOVE to get timing on every gcode command for Octolapse! With that I might be able to stabilize a timelapse without stopping or pausing (thinking a lot about this), or at least pausing VERY little.

[yet-another-average-joe]

Oh, I forgot to say : the webcam is a USB cam. I imagine it's some more work for the CPU.

[FormerLurker]

You could run top or htop to see what's going on with your CPU. The mjpegstreamer process should be obvious.

[MineThingIssues]

I just ran /home/pi/scripts/initialize-camera-erase-all-images.sh for the step 4 (ik you said to skip it but i wanted to test it anyway) and it showed this error: . did I miss something?

[FormerLurker]

Was your DSLR plugged in and turned on? Also, depending on the model of your camera, this might not work :( Look here for info about your camera's capabilities. The 'Configure' capability must exist. Let me know what you find.

[MineThingIssues]

Thanks for the feedback on the video! Yea I forgot to put it on manual and the lighting was by a standard LED torch (or flashlight in the USA) powered via USB. It was not a controlled experiment. There was some weird artifacting on the side of the model but that could be down to bad filament or bad slicer settings. I might make an enclosure using a cardboard box and use a different light source but I would also have to get an AC adapter for the DSLR to stop me from moving the cardboard box when it runs out of battery. I will do some more experimenting before I set up anything to make sure everything is reliable and working.

As for the camera, It is plugged in and on. here are the listed things: It is also detected by gphoto2 when I do "gphoto2 --auto-detect" as the 1300D

[MineThingIssues]

I will try it with my other DSLR which also has "Configuration" listed but that only has a 1GB CF card so I don't know if it would be suitable

Edit: It doesn't support trigger-capture so It won't probably work for that 20-year-old camera lol

[FormerLurker]

If I were you I'd experiment with the gphoto2 command and see if I could recreate on the command line what that script is supposed to do.

1gb may not be enough depending on how many pics you shoot and what resolution.

[MineThingIssues]

I will attempt to recreate it on the command line and will tell you how it goes

[MineThingIssues]

Well... I just had to run it as sudo. It is now deleting all 3000 images (probably) I have on my camera so it is taking a while to show anything (I think). I will tell you if it works

[MineThingIssues]

Ok, it didn't work. It came back with an error that was similar to the error I was getting before I was running it with sudo before the script path. putting sudo in the "before print start script" came back with another error.

[MineThingIssues]

It works when I do it directly through SSH though:

[MineThingIssues]

also tried adding "pi ALL = (root) NOPASSWD: /home/pi/scripts/initialize-camera-erase-all-images.sh" to the end of the file that opens when you do "sudo visudo" and saving and restarting with no luck

[FormerLurker]

So, the Octolapse settings will only accept a path, not any bash commands, and that's for security.

Adding sudo to the front of your script shouldn't matter, but it obviously does. What happens when you run the gphoto2 command within that script manually? Does it prompt you for a password?

[MineThingIssues]

Sorry, could you clarify what you mean? I am a little confused

[MineThingIssues]

It does not prompt me for a password when I do gphoto2 commands as sudo manually. I will try putting sudo in front of the commands in the script and see if that works.

[MineThingIssues]

And it worked... Is this only an issue for me or is this potentially something you might want to add to the instructions? This is the working script:

[MineThingIssues]

did the same for the other scripts for every gphoto2 command and will test it now with a test print

[FormerLurker]

Excellent! I'm not sure why my gphoto2 is different than yours, but I don't need sudo to run those commands. It seems that I only need sudo for the download command, but that could have something to do with the installation method I used, which was a custom installation script. I need to do a from-scratch flash of OctoPi and start this process from the beginning to see if I have the same issues. I'll probably create a video at the same time (why not..).

Thanks for helping me improve the guide, and I hope the print works! Send me a link to your completed timelapse if you have time, I'd love to see it!

Also, sorry I wasn't clear in my last post. I'm blaming it on fatigue :) I just meant that when you enter the values for the camera script settings in the camera profile you can only supply a path like this: /home/pi/scripts/some_script.sh Things like sudo reboot or other bash commands like rm some_script.sh won't work because I have that specifically disabled for security reasons. Does that make more sense?

[MineThingIssues]

oh yeah sorry, I just woke up when I read the messages. that makes complete sense and the last message does too now that I have read it again sorry. I installed gphoto2 through a different method because the older version of gphoto2 installed through "sudo apt-get gphoto2" did not support my more recent camera. I checked on GitHub for the latest gphoto2 and libgphoto2 versions and did a manual install to get the most recent stable release. I followed this guide and changed the version numbers: https://hyfrmn.wordpress.com/2015/02/03/install-libgphoto2-and-gphoto2-from-source-on-raspberry-pi/

I also ran into this issue (the first one I can solve easily but the next one I can't work out):

[MineThingIssues]

I say that I can solve the first one easily but apparently, I am not the only one having the issue with the same camera and the same raspberry pi lol: https://github.com/gphoto/gphoto2/issues/150. I will do some more testing and see if I can solve the first issue. And it is taking some pictures so I would expect at least a few photos to be in the DSLR camera profile unless something else went wrong

[MineThingIssues]

increasing time between snapshots did not do anything so I added "gphoto2 --reset" to the end of the trigger capture script and it stopped the error and the snapshot count went up BUT it still only actually took a picture every other attempt... weird...

[MineThingIssues]

Here is a video showing the issue https://photos.app.goo.gl/EcPRPKvjeoVTsWhT8

[MineThingIssues]

I should probably ask this after the issue is resolved but is there an easy way to add a delay between when the camera is triggered and when the printer resumes? When the camera does take an image, the shutter speed is quite long and so the printer resumes before the image is fully taken (Resulting in consistent but slightly annoying motion blur... Well, probably. I can't check that because I can't see the images)

[MineThingIssues]

Could the issue potentially be due to this?: someone else mentioned something about it. Maybe that is why the downloading part doesn't work and potentially the camera gets confused (If I interact with it after it takes a picture it crashes). I am not a developer and know basically nothing about code but that is what would make sense to me. I saw that you included an alternative to "sudo gphoto2 --set-config capturetarget=2". Should I give that a go after my print finishes? Here is where I read about it: https://github.com/FormerLurker/Octolapse/issues/325

(On a side note, My mind is blown with how fast it can print now that it is running on a newer raspberry pi. It is at least 5 times faster than running it straight off of the included knockoff 8GB micro SD card that came with the printer and about 10 times faster than off of the old raspberry Pi 1 A. Thank you for suggesting that the Raspberry Pi was going to be an issue, you have saved about 2 days off of my current print time lol)