search

OctoPrint / OctoPi-UpToDate

Latest OctoPi image with OctoPrint already updated to the latest release
GNU General Public License v3.0
58 stars 7 forks source link

Feedback for the latest camera-streamer based webcam stack builds (builds 20230523124648, 20230627083607, 20230718115729) #6

Closed foosel closed 1 year ago

foosel commented 1 year ago

👉 Latest image & feedback ticket can be found here 👈

This is a ticket to collect feedback on the latest build of the new camera stack (some background on that here) for OctoPi-UpToDate.

Currently, that is 20230718115729. Find the image here: https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.9.2-20230718115729

Config docs are available here: https://faq.octoprint.org/camera-streamer-config

The source tree used for the build is here: https://github.com/OctoPrint/OctoPi-UpToDate/tree/camera-streamer

Changelog

Changes in build 20230718115729 from 2023-07-18

Changes in build 20230627083607 from 2023-06-27

Changes in build 20230523124648 from 2023-05-23

❗ Please note

This image needs to be flashed to your SD card, wiping it in the process. So if you want to keep anything on there, make a backup first! You can also create an OctoPrint backup, download that, flash, restore from backup. There's no way to update to the contents of this image without a full reflash.

Please provide feedback on your experience with this image in this ticket, whether things work for you or not. Make sure to at least include the following:

Known issues:

Currently none.

Common pitfalls:

b-morgan commented 1 year ago

I'd like to suggest that since this image is now available through the Raspberry Pi imager that it is time to give it a version number of its own, something like 1.0.1.

For an old fart like me, it will make it easier to determine from the web page which image I have installed as my memory isn't as good as it once was.

Ruhel786 commented 1 year ago

Installed on my test system, RasPi 4B 4GB, with RasPi Cam v3. Everything seems to be working fine, with v3 camera autofocus working fine. Restored from backup and it all restored perfectly. I installed the image by downloading it from the link above and installing it using the custom image option in RasPi Imager. For some reason if I tried to install the image using the 3D Printer option on the RasPi Imager and using the second (New Camera Stack) image it would not connect to the wifi straight away. I checked the wifi settings were all correct and even did a custom image again and it connected after its initial run just fine.

foosel commented 1 year ago

I'd like to suggest that since this image is now available through the Raspberry Pi imager that it is time to give it a version number of its own, something like 1.0.1.

I'd rather not mess with the version numbers, but I'll look into some way to make it more visible in OctoPrint proper.

For some reason if I tried to install the image using the 3D Printer option on the RasPi Imager and using the second (New Camera Stack) image it would not connect to the wifi straight away. I checked the wifi settings were all correct and even did a custom image again and it connected after its initial run just fine.

Did you try that once or multiple times? That's very strange and something I can't seem to be able to reproduce. Wouldn't be the first time I've heard of some issues with firstrun.sh however, on any image.

Ruhel786 commented 1 year ago

@foosel

Did you try that once or multiple times? That's very strange and something I can't seem to be able to reproduce. Wouldn't be the first time I've heard of some issues with firstrun.sh however, on any image.

Hi. The strange thing is I tried multiple times on my Windows PC and RasPi Imager and it didn't connect to WiFi. But when I used my MacBook Pro and the Mac version of RasPi Imager it worked first time today. I'm going to reinstall the PC version of the imager and try and later this week. But as of now on Mac it images and connects to WiFi perfectly. RasPi Cam V3 and RasPi 4B 4gb.

mod38 commented 1 year ago

I installed on a Pi3B+ with a V1 Pi camera. I copied the downloaded image onto an SD card (using Rufus). Everything went OK except that there was some ripple on the webcam video. I played with the settings in libcamera.conf and found that the problem was the frame rate of 15. I changed this to 10 and everything is fine now. The video seems clearer and more responsive.

Daniel-Vitanza commented 1 year ago

Hello and first, thx for your work ;)

1: Everything working? Yes, in YUV (ok) and MJPG (2s delay), but nothing in H264

2: Pi model Raspberry Pi 3B (Rev 1.2)

3: Used Camera Logitech C920 (via usb-default.conf)

FORMAT=YUYV => Not working with H264 WIDTH=1920 HEIGHT=1080 FRAMERATE=30

My cam is supposed to have an hardware H264 encoder. image

ayufan commented 1 year ago

@Daniel-Vitanza Correct, only YUYV and MJPEG should work. You should prefer to use MJPEG and the MJPEG should give the best latency.

Daniel-Vitanza commented 1 year ago

@ayufan ok for the h264 (sad)

But for the mjpeg, it is the opposite. I achieve the best latency with YUYV (compared at 1920x1080 30fps)

doucettom commented 1 year ago

Hi,

Any 64bits releases planned ? I was about to swap the farm Octopis to 64bits but I might just wait for a new cam stack+64bits release :)

Thanks,

foosel commented 1 year ago

Sorry, but no, currently not. I don't see the advantage for OctoPrint, and it would add a huge support overhead. Already running at capacity as is.

jellyphant8 commented 1 year ago

Everything working? At first yes. Pi model 4B used cameras Pi Cam v3

Sorry, no logs at this stage as I’m lying in bed on my smartphone but just noticed the stream stopped working. It was working when I setup the print but now I get a Server Error message and black screen. In saying that the room is poorly lit so wondering if the feed goes to sleep under certain light conditions. Probably not. I thought it would at least try and connect if I reopened the feed but just gives a server error. I can possibly provide logs tomorrow?

D203C775-08CF-41DB-BD93-90BB60BDBE0C F1D62142-E8D1-4D96-B561-B6814E0A37B6

277FF4A0-6552-492A-9EB0-CB31F713544A

foosel commented 1 year ago
  • in case of issues: related logs (journalctl -u camera-streamer\* | pb and share the generated paste.octoprint.org URL)

For the record, please do THIS when running into and reporting issues here, as mentioned in the first post.

About this problem there, we recently discussed something looking very similar, but that was seemingly isolated to initial bootup. If this is happening during runtime, that is a problem.

@ayufan any insight as to what would make camera-streamer loop "failed to start camera", "can't queue buffer", "state detected"?

ayufan commented 1 year ago

@ayufan any insight as to what would make camera-streamer loop "failed to start camera", "can't queue buffer", "state detected"?

Yes, it happens in some case:

Logs will tell more which one is the case.

foosel commented 1 year ago

Let's hope we can get some more logs then.

jellyphant8 commented 1 year ago

Hi, sorry I’m a bit of a shocker. I ended up doing a fresh install as the octoprint server wouldn’t start after I tried a to reboot to resolve the problem with the camera. I also got a thermal runaway error which caused the entire print to fail (first time on my printer), so not sure if this is related. I’ve got the same build installed so we’ll definitely know if there’s a problem if it happens again.

I’m not much help. I really should have sshd and retrieved the logs before reinstall. Apologies again.

foosel commented 1 year ago

I also got a thermal runaway error which caused the entire print to fail (first time on my printer), so not sure if this is related

Thermal runaways are a firmware thing, so no, that is completely outside of the control of anything on that image, be it camera, OctoPrint or whatever. That's your printer reporting a serious issue, and you might want to investigate. Could be a false positive, but be very very diligent here in confirming that.

jjnether commented 1 year ago

Sorry if this is the wrong place to ask, but basically I've been trying to get my raspberry module v3 camera to work with my rasp pi3A+, but have been having issues. After a good amount of research, trial and error, I came across this repository and figured it would be a good thing to try.

What is the process for installing this image? It looks like I should make a backup with current OctoPi config, then I can somehow install this image with raspPi imager (can I use the repository file for this?).

Thank you!

b-morgan commented 1 year ago

This image is now available in the Raspberry Pi imager. You should make an OctoPrint backup (and save it) and backup any files you wish to keep from your current image (I use WinSCP). Then write the image onto a microSD card, boot your RPi and go through the first time setup.

I have a USB SD card reader/writer. I plug this into my RPi and use rpi-clone to make a copy of the current image. Then I plug it into my desktop and image the new microSD card (to a different card). If you forgot something from your old card you can plug the USB reader into the RPi, mount the partition(s), and grab what you forgot.

jellyphant8 commented 1 year ago

I also got a thermal runaway error which caused the entire print to fail (first time on my printer), so not sure if this is related

Thermal runaways are a firmware thing, so no, that is completely outside of the control of anything on that image, be it camera, OctoPrint or whatever. That's your printer reporting a serious issue, and you might want to investigate. Could be a false positive, but be very very diligent here in confirming that.

Checked the printer and turns out the nozzle was covered in baked on pla which was causing the temperature to drop during print and trigger the runaway thermal error mentioned. Haven’t been able to reproduce the issue with camera feed yet but will report back if it happens again. Thanks

bmorgenthaler commented 1 year ago

Success!

OctoPrint: Version 1.9.0 OctoPi: Version 1.0.0

Hardware: Raspberry Pi Zero 2 W Rev 1.0 Camera: piCam v2 - imx219

Nothing added to /boot/config.txt for the camera it was detected right away. I did have to install Charlie's development Camera Streamer Control plugin to see the stream on the OctoPrint control page.

Now to figure out how to play with the resolution and framerate to get the lowest latency and no jumping.

One bit I noticed, is the reboot/restart menu options gone in 1.9 now? They aren't appearing underneath the wrench icon as before.

krish130899 commented 1 year ago

  1. Everything working? Yes, at first.

  2. Pi model Pi model 4B

  3. used cameras Pi Cam v3

I see a "server error" similar to the post above by @jellyphant8, It used to work seamlessly before. I have updated recently to 1.9.0, was running 20230504122418 build previously. I have checked after updating and the stream was working as usual, but after booting up a couple of days later saw this happening.

https://paste.octoprint.org/1nWMHGexBa

image image
jellyphant8 commented 1 year ago

Everything working? All good at first but no go now. Same story as krish130899. Pi model: RPi4b used cameras: RPi Camera Module V3 Logs: https://paste.octoprint.org/ly9XCzepXA

image image

foosel commented 1 year ago

Interesting that these "Server Error" problems all seem to happen with the RPiCam v3 module only.

@ayufan could that be related to the issue we were looking at last week on Discord?

lps-rocks commented 1 year ago

I have an older c920 (the one with the H264 onboard) and I cannot get more than 22FPS out of camera-streamer... Under the standard mjpeg-streamer i could get 30FPS @ 1080p without a problem but it would use > 100Mbps of bandwidth on the network.

I'm using MJPEG format with 1920x1080 (granted this gets changed to 1088 cause of the pi's internal hardware encoder working in 16 pixel chunks) @ 30FPS.

I've got the variable framerate turned off on the camera. To even get this far I had to change the CPU governor on the pi to 'performance' (I've opened a github issue with octopi about fixing that). Not running the 'performance' governor can cause unnecessary and unpredictable latencies as the CPU changes frequency scaling states.

This is what I see in my webrtc debugging...

https://i.hexhost.net/5dd56712-6cd4-4ff0-b513-6000acd51dd6.jpg

Not seeing any CPU bottlenecks or USB bus bottlenecks

Bus ID 0 (Raw USB traffic, all USB buses)       To device       From device
  Device ID 1 :                 0.00 kb/s       0.00 kb/s
  Device ID 2 :                 0.00 kb/s       0.00 kb/s
  Device ID 6 :                 141.67 kb/s     8076.29 kb/s
Bus ID 1 (Raw USB traffic, bus number 1)        To device       From device
  Device ID 1 :                 0.00 kb/s       0.00 kb/s
  Device ID 2 :                 0.00 kb/s       0.00 kb/s
  Device ID 6 :                 141.67 kb/s     8074.70 kb/s
Bus ID 2 (Raw USB traffic, bus number 2)        To device       From device
  Device ID 1 :                 0.00 kb/s       0.00 kb/s
cp2004 commented 1 year ago

I cannot get more than 22FPS out of camera-streamer... Under the standard mjpeg-streamer i could get 30FPS @ 1080p without a problem but it would use > 100Mbps of bandwidth on the network.

Make sure you aren't comparing apples and oranges - test the mjpg output of camera-streamer to compare to mjpg-streamer.

The performance of the h264 streaming (however it is transferred) will be more dependent on your Pi and the hardware encoder on it. Without knowing what Pi you have we don't know if this result is reasonable or not.

lps-rocks commented 1 year ago

I cannot get more than 22FPS out of camera-streamer... Under the standard mjpeg-streamer i could get 30FPS @ 1080p without a problem but it would use > 100Mbps of bandwidth on the network.

Make sure you aren't comparing apples and oranges - test the mjpg output of camera-streamer to compare to mjpg-streamer.

The performance of the h264 streaming (however it is transferred) will be more dependent on your Pi and the hardware encoder on it. Without knowing what Pi you have we don't know if this result is reasonable or not.

I have a Pi 4b, using the camera in MJPEG mode in the camera streamer config.

ayufan commented 1 year ago

@lps-rocks If you use mjpeg stream the camera will likely be underrun with just 2 buffers (currently configured), as the transmit in mjpeg stream takes significant amount of time. Try to change this to --camera-nbufs=3 in OPTIONS.

lps-rocks commented 1 year ago

@lps-rocks If you use mjpeg stream the camera will likely be underrun with just 2 buffers (currently configured), as the transmit in mjpeg stream takes significant amount of time. Try to change this to --camera-nbufs=3 in OPTIONS.

Forgot to mention that, I found i get the most stability at nbufs=4, nbufs=3 still allowed for dropping below 22FPS, once I moved to nbufs=4, I get a solid 22FPS, but for some reason it won't go higher.

ayufan commented 1 year ago

I would refrain from saying that we should change OctoPi governor. I think we should stay with ondemand, but tune gpu_freq and alike to force specific clock. It will be more power efficient, and will remove re-encoding stuttering.

mgarratt commented 1 year ago

Everything working? At first Pi model: 3B+ used cameras: Logitech C615

Hi

Thanks for all the work on this! I've decided to try it out to get away from the bandwidth usage on mjpeg-streamer.

Everything seemed to work out of the box at first, but since then I can't get anything other than the MJPEG stream to work. I've tried changing the format and messing with resolutions and frame rates with no luck.

The logs say [h264 @ 0x68902aa0] decoding for stream 0 failed which I think is the culprit and might be coming from ffmpeg?

camera-logs.txt usb-c615.conf (as a txt file for GitHub support)

mgarratt commented 1 year ago

I noticed webrtc wasn't working on other websites, this is something to do with my Firefox config. Everything works as expected in a clean profile

steveiliop56 commented 1 year ago

Hey!

I recently updated to the new camera stack and I can't help saying that I may go back to the older one. While the new camera-streamer app is doing its job fine, it is very limited in options, for example I cannot easily set exposure, brightness and image quality as I could with a simple config line in octopi.txt. I want to mention that I use the official raspberry pi camera v2 and a raspberry pi zero 2w for my ender 3 v2. Also the docs are very limited and the --help arg gives me little to no information about basic things.

cp2004 commented 1 year ago

You should be able to configure the OPTIONS line in your /boot/camera-streamer/*.conf files. If you need help figuring out the options you want to use, make a post on the community forums or discord server, and I'll try and share some examples, if you want to try and translate your old options from mjpg streamer. Instructions on the config options are below: https://community.octoprint.org/t/camera-streamer-configuration-on-the-new-camera-stack-for-octopi/49950

steveiliop56 commented 1 year ago

Yeahh I have tried that but still I cannot make it work as it was in the older octopi...

cp2004 commented 1 year ago

It is not going to work 'as it was' because the entire camera system has been changed, your options will need adapting. I'm sure we would be able to create an equivalent OPTIONS field in discussion on the forum.

bradbee36 commented 1 year ago

Everything working? Sadly, no.

image

Pi model: Raspberry Pi 4 Model B Rev 1.5 Camera: Raspberry Pi Camera Module 3 in case of issues: https://paste.octoprint.org/9IcvFr9uOm

gbickel1 commented 1 year ago

Everything working? Yes! Pi model: 4b used cameras: generic USB Endoscope for Android from Amazon (nozzle), Hue HD Pro Doc camera (bed)

I had to re-image my installation a couple weeks ago when my power started flickering on and off during a storm, corrupting the image. (I now have it plugged into a UPS). I recalled having a hard time understanding the muEverything working? Pi model used cameras in case of issues: related logs (journalctl -u camera-streamer* | pb and share the generated paste.octoprint.org URL)le cam setup in the original image, so I decided to try this. It was SO EASY following the multiple camera setup, made for MUCH LESS stress.

Thank you @foosel . I think I need to increase my monthly patreon contribution, I only wish it could be more.

image

foosel commented 1 year ago

@bradbee36 this looks like your system isn't even detecting the camera. Please make sure you have connected it correctly, it's easy to mix up the orientation of the ribbon cable. The blue part needs to be facing the USB ports on the Pi side:

image

https://thepihut.com/blogs/raspberry-pi-tutorials/16021420-how-to-install-use-the-raspberry-pi-camera

If you are 100% sure all of that is correct, please provide the outputs of the following commands:

Alternatively, if this is a fully updated OctoPrint setup (as in, including OctoPrint-PiSupport 2023.5.24), share an OctoPrint System Info Bundle.

FreeBSDGeek commented 1 year ago

Excellent job with the camera. I have been running Octopi and Octoprint since the early days and this is the first time the Raspberry Pi camera has just come up without a day or two of paying around with it. It just worked.

I do have one issue though and it might just be with the OS distribution the release is created with. What version of Raspberry Pi OS was used? The problem is that with this version of OctoPi I have the wifi (it is connecting at 5G) cutting in and out. What I look at the logs, I see it reconnecting all the time. Same hardware with 2023-05-03 32-bit Raspberry OS (Bullseye) on it and the wifi works perfectly.... Same with the older OctoPi (0.18) version.

I don't want to hijack this thread with a problem that is not camera related. Just wanted to get some release info so I can track this issue down.

The Camera work is awesome. Great work.

--- Kirk

gbickel1 commented 1 year ago

I would like make a note that I needed to change the camera format to MJPG for my 2 USB cameras. This solved the issue of the cameras dropping out after a few minutes only to return after a reboot.

Also, one question I use the Octopod app with the Octopod plugin. I can only see the camera that is referenced as /webcam in the app. In the prior build, my cameras were /webcam and /webcam2. I assume Octopod only sees one camera because it does not yet recognize the new camera stack?

bradbee36 commented 1 year ago

@bradbee36 this looks like your system isn't even detecting the camera. Please make sure you have connected it correctly, it's easy to mix up the orientation of the ribbon cable. The blue part needs to be facing the USB ports on the Pi side:

image

https://thepihut.com/blogs/raspberry-pi-tutorials/16021420-how-to-install-use-the-raspberry-pi-camera

Thank you for taking the time to reply to my post. Your response led me to the culprit: a faulty ribbon cable. The camera is now working. Danke sehr!

Maeur1 commented 1 year ago

Everything working? Yup Pi model: Model 3B used cameras: V1 camera

Was wondering, or maybe I missed it in the configuration, is there a way to rotate the feed 90 or 270 degrees? Or is this a limitation of the new stack?

gnurbs commented 1 year ago

works great! (as far as I could see, and as a new user of octoprint) Pi model: zero W (yes the single-core one) used cameras: V2

it would be great if camera-streamer could re-configure the camera for stills:
with the V2 camera in particular, for videos the go-to mode is 1640x1232 - using the full sensor and in particular the full field of view with 2x2 pixel binning, which is a resolution that the pi's hardware video encode supports. But for stills it might be great to use the full resolution without binning - but for that the sensor mode needs to be switched, not only the cropping changed.

foosel commented 1 year ago

@Maeur1 That is a limitation of camera-streamer. However, you can achieve the same effect in OctoPrint itself, it has rotation and flip options for the camera.

@gnurbs I think that is feedback better reported the camera-streamer repo - I don't develop camera-streamer (it's WAY above my skillset!), I've only put together some glue code here to make it work well for OctoPrint streaming/timelapse purposes.

Maeur1 commented 1 year ago

@Maeur1 That is a limitation of camera-streamer. However, you can achieve the same effect in OctoPrint itself, it has rotation and flip options for the camera.

Indeed I have performed this I just have other places the stream goes like homeassistant (custom card), but I guess I can configure those to do the rotation too (Octopod seems to rotate it correctly too thanks to the octopi stream settings). It is interesting how the old stack had some rotation built in but the new stack, it seems like its not even realised to be a regression: https://forums.raspberrypi.com/viewtopic.php?t=273018&start=125#p1944694

delovelady commented 1 year ago

You should be able to configure the OPTIONS line in your /boot/camera-streamer/*.conf files. If you need help figuring out the options you want to use, make a post on the community forums or discord server, and I'll try and share some examples, if you want to try and translate your old options from mjpg streamer. Instructions on the config options are below: https://community.octoprint.org/t/camera-streamer-configuration-on-the-new-camera-stack-for-octopi/49950

Everything is working great for me (Logitech C930e, no RPi cam) except that the image is very soft/out of focus. I suspect, especially after reading this comment, that further details belong elsewhere ("post on the community forums or on the discord server..."). So before going further, I'll ask for details about how I should ask my questions intelligently to make the best of your time and mine. But since I'm already here bothering y'all, I'll also point out that I hope to use an Insta360 Link as a second camera. Initially when I tried working with that (months ago now), it got very hot and unreliable.

So any pointers you can give me regarding moving forward with my request for help, would be greatly appreciated!

foosel commented 1 year ago

FYI, since OctoPrint 1.9.1 got released this morning, a new build with OctoPrint 1.9.1 on board is available with build 20230627083607.

I'll also soon push an update for the bundled OctoPrint-PiSupport plugin that will show the current build number with the OctoPi version.

kaysond commented 1 year ago

I also ran into the "Server Error" issue with my RPi Camera Module v3 (wide). From the logs I could see it was caused by "Stale Detected". I set NBUFS=3 in the conf file, restarted the service, and now it seems to work.

This could either be set in the conf file by default or we could overwrite the systemd unit. May be worth fixing upstream if it's a common issue, though.

foosel commented 1 year ago

FYI, since OctoPrint 1.9.2 got released today, a new build with OctoPrint 1.9.2 on board is available with build 20230718115729.

And I haven't forgotten about pushing an update for the bundled OctoPrint-PiSupport plugin that will show the current build number with the OctoPi version, just had other stuff on the TODO list first.

gdombiak commented 1 year ago

Hi @foosel ,

I just came across a new issue that is 100% reproducible.

  1. Burn new image with new camera stack (the one you created 2 days ago)
  2. Configure libcamera and usb camera. I have one of each. RPi v3 and some usb camera
  3. Restore OctoPrint from backup
  4. Confirm that both cameras render fine in OctoPrint
  5. Perform a full OS upgrade (sudo apt update followed by sudo apt full-upgrade)
  6. Reboot
  7. Check that cameras still work

I check the journal before doing step 5 and it was all good. However after doing steps 5 and 6 I see this error in the journal

-- Boot 4560b9717de84b31b1512c97612705d7 --
Jul 19 22:47:07 octopi2 systemd[1]: Starting camera-streamer default...
Jul 19 22:47:07 octopi2 systemd[1]: Started camera-streamer default.
Jul 19 22:47:07 octopi2 sh[677]: /bin/sh: 1: exec: /usr/bin/camera-streamer: not found
Jul 19 22:47:07 octopi2 systemd[1]: camera-streamer-usb@default.service: Main process exited, code=exited, status=127/n/a
Jul 19 22:47:07 octopi2 systemd[1]: camera-streamer-usb@default.service: Failed with result 'exit-code'.

indeed camera-streamer is gone. This is what I have after steps 5 & 6.

pi@octopi2:~ $ ls -l /usr/bin/camera-*
-rwxr-xr-x 1 root root 5131 Jul 11 03:42 /usr/bin/camera-bug-report
-rwxr-xr-x 1 root root 1755 Jul 18 05:08 /usr/bin/camera-streamer-control

I checked before doing step 5 that /usr/bin/camera-streamer existed and it was there. Of course, since no journal errors and cameras worked fine. :)

IOW, I cannot do an OS upgrade with the latest image since it breaks cameras support. Things were fine with the previous image release. Not sure what changed.

FYI, I remember seeing libcamera0 being updated when I did the upgrade. I think there was another libcamera file being updated but cannot recall its exact name.

Danke Gaston