OctoPrint / OctoPi-UpToDate

Latest OctoPi image with OctoPrint already updated to the latest release
GNU General Public License v3.0
58 stars 7 forks source link

Feedback for the new camera-streamer based webcam stack #2

Closed foosel closed 1 year ago

foosel commented 1 year ago

πŸ‘‰ Latest image & feedback ticket can be found here πŸ‘ˆ


OctoPi 1.0.0 is out, and I also have a new camera stack for you all to test!

Over the course of the past two weeks I've been setting up a CustoPiZer build script to swap the mjpg-streamer/webcamd combo on OctoPi that currently only supports the old legacy camera stack with a new camera-streamer and systemd based setup that supports libcamera (think RPiCam v3 as well as newer Arducams on top of the old ones). Some systemd magic takes care of camera detection, hotplug and even multicam support (successfully tested with two USB cameras and one RPiv3cam).

You can find the image here: https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.8.7-20230309160428. Image last updated on 2023-03-09 @ 16:50UTC!

Some first config docs are available here: https://faq.octoprint.org/camera-streamer-config

And finally, this is the source tree used for the build: https://github.com/OctoPrint/OctoPi-UpToDate/tree/camera-streamer

❗ Please note

This image needs to be flashed to your SD card, wiping it in the process. So if you want to keep anything on there, make a backup first! You can also create an OctoPrint backup, download that, flash, restore from backup. There's no way to update to the contents of this image without a full reflash.

I hope to be able to merge this into the OctoPi-UpToDate main branch ASAP, after which it will become part of the OctoPi image distributed on the download page and RPi Imager, but some testing from others would certainly help here πŸ™‚

Please provide feedback on your experience with this image in this ticket, whether things work for you or not. Make sure to at least include the following:

Draknars commented 1 year ago

New image here: https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.8.6-20230228172408

@ayufan well, I hope we'll get some feedback out of this for you :)

Guess I will have to re-image and restore a backup to update? Curious to see if address the issue I experienced

foosel commented 1 year ago

Correct, backup, flash, restore.

TrainorEC commented 1 year ago

Everything is working with newest image. Using Raspberry Pi 4B with RPi Camera Module 3 Wide. Image is super clean using the default settings. I've printed a couple small prints without any issues. Thank you.

b-morgan commented 1 year ago

This issue still exists: "A stop job is running for camera-streamer (20s / 1min 30s)", finishes after the 1min 30s.

I'm also having the problem of "firstrun.sh" not being executed. This time on my RPi 3B with my printer attached.

Good news is my RPi 4B with camera 3 is working (without a printer attached).

b-morgan commented 1 year ago

@ayufan In the two replies I've linked below, I posted some images of a problem I'm seeing with my RPi 3B with an RPi camera V2 attached. In the second link, I have narrowed the issue down to camera-streamer's default settings for the V2 camera (i.e. libstreamer-jpeg gives me the image I expect, but camera-streamer's image in OctoPrint is zoomed in.

Can you help me get to the bottom of this? Do I need to specify options and if so, which ones?

Thanks!

https://github.com/OctoPrint/OctoPi-UpToDate/issues/2#issuecomment-1444341655 https://github.com/OctoPrint/OctoPi-UpToDate/issues/2#issuecomment-1445374954

foosel commented 1 year ago

This issue still exists: "A stop job is running for camera-streamer (20s / 1min 30s)", finishes after the 1min 30s.

Confirmed. I couldn't reproduce it with my fix yesterday, repeatedly, but this morning sure as hell it's back. Need to dig deeper.

I'm also having the problem of "firstrun.sh" not being executed.

I still have not even remotely been able to reproduce this, and I still don't even see how the modifications on the camera-streamer branch could achieve this, so I need more info here. What version of the RPi Imager are you using here? Does your SSID contain a single quote ' by any chance? Is it isolated to SD card/RPi/any other particular combination of factors?

foosel commented 1 year ago

New image build up, https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.8.6-20230301124943. Outdated, latest always see https://github.com/OctoPrint/OctoPi-UpToDate/issues/2#issue-1595057632

Spent half the day wrapping my head around systemd ordering which apparently was the issue with the blocking on reboot, until I finally came across --no-block which together with some cleanup in the generated path files now really seems to have solved it, at least I have not been able to trigger this again after 10+ reboots.

b-morgan commented 1 year ago

All of my images have been written with RPi Imager 1.7.3 onto Samsung microSD cards. My SSID contains alphabetic characters including a space, passphrase alphanumerics, no space. While I was confused as to the cause in the beginning, I believe the underlying issue is that firstrun.sh is not being executed.

My first experience was on my RPi 4B. Repeatedly failed multiple times with image 4422 until I switched from a 32GB micro SD card to a 16GB micro SD card. The 32GB micro SD card that failed with 4422 worked with 5002.

My most recent experience was with my RPi 3B. The microSD card with 4422 (that worked first time) was reimaged with 2408 and it failed. I was able to attach a 7" HDMI monitor and a keyboard, logged in and did "sudo /boot/firstrun.sh". Rebooted and I'm back in business (except the RPi started complaining about low voltage). Removing the monitor and keyboard solved that problem.

I have ordered more micro SD cards including some non-Samsung brands. I have an RPi 3B+ that I can test with to add a third system to the experiment. I don't have a third RPi camera but I can steal a Logitech USB camera from another system.

foosel commented 1 year ago

It's really weird. I just tried earlier to explicitly flash fully manually with the latest rpi imager, and here it just works like it should, as it has also done for the automatically created firstrun.sh files in my whole testrig setup (which basically mirrors what the rpi imager does). So currently I really have no explanation but I have high doubts this is something that's caused by the new camera stack, given that the firstrun mechanism should trigger before it even has a chance to do anything.

bmorgan99 commented 1 year ago

I have 2 3Bs working with the latest image, one with a 2.1 camera and module 3. Let me know if I can help.

foosel commented 1 year ago

@bmorgan99 did you set wifi credentials etc via Raspberry Pi Imager, and was that successful?

b-morgan commented 1 year ago

I agree with your logic. I'm just reporting what is happening to me! I've looked for how firstrun.sh is called and haven't found anything yet.

BTW, @bmorgan99 and @b-morgan are different people just in case you are as confused as I was πŸ˜„

foosel commented 1 year ago

@b-morgan got confused a couple days ago as well but now know I need to look twice ^^

bmorgan99 commented 1 year ago

even I did a double take 😞

foosel commented 1 year ago

@b-morgan regarding your RPiCamv2 issue, could you please try the following in /boot/camera-streamer/libcamera.conf:

### Options for libcamera based cameras (PiCam, Arducam, ...)

# The port on which the webcam server for the camera should listen on. 
PORT=8080

# The resolution to request on the camera sensor.
WIDTH=3280
HEIGHT=2464

# The height to use for the video stream.
VIDEO_HEIGHT=720

# The height to use for the snapshots.
SNAPSHOT_HEIGHT=1080

# The framerate to set on the camera.
FRAMERATE=30

# Additional options.
OPTIONS='-camera-options=brightness=0.1'

Restart the Pi or alternatively just the camera-streamer via sudo systemctl restart camera-streamer after changing.

b-morgan commented 1 year ago

@foosel I will try that libcamera.conf as soon as I reimage both systems to the latest image.

With regards to the firstrun.sh issue, firstrun.sh is executed from cmdline.txt:

console=serial0,115200 console=tty1 root=PARTUUID=b1214a26-02 rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspberrypi-sys-mods/firstboot systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

At the end of firstrun.sh are the lines:

rm -f /boot/firstrun.sh
sed -i 's| systemd.run.*||g' /boot/cmdline.txt

which remove all the firstrun.sh references. Is there something you are doing with systemd to setup the camera-streamer that might be causing a race condition with this cmdline.txt execution of firstrun.sh?

foosel commented 1 year ago

Not that I'm aware of. The unit files want multi-user.target, same as OctoPrint, so from my (admittedly still limited!) understanding they should not even be started here.

Maybe there's something helpful in the logs, can you run journalctl -b 1 > firstboot.log.txt and share that here? That should contain everything that was logged during the very first boot (which should have been running the firstrun.sh file).

b-morgan commented 1 year ago

Murphy strikes again! With the latest image (4943) it worked on the RPi 3B. The libcamera.conf changes also worked.

I'll test the reboot issue after I image a card for the RPi 4B (it's much easier to attach the monitor to that one).

b-morgan commented 1 year ago

RPi 4B booted and ran firstrun.sh successfully with 4943 image. However, it still did the 1m30s stop job. I suspect the RPi 3B does the same thing because it takes longer than I expect to come back after a reboot.

foosel commented 1 year ago

Please share the logs as described in the top post. I even reduced the timeout to 10s (see eb4b339), so this sounds very odd and I need some logs.

b-morgan commented 1 year ago

I just ran journalctl -u camera-streamer\* > logs.txt and renamed the file after I moved it to my desktop with WinSCP.

rpi4bcamera-streamer.zip

Do you want the output from the RPi 3B as well?

Draknars commented 1 year ago

@bmorgan99 did you set wifi credentials etc via Raspberry Pi Imager, and was that successful?

@foosel In my case, with the first image I tested I was not able to set credentials using the RPI Imager, I was with able with the built-in one, but no when trying to use the one downloaded here, but maybe my process was wrong?

So just to confirm, are you suppose to download the image here and "Use custom" in the wizard?

b-morgan commented 1 year ago

@Draknars Yes, download the (latest) image posted in this topic and "Use custom" in the RPi imager (1.7.3). You can setup the hostname, SSH, password, WiFi credentials, locale, etc. using the "gear" icon below (and before) clicking the "Write" button.

Ruhel786 commented 1 year ago

Camera V3 working on Raspberry Pi 4 2GB (with HyperPixel 4.0 square touchscreen) The new image was installed with the RPi imager 1.7.4.

I manually configured the wifi settings in octopi-wpa-supplicant.txt using SSH and uncommented the relevant sections.

I followed @paulmaunders (thank you) instructions posted above (sorry I don't know how to link comments, new to GitHub)

Hardware: Raspberry Pi 4 Model B 2GB. Raspberry Pi Camera Module 3.

Camera feed working in OctoPrint. I will test TimeLapse tomorrow morning and report back.

Screenshot 2023-03-01 at 22 08 47

I was unable to get the camera to work or be found with the HyperPixel 4.0 square screen until I added these 2 lines to the config.txt

disable_poe_fan=1 force_eeprom_read=0

Once I added these I have are to get the libcamera-hello --list-cameras command to work:

Available cameras

0 : imx708 [4608x2592] (/base/soc/i2c0mux/i2c@1/imx708@1a) Modes: 'SBGGR10_CSI2P' : 1536x864 [30.00 fps - (0, 0)/0x0 crop] 2304x1296 [30.00 fps - (0, 0)/0x0 crop] 4608x2592 [30.00 fps - (0, 0)/0x0 crop] PXL_20230226_100046224

GitIssueBot commented 1 year ago

This issue has been mentioned on OctoPrint Community Forum. There might be relevant details there:

https://community.octoprint.org/t/raspberry-pi-camera-setup-in-latest-stable-octopi-build/50231/4

dmcke5 commented 1 year ago

Hi guys, I was sent here to try this build in order to get my v1.3 raspberry pi camera working but I haven't had success yet. I've flashed the "1.0.0-1.8.6-20230301124943" image but I'm getting the "Webcam stream not loaded" issue in octopi.

I've had a read through this thread and I tried this line libcamera-hello --list-cameras but all it returns is no cameras available. The camera is definitely connected the right way and I do get a couple of blinks from the cameras status light as the pi boots up.

Any suggestions of where to go from here? I haven't been able to find any logs that I can share, so if you need those you'll have to point me in the right direction for them. Cheers!

ltlowe commented 1 year ago

I've just flashed octopi-1.0.0-1.8.6-20230301124943 and everything seems to be working fine using defaults. RPi 3B+ with imx708-wide camera module v3.

I do not use wifi so my procedure was simply:

and the camera was streaming! I did not modify anything. I have not had a chance to try octolapse or the built-in timelapse yet. Some people had problems with still capture timing and the temporary Camera Module 3 support that we were previously running, although it was working ok for me by the end.

foosel commented 1 year ago

So, quite embarrassing, but it turns out the timeout issue on reboot was still there because while I had made the commits to fix that and thought I'd pushed them before triggering yesterday's build, apparently that was not the case because that build built from a05e8c8d4c176c92d7cc94283e33f34af2c8ea71.

New image, built from commit 50dafcec5892c5303c58cd7b2232e5025ae3a024: https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.8.6-20230302093328 Outdated, see https://github.com/OctoPrint/OctoPi-UpToDate/issues/2#issue-1595057632 for link to latest

(Btw, just for the record, the generic looking number at the end of the image names is the date and time it was built at, so this one on 2023-03-02 at 09:33:28 UTC.)

Also huge thanks @b-morgan for those logs which made me look at the service file for the changed timeout, which made me discover the issue.

foosel commented 1 year ago

I've had a read through this thread and I tried this line libcamera-hello --list-cameras but all it returns is no cameras available. The camera is definitely connected the right way and I do get a couple of blinks from the cameras status light as the pi boots up.

@dmcke5 Given that from that thread I gather that your camera also didn't work with the old camera stac, I'm leaning towards something being wrong with that camera tbh. If libcamera-hello --list-cameras doesn't see it, it certainly sounds off, if that's a genuine rpi camera I'd expect it to be detected.

What does echo /sys/bus/i2c/devices/*/video4linux output?

Ruhel786 commented 1 year ago

@foosel Hi I'm new to the Pi and coding. Regarding your new image https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.8.6-20230302093328 Do I need to reinstall EVERYTHING from scratch or can I install the latest version over top of the previous version? Thank you Ruhel

foosel commented 1 year ago

All of these images need to be flashed to your card, as in, they completely wipe your SD card and replace it with a fresh install of the image.

If you want to preserve your OctoPrint data, you need to make a backup first, which you can then restore from.

b-morgan commented 1 year ago

The reboot issue is fixed and aside from the RPi imager 1.7.4 changing my username (i.e. I missed it) things look good here.

b-morgan commented 1 year ago

I assume that the next UpToDate image will include 1.8.7, correct?

foosel commented 1 year ago

Yes. Didn't want to build yet another one only for that and spam y'all even more with images ;)

b-morgan commented 1 year ago

Except I'm in the middle of updating to 3328 πŸ˜‚πŸ₯°. I'll survive, when I logged in to shutdown 4943 it prompted me to update to 1.8.7. Here's hoping there will only be the "official" image soon!

Ruhel786 commented 1 year ago

Except I'm in the middle of updating to 3328 πŸ˜‚πŸ₯°. I'll survive, when I logged in to shutdown 4943 it prompted me to update to 1.8.7. Here's hoping there will only be the "official" image soon!

Did you update to 1.8.7. without any issues? I just got the prompt earlier and did not know if I should update it.

foosel commented 1 year ago

It really should behave just like a regular image, so by all means, update to 1.8.7 through the update dialog, like you normally would. Consider it an additional test.

b-morgan commented 1 year ago

Another data point... RPi 3B+, Logitech C525 (identified as C505) USB camera. The camera-streamer found it and OctoPrint seems happy with the video stream.

dmcke5 commented 1 year ago

I've had a read through this thread and I tried this line libcamera-hello --list-cameras but all it returns is no cameras available. The camera is definitely connected the right way and I do get a couple of blinks from the cameras status light as the pi boots up.

@dmcke5 Given that from that thread I gather that your camera also didn't work with the old camera stac, I'm leaning towards something being wrong with that camera tbh. If libcamera-hello --list-cameras doesn't see it, it certainly sounds off, if that's a genuine rpi camera I'd expect it to be detected.

What does echo /sys/bus/i2c/devices/*/video4linux output?

Yeah, I think you might be right. I decided to grab a cheap USB webcam this afternoon and it worked first try, no configuration changes necessary. Not sure what to do with the Pi camera, it was brand new and I bought it specficially for this project. Oh well, at least it was cheap I guess!

ayufan commented 1 year ago

@dmcke5 What type of camera you have? Can you attach dmesg and v4l2-ctl --list-devices?

dmcke5 commented 1 year ago

You mean the new camera? Its a cheap no-name thing from a local retailer. Here's a link anyway for reference https://www.kmart.com.au/product/webcam-ring-light-43104994/? Probably not much use to anyone outside of Australia.

I assume you only want the relevant line from what dmesg returns since its so long? I think this is the only line that references the camera: input: USB PHY 2.0: USB CAMERA as /devices/platform/soc/3f980000. usb/usb1/1-1/1-1.2/1-1.2:1.0/input/input0

v4l2-ctl --list-devices returns:

bcm2835-codec-decode (platform:bcm2835-codec): /dev/video10 /dev/video11 /dev/video12 /dev/video18 /dev/video31 /dev/media2

bcm2835-isp (platform:bcm2835-isp): /dev/video13 /dev/video14 /dev/video15 /dev/video16 /dev/video20 /dev/video21 /dev/video22 /dev/video23 /dev/media0 /dev/media1

USB PHY 2.0: USB CAMERA (usb-3f980000.usb-1.2): /dev/video0 /dev/video1 /dev/media3

ayufan commented 1 year ago

@dmcke5

The camera is definitely connected the right way and I do get a couple of blinks from the cameras status light as the pi boots up.

I talk about CSI camera connected. USB will universally work, always.

dmcke5 commented 1 year ago

@dmcke5

The camera is definitely connected the right way and I do get a couple of blinks from the cameras status light as the pi boots up.

I talk about CSI camera connected. USB will universally work, always.

Its all unplugged now and for the time being I'd rather leave it alone since its working the way it is, sorry! The camera is a Raspberry pi camera V1.3. If I get time over the weekend I have a spare pi 4 here that I will try the camera on and see if I can get it working and I'll run those commands and report back.

ayufan commented 1 year ago

@dmcke5 Please do. This will help to understand why it does not work - it should!

b-morgan commented 1 year ago

If I remember right, @foosel has an RPi camera V1 and an RPi camera V3 on her test rigs.

Draknars commented 1 year ago

Latest version not working for me on the following Configuration :

Image : https://github.com/OctoPrint/OctoPi-UpToDate/releases/tag/1.0.0-1.8.6-20230302093328. Imager: Raspberry Pi Imager 1.74

Hardware: Raspberry Pi 4B 4GB Storage: Samsung PRO Endurance 32GB Micro SDHC

I download the Image from GitHub, extract the .img from the zip file, Launch Raspberry Pi image and choose "Use Custom", setup Advance Settings, select the SD Card and then Image.

Once completed, I modify the octopi-wpa-supplicant.txt to add my Wi-Fi Settings. Finally I plug the SD Card and the Pi and turn it on.

image

image

Tried 3 times, and same result each time, it almost like OctoPi is never setup, based on the fact that there is no log directory

foosel commented 1 year ago

Does it work without changing the user name? Does rebooting help after changing the user name? Could be something up with the user-fix service, unrelated to the changes here.

Draknars commented 1 year ago

Does it work without changing the user name? Does rebooting help after changing the user name? Could be something up with the user-fix service, unrelated to the changes here.

Rebooting and updating didn't do anything, let me format again, without changing the username

APEbbers commented 1 year ago

Hi, I was able to get OctoPi working with one rpi camera and 1 usb camera on a RPI4 with OctoPi 1.0.0 and OctoPrint 1.8.7. I've uploaded a document with the steps I took to get it working for anyone interested. https://discord.com/channels/704958479194128507/708230099811434497/1081559132622028850

markaabo commented 1 year ago

Im new to pi and octoprint and cannot for some reason get the camera working, i did everything you said reflashed the sd card, checked the settings in libcam was correct and everything but it doesnt work, and when i do libcamera-hello it returns with Preview window unavailable [0:01:02.321712327] [1144] INFO Camera camera_manager.cpp:299 libcamera v0.0.3+40-9b860a66 [0:01:02.368431275] [1145] INFO RPI raspberrypi.cpp:1425 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media4 and ISP device /dev/media1 [0:01:02.368693384] [1144] INFO Camera camera.cpp:841 Pipeline handler in use by another process ERROR: *** failed to acquire camera /base/soc/i2c0mux/i2c@1/imx708@1a ***

I have the official pi cam 3, using a pi 4 model b with 4gb ram