Pi Zero 2 W camera for equipment monitoring

[sgbaird]

Need to order tripod, pi zero case, and tripod nut.

Add documentation page to devices dir (aside: need to expose device pages)

Possible setups:

• YT live streaming
• MQTT "point and shoot"
• other streaming/camera feed/API access (Kafka?)

Possible demo extensions:

• HeinSight
• observing linear actuator (picture OK), fan (livestream, zoomed in), MyCobot 280, capper/decapper (picture or video OK), Jubilee system, OT-2, microscopes
• Custom tool on Jubilee

Related: OpenMV demo.

[sgbaird]

Also, setting up the tripod and case.

There's a second case type, with no tripod attachment

[sgbaird]

I ordered ball bearing attachments for the prime cables clamp tripods.

The additional four should be set up in a similar manner. There's one extra in the cabinet (might move).

[sgbaird]

For the setup, also important to have a micro USB to female USB-A cable (comes in the pi zero 2 W kits), and a keyboard/mouse combo (however in this case maybe you need two USB ports or a USB splitter)

For the camera mount, a small Phillips screwdriver works

Mount item and instructions:

https://www.pishop.ca/product/pro-mini-camera-mount-for-raspberry-pi-zero/

https://thepihut.com/blogs/raspberry-pi-tutorials/pro-mini-camera-mount-assembly-guide

[sgbaird]

@Jonathan-Woo any updates?

[Jonathan-Woo]

https://github.com/user-attachments/assets/75b5787e-c508-45b6-beb6-4ccbc051d9f6

I have the hardware setup but I think we need a smaller nut to connect to the tripod or PrimeCables mount. The current one is too thick so the standoffs can't sit flush on the case.

On the software side of things, I believe we'll building using picamera2. What sort of MWE/system would you like me to build?

[sgbaird]

Nice! Ok, can you identify a nut (maybe on Amazon) that you think will work OK? For the software, it would be great if you could try to get a YouTube livestream set up with it. I'll send credentials for an "AC hardware stream" account, or you can use a personal account for testing.

Before that, following a tutorial to get the video feed displaying on the desktop of the Pi Zero 2W would be a good start (and a really short video demo of it from your phone would be nice, too)

EDIT: and make sure to invert the camera feed with the software (specific to this camera mount)

[Jonathan-Woo]

Sounds good, I'll look into that. Regarding the video feed displaying on the desktop of the Pi Zero 2W, I was wondering if there was something specific you wanted differently than the attached video above. Thanks!

[sgbaird]

Oh, sorry! Somehow I missed that. That looks great. I'm also seeing that the PDF you linked to is very comprehensive. Anything in particular you'd want to note about following that guide? E.g., certain install commands you used. It would be great if you could also include the code you ran to produce the preview.

Past that, I'd say you're good to move onto setting up a livestream as well some of the other camera modules (the latter of which I imagine will be straightforward).

Thanks for working on this.

[sgbaird]

Sent you a stream key and the stream URL attached to https://www.youtube.com/@ac-hardware-streams channel.

[sgbaird]

How is https://www.amazon.ca/Stainless-Small-Pattern-Machine-Finish/dp/B009EEZYE0/ ?

[Jonathan-Woo]

Yeah that looks good, 1/4"

[sgbaird]

The item arrived from amazon

[sgbaird]

Accidentally got something that was 1/4" outside dimensions.. planning to order a new part.

[Jonathan-Woo]

I've recently tested various resolutions, bitrates, and framerates and the best I've been able to do is 720p @30fps with a bitrate of 4 Mb/s. Anything more demanding and we get dropped frames because the Pi Zero can't encode fast enough and we get dropped frames.

This is in line with youtube's recommendations.

Something to note is that sometimes immediately restarting the stream fails. I'm not too sure why. I suspect it has to do with something on youtube's side where they have a cooldown period between streams.

I've been using the built in command line tool with the below command. I might be overdoing it with the thread_queue_size.

rpicam-vid -t 0 -o - --width 1280 --height 720 --vflip --codec libav --libav-format mpegts --bitrate 4000000 --framerate 30 | ffmpeg -re -f s16le -thread_queue_size 4096 -i /dev/zero -thread_queue_size 4096 -i - -vcodec copy -framerate 30 -f flv rtmp://a.rtmp.youtube.com/live2/STREAM_KEY

Regarding next steps, do you think we should pack this command into something friendlier as in a bash script. I can also get started on assembling the other camera modules if they're ready.

[sgbaird]

Ok, nice! Not a huge deal if the fps is lower (e.g., 10-20), and overall that seems good. Thanks for trying out different things to get it working. If frames drop, how does that affect the user? Weird lag / skip, or something more dramatic? Weird about the immediate restart. If you have it in a bash script, it would be easier to break over multiple lines and add comments, but it's up to you.

[sgbaird]

It looks like I may be able to create a shared YouTube channel, in which case I could add you and others as managers.

This might replace the account attached to https://www.youtube.com/live/g6IhiD4XEMU?si=rAw4ZNUmQB0fNuR6

[Jonathan-Woo]

@sgbaird Any update on the account?

[sgbaird]

Sent the credentials for an account with editor access to the account via slack. Turns out invites can be made for regular YouTube accounts with granular permissions.

[sgbaird]

I'm not sure if I brought this up already, but it would be great to also see an example of the YouTube video embedded into a hugging face space: https://ac-training-lab.readthedocs.io/en/latest/devices/setup_iolt_devices.html#hugging-face-spaces.

[sgbaird]

Added you to HF. Could you create both a standalone example of embedding a YT video as well as incorporating the same camera stream into the MyCobot app (as a second step)?

[sgbaird]

@Jonathan-Woo the nuts should be in tomorrow. Ended up ordering from McMaster, since Amazon randomly cancelled and refunded the order.

[sgbaird]

@Jonathan-Woo it says the hex nuts arrived. They should be in a McMaster package if not already opened. Feel free to open it.

[Jonathan-Woo]

Updates on the Streamlit HF side of things, I had trouble earlier with pushing but that has been resolved now.

I've pushed changes to embed the youtube livestream in the cobot space and a new space I intend to contain all the streams.

[Jonathan-Woo]

I've also modified the /etc/xdg/lxsession/LXDE-pi/autostart file according to this tutorial enabling the autostart of GUI apps which rpicam-vid requires to have a preview show up on the desktop.

crontab does not support gui apps.

[sgbaird]

@Jonathan-Woo great to see that the livestream is up: https://www.youtube.com/live/fF4zEp6LSkg

Yesterday, I think you ran into some issues. Do you mind summarizing here and what you did to resolve?

As next steps (separate gh issue perhaps), maybe we can look at the programmatic access of the stream data. E.g., some kind of python API for bringing in the video data.

[Jonathan-Woo]

The command I used originally displayed the feed full screen on the rpi desktop. I had to update the stream key but wasn't able to because the Pi Zero was struggling to display the feed while streaming to youtube.

So I had to modify the launch script to make it no longer display fullscreen on the desktop. The issue was that since the script auto-started and the Pi Zero was already under maximum load I couldn't stop the full screen feed. So disconnected the camera to cause the script to crash when auto starting so I could modify it before restarting the rpi.

[sgbaird]

Thanks! Also, any ideas why the HF space embedding isn't displaying properly?

A workaround might be to use HTML embed code directly (i.e., what's copy pasted from the YouTube share - embed option). Maybe this would be HTML within a markdown block. Maybe some issue with the video block, which honestly might just be a really shallow wrapper.

[sgbaird]

Also, could you add a Sprite of the camera that's currently set up? Maybe with the camera and the mount. Instructions at: https://ac-training-lab.readthedocs.io/en/latest/devices/setup_iolt_devices.html#embedding-into-gather-town. You could embed the YouTube video directly if you want (Gather Town supports this I think) or embed the HF space.

[sgbaird]

Also, the livestream seems to have cut out 8 days ago:

Looks like the duration was 36:40:08.

I went ahead and created a new livestream with "reuse settings" with the previous one (which I believe preserves the stream key as the same).

Doesn't seem to be getting data back.

[Jonathan-Woo]

Hi Sterling,

So I've been working on a significant overhaul of the device code in src/ac-training-lab/picam. The motivation is that the current system relies on a single command line call which is susceptible to lots of errors such as:

• starting too early before internet is connected
• difficult to modify stream keys
• difficult to sync code versions across devices

So to address these issues I've switched to using a parent python script to handle:

1. Downloading the latest version of the script on device reboot
2. Executing the downloaded script
3. Checking for internet connection before streaming
4. Improved preview efficiency by using QtGL
5. Cleaner way to update stream keys

My current direction is to have the code autostart the stream using a cached stream key. This means that to update the stream key the user would have to interrupt the start procedure, update the cached stream key, and then reboot. To make this smoother I considered setting up a system where rpis could query for stream keys from a DB keyed by the device serial number so we could centralize which device would pertain to which stream key instead of going in and updating the stream keys for each device. However, I thought this might overcomplicating things. I considered putting the stream keys in a json and having the rpis wget the serial number to stream key json automatically but that would expose all the stream keys which would be less than ideal haha.

For this undertaking I've had to implement all the preview and streaming functionality using picamera2 so there are still some issues I'm ironing out especially with ffmpeg.

This should be a much more robust system and I'm hoping to get the streams back up ASAP.

Will address the HF spaces and gathertown issues ASAP.

Also, any ideas for where to setup the other cameras? I was thinking one for the microscope and one for the OT-2?

[sgbaird]

Thanks for digging into this so deeply. I think it will be worth it to spend the time for a robust and replicable solution. One for MyCobot, one for microscope, and one for OT-2 sounds good. There are a number of other devices with motion that would probably go in the same area with an overhead cam. If there's a 4th, this can go into the room with the 3D printer, but it doesn't have to point to anything in particular to start with, but still positioned in a way to avoid catching faces by accident.

For the stream key storage, that does seem tough. One MongoDB user per camera with permissions only to the corresponding stream key is possible, but i think since YT allows you to reuse stream keys (i.e., "reuse settings" option), then maybe it would be OK to have the cached version and document the switchover instructions. Some alternatives: maybe there's a way to add a file to the SD card directly that could be read by the OS. I don't know if this is feasible. I'm not sure if there would be a way to auto start that would allow you to connect a monitor and keyboard and stop the auto-running script to then update the stream key.

Sounds like you're on a good path though.