jeffbass / imagezmq

A set of Python classes that transport OpenCV images from one computer to another using PyZMQ messaging.
MIT License
1.01k stars 160 forks source link

Looking for Advice #47

Closed pageauc closed 8 months ago

pageauc commented 3 years ago

Jeff. Love your project. I am working on a project that uses multiple RPI 's with camera's could be webcams as well. The hub program starts and waits for the currently three sender RPI's to send an image. Each image is saved with a sequence number. The hub then sends each sender a timestamp for the next timelapse to be taken. This will be identical for each sender. The hub then does an image stitch to make a pano image and saves with sequence number then increments for next cycle. This is all working well although I had to make custom camera mounts out of foam board, small block of wood and a short dowel.

The problem I have is that the hub Must start before the senders. I want to automate the system so I was thinking of setting up a watch program on each sender that listens on another port. When the hub is started or restarted it sends each sender watch program a new python configuration file of settings eg resolution via zmq. Watch saves File as a config.py. Watch program then starts/restarts the sender program in background via subprocess Popen. Sender then restarts and reads the new config.py as an import. Still working on this.

Would it be possible to send a text file and return a confirming text message between the hub and sender rather than a jpeg and text per imagezmq. Currently I was looking at just using basic zmq commands to the hub and watch program on the senders but would be nicer if it was possible with imagezmq. Could not find this feature in imagezmq code but thought I would drop you a line.

Note I had to modify the https://github.com/ppwwyyxx/OpenPano c++ code to get it to accept an output file path for the pano jpg/png since it would only generate a fixed out.jpg named file in the same folder. My version is here https://github.com/pageauc/OpenPano. Created curl bash scripts for easier install. I could have used Adrians opencv image stitching but prefer the self contained openpano approach since users would not need the latest/greatest opencv contrib version.

FYI I have attached my camera holder template in pdf format. When the pipano project is ready I will post on my GitHub repo. Still a work in progress. There are lots of issues to work out. For one, the cropping of pano's is not consistent so doing a timelapse video would need some image stabilization during video editing or pruning some images. If lighting is stable then most of the pano's crop pretty consistently but low light can throw things off easily. I had to build the stands to allow accurate pointing. Don't want to use a pan/tilt mechanism because images would not be synchronized with same timestamp. Might add pano stitch to my robot as a feature since full view would be in one stitched image.

my pano stand. Currently have three on a board with various holes space around for stand dowels to fit into. Then rotate (pan) via dowel point and tilt via side lower hinge leg screws. This works quite well and easy to mount on a fence or even a vertical surface as long as you don't mind a small hole at mounting point

[cam-stand.pdf](https://github.com/jeffbass/imagezmq/files/5180357/cam-stand.pdf

Excuse me if I got a little chatty Claude ...

jeffbass commented 3 years ago

Hi Claude @pageauc, Glad you love my project. Thanks. Your project sounds really interesting. I'll definitely keep an eye on it on GitHub as you make progress.

Regarding your text sending question, I do use imageZMQ to send and receive text messages in a number of my projects, using the ImageSender.send_jpgmethod. The jpg_buffer can be any bytestring; it doesn't actually have to be a jpg. To use imageZMQ to send text messages only, I just set the jpg_buffer to a short bytestring in the sender. Then I receive and ignore that short bytestring in the hub that receives the text message. For example:

    # Sender that sends text messages
    # in the text sending program, where sender is an imageZMQ.ImageSender instance
    msg_text = 'Some message text.'
    hub_reply = sender.send_jpg(msg_text, b'0')  # set jpg_buffer parameter to a single byte

    # Receiver that receives the text messages
    # in the text receiving program, where hub is an imageZMQ.ImageHub instance
    msg_text, buf = self.recv_jpg()  # buf contains b'0', but just don't bother to use it

    #   or, another way to not use the jpg_buffer that is received.
    msg_text, _ = hub.recv_jpg()  # another way to receive and not use the jpg buffer

This code works well for me in a number of my projects. I do sometimes have to use the text.encode and byte.decode methods to avoid errors about text versus bytestrings in my message text:

    # convert bytes to Python 3 string:
    mst_text = msg_text_in_bytes.decode('utf-8')  # decode from bytes to Python 3 string

    # or, converting from Python 3 string to bytes:
    msg_text_in_bytes = msg_text.encode()  # convert Python 3 string to bytes to make ZMQ happy

Your method of using config.py text files and a watch program to control startup and options of your cameras versus the hub sounds similar to what I am doing with my imagenode and imagehub programs. I have a threaded function that watches for imagehub restarts (or other glitches) and it then restarts the imagenodes accordingly. To change the imagenode settings (there are many of those!), I copy a new imagenode.yaml text file to the imagenode and restart it. I use systemctl for starting and restarting my imagenodes using an imagenode.service file. You can see that in my imagenode repository here. My approach to the fact that hubs have to start before senders is to have all my imagenodes running continuously as systemctl services with the (Restart=always, RestartSec=5) options. Then, whenever the imagenode detects an imagehub restart, it simply exits. Then systemctl waits 5 seconds and restarts it.

Thanks for sending your RPi camera stand template. I'm going to give it a try. No need to apologize for being chatty. I love to hear about the projects that benefit from imageZMQ. I learn a lot from what others are doing. It is why I find GitHub so useful. Jeff

pageauc commented 3 years ago

Thanks for the quick reply. I will implement per you sample code. I was hoping I could use text instead of jpg data and now I know. I like the systemd service. I have used it on several other projects and the systemd script file can be pretty simple, just a [Unit], [Service], [Install] and a related bash script if req'd. Attached a small image of my 3 cam pano board. These are on old RPI 3's with no built in wifi but they work fine. Can be moved around. Thing1 is a sender and hub but might move hub to one of my RPI4's with mounted HD. Stitching would be faster. Also can use wireless ad-hoc network setup if in a remote location away from home wifi. IMG_1423

Note The lower hinge leg can be made shorter if necessary. Mine is higher to allow camera to be above a window frame when siting on the sill. Avoids having to use a spacer to raise assembly up.

Thanks for your help. Appreciated. Regards Claude.

jeffbass commented 3 years ago

I love your DIY hardware build. But I haven't seen those RPi cases before. It looks like the case holds both the RPi and the PiCamera in the same case? Where did you get that?

pageauc commented 3 years ago

I bought mine a few years ago but found similar RPI3 case on amazon at a good price.

RPI3, 3+ 2 case with built in camera mount https://www.amazon.com/Keyestudio-Supporting-Camera-Installation-Raspberry/dp/B076PQVMN2/ref=sr_1_2?dchild=1&keywords=raspberry+pi+case+with+camera+mount&qid=1599445678&sr=8-2

RPI 4 Case with built in camera mount https://www.amazon.com/LABISTS-Raspberry-Heatsink-Heatsinks-Supply/dp/B085GBCLYR/ref=sr_1_11?dchild=1&keywords=raspberry+pi+case+with+camera+mount&qid=1599445294&sr=8-11

We live in Canada but winter in Texas. This will be our 13 season Will be delayed this year due to covid issues in US. Probably go down possibly spring or summer next year. Hope you are doing well. Here is my YouTube channel if you are interested https://www.youtube.com/channel/UCybkT8zNfJUV9cGjLE-Fzsw

jeffbass commented 3 years ago

Thanks!

pageauc commented 3 years ago

Jeff. Hope you are doing well. I have the initial issue of my timelapse panorama project.

https://github.com/pageauc/panopi

Got panosend.py auto start and stop by using a panowatch.py program (runs in background using panowatch.sh). uses zmq port 5556 for communications. Using a RPI4 for panohub stitching (much faster than my older machines) Still a few items to implement. I plan to read the panosend.yaml settings from panohub.yaml (eliminates panosend.yaml). Also will auto change the hub IP address in the panosend yaml settings that are sent to panosend RPI's when the panohub is started. Will use logging lib rather than print statements. So far it seems to be working OK. Needs zeroconf since I am getting IP addresses from host names sockets. This helps if DHCP changes IP addresses. I think RPI's use zeroconf out of the box as far as I can tell. Made install easy using curl scripts. Documentation needs work and I plan to make a YouTube video on the project.

Stay Safe Regards Claude ...

pageauc commented 3 years ago

Just an update. I now read the panosend yaml settings from the panohub.yaml file in panohub folder. The panosend_settings are read and the ZMQ_PANOHUB_IP is dynamically updated. This avoids having to manually edit panohub.yaml setting and also auto adjusts if DHCP makes changes or you change the hub to another machine. I did this recently and all I had to do was install panohub onto new machine, start panohub.py and everything just worked since panowatch was running on all the panosend RPI's. New RPI4 hub can stitch three 720p images in a few seconds. Will relocate setup for more interesting scene and do a pano video from the pano images. I am going to do a pan tilt camera image stitching setup. I have a remote pan tilt camera setup in Texas running pi-timolo that sends images up to my google drive but it would be a lot nicer to just send one pano image instead of multiple pan timelapse images. I will also add feature to my robot since it can rotate as well as pan camera. It would be possible to get 360 + view or at least two 180 + pano's if robot rotation is a problem with stitching.

There are lots of enhancements possible but may just try to keep it simpler and do multiple simpler projects. My PI-TIMOLO and SPEED CAMERA projects grew in complexity and feature creep but lots of people like these projects.

Thanks for your assistance Stay Safe Claude .....

jeffbass commented 3 years ago

Hi Claude, Thanks for these updates. It sounds like you are making great progress. I am following your work with interest because I plan do panoramic multi-camera wildlife tracking to my own projects in the future. Thanks for sending links to your videos, too. I've been enjoying watching them, especially the ones about your PI-TIMOLO, MOTION_TRACK and SPEED_CAMERA projects. Looking forward to continued updates! Thanks, Jeff

pageauc commented 3 years ago

Thanks Jeff

I appreciate your help and support. Today I brought the PANOPI camera setup upstairs and put it outside on the window sill. I used a powered hub and 1 ft usb cables for a clean setup that is easy to move. The new version reads all settings from the panohub.yaml. This is my first project using yaml files. Had to put my thinking cap on to figure out how to dynamically edit the ZMQ_PANOHUB_IP variable from the panosend_settings and transmit the stream to panowatch.py. This means you can easily change hub machines and not have to worry about editing the zmq ip address. I am thinking of adding a motion tracking feature on one of the senders that would trigger the pano image sequence. Also added image resolution rounding so picamera does not generate warning. Also updated my PI-TIMOLO to use image resolution rounding and implemented change to reduce cpu usage when only timelapse mode is used per a request by a user.

I had fun with this project. I want to do a howto YouTube video and also generate a PANO Video from the full size pano images. Looks like there is consistent cropping if there is good lighting and stitch points. I want to try different resolutions. 1080p stitches well due to higher resolution so overlap can be narrower. Tomorrow I am going to try a 1920 x 720 resolution. A combination of 1080p width and 720p height or may try the max camera width and 720p..

Had a slight issue with two of the panosend camera's. The images were not very good and I had to clean the camera lenses. Used a blunted tooth pick (cut with scissors) with lens cleaning cloth over it. Might try a bit of lense cleaner solution next time. Canned air did not work but physical cleaning worked ok. Will try another cleaning tomorrow. Will do some research to see if anyone has a different way. The camera pin hole lense opening is pretty small and I would not want to damage a perfectly good camera module but my cleaning method improved things a lot as confirmed by my wife.

I think my speed camera needs cleaning after looking at the images. Seems a little foggy.

Stay Safe and Healthy Regards Claude .

On Fri, Sep 25, 2020 at 6:33 PM Jeff Bass notifications@github.com wrote:

Hi Claude, Thanks for these updates. It sounds like you are making great progress. I am following your work with interest because I plan do panoramic multi-camera wildlife tracking to my own projects in the future. Thanks for sending links to your videos, too. I've been enjoying watching them, especially the ones about your PI-TIMOLO, MOTION_TRACK and SPEED_CAMERA projects. Looking forward to continued updates! Thanks, Jeff

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jeffbass/imagezmq/issues/47#issuecomment-699190403, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNPKZGY4GDPBTBRZCT7PYDSHULDDANCNFSM4Q47CBHQ .

-- See my YouTube Channel at http://www.youtube.com/user/pageaucp

jeffbass commented 8 months ago

Hi Claude, Thanks for your continued updates. Sounds like you continue to make good progress. Le me know if you do a howto YouTube video about it. Jeff