Support for multiple cameras

TheSpaghettiDetective / obico-server

Obico is a community-built, open-source smart 3D printing platform used by makers, enthusiasts, and tinkerers around the world.

https://obico.io

GNU Affero General Public License v3.0

1.47k stars 298 forks source link

Support for multiple cameras #98

Open biocrypto730 opened 5 years ago

biocrypto730 commented 5 years ago

It would be great to support multiple cameras because:

Multiple cameras add increased security. This could be because one camera gets dislodged, a printhead blocks a spaghetti, or simply from a certain angle the A.I. can't recognize a spaghetti.
Multiple cameras would improve spaghetti detection algorithms! There's no standard way to record a spaghetti - more angles means more datas!
Some CoreXY printers with big beds have trouble getting the entire spaghetti in frame. The bed rises up and down, so ideally you attach the camera to the bed. But, attaching it right on the bed could mean you miss a part of the bed.

A lot of Octoprint plugins already exist for adding multiple webcams for your printer. It would be as easy as picking one of the plugins, and adding support for it in TSD.

Since there's already support for multiple printers, TSD is built with running multiple computer vision algorithms at the same time in mind.

I would suggest telling people to make sure their USB ports can handle an extra camera. 1 camera and 1 3D printer would probably max out a laptop's USB ports data range if the computer was made in the first half of the decade. Support for USB 3.0 on your laptop should be recommended, with a USB 3.0 hub plugged in to a USB 3.0 port.

kennethjiang commented 5 years ago

Thank you for opening this issue with great level of details! Will put this on our roadmap! In the mean time if anyone wants to give it a shot, please pitch in!

jimmyli56 commented 3 years ago

Are there any updates regarding the feature on the roadmap? I am also interested in having the feature in TSD as a better safeguard and redundancy in spaghetti detection.

kennethjiang commented 3 years ago

We haven't figured out a good solution to stream video from multiple webcams. Streaming from one webcam is tricky enough since we have to hit a balance between bandwidth, frame rate, resolution, and CPU usage.

One possible solution is to have video streaming on only 1 webcam (call it the main webcam). For the additional webcam, we only do 1 frame per 10s.

Will this solution be good enough for you?

marktoml commented 3 years ago

Speaking only for myself (and I did not raise the issue) another alternative might be to do a timed cycle through the available cameras.

jimmyli56 commented 3 years ago

@kennethjiang Yeah, that would definitely work for my setup, I have one camera that is mounted on the z-axis that looks at the extruder and I would designate that to be the main camera, my second camera overlooks the entire printer and print surface. I can see myself using it.

greenveg commented 3 years ago

This would definitely nudge me on the go-pro decision as well.

I would be quite happy with one stream at even 1fps, as long as the magic algorithm is running.

For example: I currently have a standard Raspberry camera module connected to the Z-axis constantly pointing at the extruder. I would like to add a camera at a different angle looking down at the whole print volume. This would be my 1fps feed.

jagr200 commented 3 years ago

Any update on this? I have 2 instances running controlling 2 minis. And TSD can't seem to manage the individual streams. Even though Octoprint can assign them to the correct instance. It changes to each camera every 3-4 seconds.

kennethjiang commented 3 years ago

The reason why we haven't had a good solution is different users seem to handle multi cameras in different ways. Can you elaborate on how you set up your Pis and webcams? @jagr200

jagr200 commented 3 years ago

The reason why we haven't had a good solution is different users seem to handle multi cameras in different ways. Can you elaborate on how you set up your Pis and webcams? @jagr200

Hi. Thanks for the response. I setup 2 instances of Octoprint and 1 camera for each instance on one Pi 4. I've been running 2 instances and 1 camera for over a year on the same Pi. Added the 2nd camera recently once THIS install method came out. That is the video I used to setup the updated instances to allow use with 2 cameras.

I have the TSD plugins running in each instance. They connect fine. Controls work. But the video for them changes between the 2 cameras. If I turn off one camera then the correct one is displayed for each TSD. I run TSD on my own private server as well.

kennethjiang commented 3 years ago

@jagr200 I haven't tried that method myself, but I believe you best bet is to disable premium streaming based on how the method works. :(

jagr200 commented 3 years ago

@jagr200 I haven't tried that method myself, but I believe you best bet is to disable premium streaming based on how the method works. :(

I'd obviously prefer to not do that. Because I do use a VPN to access them remotely so I can actually see the live webcam. Is there not a way to change which stream url each one uses? Just looking at the discord there are multiple people who would really love this feature. A 4b is capable of handling 2 machines and 2 cameras. And they aren't cheap at $60 just for the Pi4b 4gb.

kennethjiang commented 3 years ago

I can think about if it's feasible to do 2 premium webcam streams on a Pi 4. It may be a stretch though, especially in compatibility mode.

jagr200 commented 3 years ago

I can think about if it's feasible to do 2 premium webcam streams on a Pi 4. It may be a stretch though, especially in compatibility mode.

I've been running 2 machines and 1 premium stream off one Pi 4 4gb for over a year now without a single issue. If I can get the 2nd camera to work I have no doubts that it would handle it.

kennethjiang commented 3 years ago

@jagr200 I'm less confident on that. I spent a lot of time in early 2020 to optimize our premium streaming for the CPU load. Trust me it was not easy. If you have a Pi Camera it's a piece of cake as it costs minimum amount of CPU. But advanced streaming on USB camera is already cutting very close to overloading a Pi 3B+. And compatibility mode has to be streamed at a reduced frame rate, even on a Pi 4.