search

mainsail-crew / crowsnest

Webcam Service for multiple Cams
GNU General Public License v3.0
324 stars 77 forks source link

Arudcam 64MP #51

Closed TheGameXIV closed 1 year ago

TheGameXIV commented 2 years ago

Is your feature request related to a problem? Please describe

Im not sure, due to my limited understanding of the issue. My request is i remember earlier in the year that the Arducam was not supported due to the proprietary kernal module. I was curious if this was still unsupported, or a workaround has been found?

Describe the solution you'd like

A way to use the Arudcam 64MP camera with crownsnest/mainsail in any form.

Describe alternatives you've considered

I am personally okay with not being able to use the full feature set of the camera as long as it would be functional.

Additional information

No response

KwadFan commented 2 years ago

At this Stage, there will be no support for Arducams.

The reason is that they, at the point of my research, are using "proprietary" Drivers and relie on an extension of mjpg-streamer, which isnt supported by crowsnest and never will again.

I am about to switch the backend, but if that will support that kind of camera I couldnt tell. I will provide as much support as I can for devices that gets into the upstream of the Kernel / OS Sources. But in my experience mostly Arducam does'nt upload to upstream, so chances are small.

EDIT: To be sure I did some research again, and it seems at least its overlay found its Way into the 5.15 kernel. https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/ by using dtoverlay=arducam-pivariety How well this will work with the needed legacy-camera stack I could not tell.

Therefor you are a bit on your own for now. But I would implement workarounds if you do the tests and documentation for it. Regards Kwad

Kcdc3d commented 2 years ago

I use this on a separate raspi to directly access my arducam and stream 30fps to mainsail by using ISP: https://github.com/ayufan-research/camera-streamer Seems to have some updates in the past months for webrtc as well, not sure if it is of any benefit for this, but thought it may spring an idea.

TheGameXIV commented 2 years ago

I use this on a separate raspi to directly access my arducam and stream 30fps to mainsail by using ISP: https://github.com/ayufan-research/camera-streamer Seems to have some updates in the past months for webrtc as well, not sure if it is of any benefit for this, but thought it may spring an idea.

That may be a way. Is there any technical reason it needs to be done a separate raspi, vs the one mainsail runs on?

TheGameXIV commented 2 years ago

At this Stage, there will be no support for Arducams.

The reason is that they, at the point of my research, are using "proprietary" Drivers and relie on an extension of mjpg-streamer, which isnt supported by crowsnest and never will again.

I am about to switch the backend, but if that will support that kind of camera I couldnt tell. I will provide as much support as I can for devices that gets into the upstream of the Kernel / OS Sources. But in my experience mostly Arducam does'nt upload to upstream, so chances are small.

EDIT: To be sure I did some research again, and it seems at least its overlay found its Way into the 5.15 kernel. https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/ by using dtoverlay=arducam-pivariety How well this will work with the needed legacy-camera stack I could not tell.

Therefor you are a bit on your own for now. But I would implement workarounds if you do the tests and documentation for it. Regards Kwad

Yea i saw that it made it into the 5.15 kernel, so I was curious to come back and check to see if it works. I'm going to see what I can do to make it work. If anything manages to work ill be sure to let you know along with documenting the steps taken.

Kcdc3d commented 2 years ago

I use this on a separate raspi to directly access my arducam and stream 30fps to mainsail by using ISP: https://github.com/ayufan-research/camera-streamer Seems to have some updates in the past months for webrtc as well, not sure if it is of any benefit for this, but thought it may spring an idea.

That may be a way. Is there any technical reason it needs to be done a separate raspi, vs the one mainsail runs on?

I was getting crashes since I was also using two USB cams, the 4B wasn't enjoying having it's USB being used this way, with or without a powered hub, so I decided to just make a separate raspi cam server, take the load off my klipper raspi. biggest benefit is solid FPS gains, 30fps constant on all. the downsides with that specific method is manually needing to adjust focus with 12ctools, not a huge issue, but I can't seem to get the settings to stick after a restart. I haven't used it in a long time, so I am curious if the updates did anything.

KwadFan commented 2 years ago

I use this on a separate raspi to directly access my arducam and stream 30fps to mainsail by using ISP: https://github.com/ayufan-research/camera-streamer Seems to have some updates in the past months for webrtc as well, not sure if it is of any benefit for this, but thought it may spring an idea.

I am about to switch the backend, ...

Guess which one ;)

TheGameVIX commented 2 years ago

Let me guess...the one in the link....haha

Awesome let me know when that's live and I'll give it a test tonight.

KwadFan commented 2 years ago

Let me guess...the one in the link....haha

Awesome let me know when that's live and I'll give it a test tonight.

I will let you know when I have something thats worth to test. Testers are always welcome.

KwadFan commented 2 years ago

@TheGameVIX Well, it is kinda "rudimental" right know and I have to figure out a bunch off stuff, but .... At least its working, somehow :man_shrugging:

Use https://github.com/mainsail-crew/crowsnest/tree/feat/ayucamstream branch

TheGameVIX commented 2 years ago

Will do, I should hopefully have a update for you by end of the week. Thanks for the branch.

jinnn1989 commented 1 year ago

@Kcdc3d can you provide me details on how did you focus on your 64 mp arducam. A little depth guide would help me.

I have 16 mp ardu cam and not able to focus using the commands provided on AYUFAN camera_streamer git

AMFkuna commented 1 year ago

@TheGameVIX Well, it is kinda "rudimental" right know and I have to figure out a bunch off stuff, but .... At least its working, somehow 🤷‍♂️

Use https://github.com/mainsail-crew/crowsnest/tree/feat/ayucamstream branch

I cloned and installed this branch, but still NO SIGNAL How do i force loading cstreamer instead of ustreamer?

KwadFan commented 1 year ago

@TheGameVIX Well, it is kinda "rudimental" right know and I have to figure out a bunch off stuff, but .... At least its working, somehow man_shrugging Use https://github.com/mainsail-crew/crowsnest/tree/feat/ayucamstream branch

I cloned and installed this branch, but still NO SIGNAL How do i force loading cstreamer instead of ustreamer?

Sorry for the late response, please use mode: multi to use cstreamer instead

AMFkuna commented 1 year ago

@TheGameVIX Well, it is kinda "rudimental" right know and I have to figure out a bunch off stuff, but .... At least its working, somehow man_shrugging Use https://github.com/mainsail-crew/crowsnest/tree/feat/ayucamstream branch

I cloned and installed this branch, but still NO SIGNAL How do i force loading cstreamer instead of ustreamer?

Sorry for the late response, please use mode: multi to use cstreamer instead

Thank for reply, cstreamer started, but how can I force camera-streamer-arducam-16MP.service?

Kcdc3d commented 1 year ago

@Kcdc3d can you provide me details on how did you focus on your 64 mp arducam. A little depth guide would help me.

I have 16 mp ardu cam and not able to focus using the commands provided on AYUFAN camera_streamer git

Hi there, apologies for being behind on this response, I am not sure if you've found another route. I used camera-streamer (it has been a while since it's been up) and at first it was a tad wonky, but it was basic once you got the hang of it, if you have a signal, then the hardest part part is done, Hopefully you're using the CSI hardware or else it won't be as worth it. now you just need to automate it to start when the raspi does - many ways to do this, and I found this to be the simplest (by installing Screen, lightweight, does what it's supposed to, thats it): https://hackaday.io/project/167713-bstya-20-work-in-progress/log/193961-dual-stream-setup-how-to

This may help you with ustreamer as well (i ran both on the same pi400 for the arducam, a Razer Kiyo Pro and both were getting 30fps thanks to hardware encoding 1080p on the razer and i forget the ardu native res.. Cstreamer not so good for usb cam), But you can skip to the automation procedure closer to the end. If you need help there, I am trying to dig up my old script, but it should make sense if you've come this far.

So the only annoying issue is setting the focus, I tried to add the commands to the autostart script, but nothing worked, I am no master, so I may have been using the wrong commands, but essentially you need i2ctools installed at root. the rest of this is pulled from cstreamer https://github.com/ayufan/camera-streamer:

"

after starting camera execute to control the focus with 0xXX, any value between 0x00 to 0xff

RPI02W (and possible 2+, 3+):

i2ctransfer -y 22 w4@0x0c 0x0 0x85 0x00 0x00

RPI4:

i2ctransfer -y 11 w4@0x0c 0x0 0xXX 0x00 0x00

"

For me that took a lot of fumbling, trying to figure out the right combination, but once I did, I copied it down and every time I needed to refocus or restart, I ran that command at start. If I find my old scripts, I will send them over.

I am currently bringing my printer back online after a few months break, I got too obsessed, but it's xmas time and need to make gifts and finish these speaker projects, so I may be more helpful now. Best of luck!

jinnn1989 commented 1 year ago

Camera-streamer does not work with Crowsnest. I managed to get it working with 16 mp Arducam, with manual focus.

we can create a dscord channel to discuss this on fast pace basis and can help you get it working. let me know if you would like to go ahead this way.

AMFkuna commented 1 year ago

[cam 1] mode: multi # mjpg/rtsp port: 8080 # Port device: /dev/video0 # See Log for available ... resolution: 2328x1748 # widthxheight format max_fps: 15 # If Hardware Supports this it will be forced, ohterwise ignored/coerced. custom_flags: -camera-path=/base/soc/i2c0mux/i2c@1/imx519@1a -camera-type=libcamera -camera-format=YUYV -camera-nbufs=2 -camera-high_res_factor=1.5 -camera-low_res_factor=3.0 -camera-options=brightness=0.1 -camera-options=sharpness=2

This works to my satisfaction for Arducam 16MP.

TheGameVIX commented 1 year ago

Camera-streamer does not work with Crowsnest. I managed to get it working with 16 mp Arducam, with manual focus.

we can create a dscord channel to discuss this on fast pace basis and can help you get it working. let me know if you would like to go ahead this way.

Would this also work witht the 64MP arducam?

Kcdc3d commented 1 year ago

Was just answering the persons question over arducam since it doesnt work with your software and that was the only way I got it to work without it hanging at like 10fps. I'll see myself out.

jinnn1989 commented 1 year ago

Camera-streamer does not work with Crowsnest. I managed to get it working with 16 mp Arducam, with manual focus. we can create a dscord channel to discuss this on fast pace basis and can help you get it working. let me know if you would like to go ahead this way.

Would this also work witht the 64MP arducam?

most probably it should work on 64MP cam aswell. Afte lots of months, I finally managed to work.

TheGameVIX commented 1 year ago

Camera-streamer does not work with Crowsnest. I managed to get it working with 16 mp Arducam, with manual focus. we can create a dscord channel to discuss this on fast pace basis and can help you get it working. let me know if you would like to go ahead this way.

Would this also work witht the 64MP arducam?

most probably it should work on 64MP cam aswell. Afte lots of months, I finally managed to work.

If you have any availability and would be willing. I would love to jump on a discord channel to get this working on my end.

jinnn1989 commented 1 year ago

please join the below discord

https://discord.gg/udsA5Pwy

jinnn1989 commented 1 year ago

@TheGameVIX jon the server I will be available for next 4-5 hours for assistance

TheGameVIX commented 1 year ago

@TheGameVIX jon the server I will be available for next 4-5 hours for assistance

Oh my apologies, I won't be available until 7-8 PM EST as I am currently at work. I will still join the discord if you have any availability later this evening or another day. Thank you for your assistance though.

viappidu commented 1 year ago

@jinnn1989 I got the 16MP working with crowsnest :) @TheGameVIX for the 16MP the only real problem seemed to be the installation of Arducam version of libcamera following https://docs.arducam.com/Raspberry-Pi-Camera/Native-camera/Libcamera-User-Guide/ For Crowsnest use the branch "feat/ayucamstream". 2c instructions, clone the repo, REMEMBER TO CHECKOUT to the correct branch, run the tools/config.sh and follow the instructions Just finished the install so no tests/extra configuration other than seeing it working in mainsail after restart.

My kernel is 5.15.76-v7l+ #1597 SMP Fri Nov 4 12:14:58 GMT 2022 armv7l GNU/Linux Added these lines to /boot/config.txt: dtoverlay=imx519 dtoverlay=imx519,cam0

I did try once few months ago and (at the time) I found instructions saying to use... dtoverlay=imx519,media-controller=0 ...though it never worked. Not sure now...

jinnn1989 commented 1 year ago

@jinnn1989 I got the 16MP working with crowsnest :) @TheGameVIX for the 16MP the only real problem seemed to be the installation of Arducam version of libcamera following https://docs.arducam.com/Raspberry-Pi-Camera/Native-camera/Libcamera-User-Guide/ For Crowsnest use the branch "feat/ayucamstream". 2c instructions, clone the repo, REMEMBER TO CHECKOUT to the correct branch, run the tools/config.sh and follow the instructions Just finished the install so no tests/extra configuration other than seeing it working in mainsail after restart.

My kernel is 5.15.76-v7l+ #1597 SMP Fri Nov 4 12:14:58 GMT 2022 armv7l GNU/Linux Added these lines to /boot/config.txt: dtoverlay=imx519 dtoverlay=imx519,cam0

I did try once few months ago and (at the time) I found instructions saying to use... dtoverlay=imx519,media-controller=0 ...though it never worked. Not sure now...

thanks for the info. if you could join the discord channel, I would like your help to get this working with crowsnest.

we need experts like to help other people out there who are unaware about this. so lets going together to help them.

I do have some issues as well, maybe we together can sort this out

jinnn1989 commented 1 year ago

@viappidu still not working for me. I thought i had missed some steps, but its the same what i did earlier but still not working as per your instructions.

viappidu commented 1 year ago

@jinnn1989 sorry for long delay. It turned out that some tinkering was necessary. I managed to get it through even if it's still not convincing me. I'll open a new issue for what I found... It did work initially as I did enable the $PWD/crowsnest/bin/camera-streamer/service/camera-streamer-arducam-16MP.service. Like this crowsnest became practically useless... So, the trick was to copy the values from the above services directly into the crowsnest.conf value. Simple (lol, not really) as that! I copy below my (WORKING) .conf (thank you for the -camera-options=lensposition=360 tip ;))

[crowsnest]
log_path: ~/klipper_logs/crowsnest.log
log_level: quiet
delete_log: true

[cam Arducam_16MP]
mode: multi
port: 8080
device: /base/soc/i2c0mux/i2c@1/imx519@1a
max_fps: 30
resolution: 1920x1080
#v4l2ctl:                               # Add v4l2-ctl parameters to setup your camera, see Log what your cam is capable of.
custom_flags: -camera-options=rotation=90 -camera-type=libcamera -camera-format=YUYV -camera-nbufs=2 -camera-high_res_factor=1.5 -camera-low_res_factor=3.0 -camera-options=brightness=0.2 -camera-options=sharpness=2 -camera-options=lensposition=360

[cam Endoscope]
mode: multi
port: 8081
device: /dev/video0
max_fps: 30
resolution: 640x480
#v4l2ctl:                               # Add v4l2-ctl parameters to setup your camera, see Log what your cam is capable of.
custom_flags: -camera-path=/dev/video0 -camera-format=YUYV

The "not convincing" (right wrong) part is the device path wich is actually /sys/firmware/devicetree/base/soc/i2c0mux/i2c@1/imx519@1a but if I put the right path it does not work. I tried few options if the /dev folder without success. The only way I had it work was that. I also get a (lovely) crowsnest: WATCHDOG: Lost Device: '/base/soc/i2c0mux/i2c@1/imx519@1a' log entry which makes sense altogether...

BIG NOTE: My 'Endoscope' camera is a USB (endoscoping yep) camera which goes up in /dev/video0. I think without that one installed the device of the arducam might be simpler. Maybe changing the dtoverlay in the boot config might get to different results, I'll have a look at it and I'll post my findings here.

Let me know if this helps out

viappidu commented 1 year ago

"Maybe changing the dtoverlay in the boot config might get to different results, I'll have a look at it and I'll post my findings here."

Had a quick look at https://github.com/ArduCAM/IMX519_AK7375 I can't say I understand much there. Maybe someone else does...

viappidu commented 1 year ago

RE camera path I opened #67

jinnn1989 commented 1 year ago

@viappidu Thanks for the config file.

my issue here is different, when I use it directly using camera streamer, I do get video and timelapse working, but sometimes the stream does not pickup the correct frame. due to which the timelapse does not look good.

the feed is very very good, but some times there is a lag even at 15 FPS - which drosp to 3-5 frames, while my cpu load is around 3-5 % and Memory around 25-30%.

jinnn1989 commented 1 year ago

@viappidu thank you so much. finally this config works for me.

viappidu commented 1 year ago

my issue here is different, when I use it directly using camera streamer, I do get video and timelapse working, but sometimes the stream does not pickup the correct frame. due to which the timelapse does not look good.

This is a totally different issue. I'm not using timelapse at the moment though I've noticed plenty of issues there looking like yours. I'm going now to check the code / open an issue at ayufan/camera-streamer as I think the path "weirdness" is, in some way, to be addressed there.

KwadFan commented 1 year ago

I couldnt test this fully, but I think the new backend will solve this issue.

Regards Kwad