search

stephendade / Rpanion-server

Web-based configurator for companion computers of MAVLink vehicles
https://www.docs.rpanion.com/software/rpanion-server
GNU General Public License v3.0
146 stars 62 forks source link

Can't access video WebGUI #234

Closed lida2003 closed 5 months ago

lida2003 commented 5 months ago 
daniel@rpanion:~/Rpanion-server $ git log -n 1
commit daba5a36bc361e8ddc4afe8c368b665898251d01 (HEAD -> I420Pi, origin/I420Pi)
Author: Stephen Dade <stephen_dade@hotmail.com>
Date:   Thu Jun 6 20:41:51 2024 +1000

    Video: Change to I420 format for Pi cam
daniel@rpanion:~/Rpanion-server $ git status
On branch I420Pi
Your branch is up to date with 'origin/I420Pi'.

nothing to commit, working tree clean

图片

2024-06-13 05:52:53 info [server/index.js]: NTRIP stopped
2024-06-13 05:51:09 info [server/index.js]: Express server is running on localhost:3000
2024-06-13 05:51:09 info [server/index.js]: Opening Link /dev/ttyUSB0 @ 921600, MAV v2
2024-06-13 05:51:09 info [server/index.js]: Opened Router
2024-06-13 05:51:09 info [mavlink/mavManager.js]: Vehicle is S/C: 1/1
2024-06-13 05:51:09 info [mavlink/mavManager.js]: 4.5.3-official
2024-06-13 05:51:16 info [server/index.js]:
2024-06-13 05:51:16 error [server/networkManager.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [server/aboutInfo.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [mavlink/mavManager.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [server/networkClients.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [server/cloudUpload.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [server/vpn.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [server/logConverter.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
2024-06-13 05:51:16 error [server/index.js]: uncaughtException: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /home/daniel/Rpanion-server/server/videostream.js:78:29
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
daniel@rpanion:~/Rpanion-server $ python --version
Python 3.9.2
daniel@rpanion:~/Rpanion-server $ python3 ./python/gstcaps.py

(gstcaps.py:869): GStreamer-CRITICAL **: 05:58:03.242: gst_element_message_full_with_details: assertion 'GST_IS_ELEMENT (element)' failed
Segmentation fault
lida2003 commented 5 months ago

Remove log and reboot the device

$ sudo systemctl stop rpanion.service
$ rm logs/app.log
$ sudo reboot

refresh video web page, and check camera on cmd line:

图片

daniel@rpanion:~/Rpanion-server $ cat logs/app.log
2024-06-13 06:07:49 info [server/index.js]: NTRIP stopped
2024-06-13 06:07:50 info [server/index.js]: Express server is running on localhost:3000
2024-06-13 06:07:50 info [server/index.js]: Opening Link /dev/ttyUSB0 @ 921600, MAV v2
2024-06-13 06:07:50 info [server/index.js]: Opened Router
2024-06-13 06:07:51 info [mavlink/mavManager.js]: Vehicle is S/C: 1/1
2024-06-13 06:07:51 info [mavlink/mavManager.js]: 4.5.3-official
2024-06-13 06:10:00 info [server/index.js]: [{"value": "testsrc", "label": "Test Source", "caps": [{"value": "1920x1080xx-raw", "label": "1920x1080", "height": 1080, "width": 1920, "format": "video/x-raw", "fpsmax": "30", "fps": []}, {"value": "1280x720xx-raw", "label": "1280x720", "height": 720, "width": 1280, "format": "video/x-raw", "fpsmax": "30", "fps": []}, {"value": "640x480xx-raw", "label": "640x480", "height": 480, "width": 640, "format": "video/x-raw", "fpsmax": "30", "fps": []}]}]

2024-06-13 06:10:00 info [server/index.js]: [object Object]
daniel@rpanion:~/Rpanion-server $ libcamera-hello --list-camera
Available cameras
-----------------
0 : ov5647 [2592x1944] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 [58.92 fps - (16, 0)/2560x1920 crop]
                             1296x972 [43.25 fps - (0, 0)/2592x1944 crop]
                             1920x1080 [30.62 fps - (348, 434)/1928x1080 crop]
                             2592x1944 [15.63 fps - (0, 0)/2592x1944 crop]
daniel@rpanion:~/Rpanion-server $ python3 ./python/gstcaps.py

(gstcaps.py:918): GStreamer-CRITICAL **: 06:14:19.193: gst_element_message_full_with_details: assertion 'GST_IS_ELEMENT (element)' failed
[{"value": "testsrc", "label": "Test Source", "caps": [{"value": "1920x1080xx-raw", "label": "1920x1080", "height": 1080, "width": 1920, "format": "video/x-raw", "fpsmax": "30", "fps": []}, {"value": "1280x720xx-raw", "label": "1280x720", "height": 720, "width": 1280, "format": "video/x-raw", "fpsmax": "30", "fps": []}, {"value": "640x480xx-raw", "label": "640x480", "height": 480, "width": 640, "format": "video/x-raw", "fpsmax": "30", "fps": []}]}]

Camera is right there, but the code didn't pick up the camera.

devices = device_provider.get_devices() in ./python/gstcaps.py gives error.

lida2003 commented 5 months ago

It seems something to do with camera firmware.

@stephendade Is it possible to use libcamera instead of gst for RTP/RTSP etc?

something like: "libcamera-vid -t 0 -n --inline -o - | gst-launch-1.0 fdsrc fd=0 ! h264parse ! rtph264pay ! udpsink host=192.168.235.166 port=5600"

it works fine with OV5647.

Maybe, there is an advanced option for user to select "gstream/libcamera"?

stephendade commented 5 months ago

Thanks for the report. This is a strange one - I've not seen it occur before.

I should have time to investigate next week.

lida2003 commented 5 months ago

Yeah. It's weird. Add some points which you can consider:

  1. IMX219 - Raspberry Pi V2.1 has the same issue on my Raspberry Pi //I have some OV5647 with different EFL/FOV parameters
  2. I have installed bullseys 64bit on Rasbperry Pi3B+
  3. The camera will be recognized occasionally, after I upgraded the system 1 or 2 weeks ago.
lida2003 commented 5 months ago

It might be something to do with gst library, check below links:

And I have sucessfully found the camera with following binary commands:

$ sudo v4l2-ctl --list-devices
unicam (platform:3f801000.csi):
        /dev/video0
        /dev/media2

bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12
        /dev/video18
        /dev/video31
        /dev/media3

bcm2835-isp (platform:bcm2835-isp):
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16
        /dev/video20
        /dev/video21
        /dev/video22
        /dev/video23
        /dev/media0
        /dev/media1

$ libcamera-hello  --list-camera
Available cameras
-----------------
0 : ov5647 [2592x1944] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 [58.92 fps - (16, 0)/2560x1920 crop]
                             1296x972 [43.25 fps - (0, 0)/2592x1944 crop]
                             1920x1080 [30.62 fps - (348, 434)/1928x1080 crop]
                             2592x1944 [15.63 fps - (0, 0)/2592x1944 crop]
stephendade commented 5 months ago

I've had a look at this, running a RPi 3B+, OV5647, Bullseye x64.

I've been unable to replicate your issue - I can consistently detect the camera.

Whay version of Gstreamer and libcamera are you running: gst-launch-1.0 --version I'm running 1.18.4 libcamera-hello --version I'm running v0.0.5+83-bde9b04f pip show picamera2 I'm running 0.3.12 pip show PyGObject I'm running 3.38.0

lida2003 commented 5 months ago

No big difference, just picamera2 0.3.18

daniel@rpanion:~ $ gst-launch-1.0 --version
gst-launch-1.0 version 1.18.4
GStreamer 1.18.4
http://packages.qa.debian.org/gstreamer1.0
daniel@rpanion:~ $ libcamera-hello --version
libcamera-apps build: 7e4d3d71867f 17-07-2023 (07:39:30)
libcamera build: v0.0.5+83-bde9b04f
daniel@rpanion:~ $ pip show picamera2
Name: picamera2
Version: 0.3.18
Summary: The libcamera-based Python interface to Raspberry Pi cameras, based on the original Picamera library
Home-page: https://github.com/RaspberryPi/picamera2
Author: Raspberry Pi & Raspberry Pi Foundation
Author-email: picamera2@raspberrypi.com
License: BSD 2-Clause License
Location: /home/daniel/.local/lib/python3.9/site-packages
Requires: av, numpy, PiDNG, piexif, pillow, python-prctl, simplejpeg, v4l2-python3
Required-by:
daniel@rpanion:~ $ pip show PyGObject
Name: PyGObject
Version: 3.38.0
Summary: Python bindings for GObject Introspection
Home-page: https://pygobject.readthedocs.io
Author: James Henstridge
Author-email: james@daa.com.au
License: GNU LGPL
Location: /usr/lib/python3/dist-packages
Requires: pycairo
Required-by:

--- EDIT I reinstall picamera2 0.3.12, OV5647 comes back :). Will you please install picamera2 0.3.18 and try if it behaves as i did.

stephendade commented 5 months ago

--- EDIT I reinstall picamera2 0.3.12, OV5647 comes back :). Will you please install picamera2 0.3.18 and try if it behaves as i did.

Yep, confirmed. Tried a few other versions. picamera > 0.3.14 has the issue.

EDIT: I suspect it's something to do with libcamera needing to be updated too. Stepping through the ./python/gstcaps.py code shows picamera2 trying to call some non-existant libcamera functions

lida2003 commented 5 months ago

EDIT: I suspect it's something to do with libcamera needing to be updated too. Stepping through the ./python/gstcaps.py code shows picamera2 trying to call some non-existant libcamera functions

Is it possible to trace the code (call stack), maybe we can fire an issue for libcamera or picamera2

--- EDIT: I got gdb segfault, and can't trace the call stack. Not sure how to handle this.

$ sudo apt-get install gdb python3-dbg
$ sudo apt-get install --reinstall python3
$ gdb python3
Segmentation fault
stephendade commented 5 months ago

The way I do is via code fragments. Here's a simple PoC based on ./python/gstcaps.py:

from picamera2 import Picamera2
print(Picamera2.global_camera_info())

Gives the following exception (when using picamera2 > 0.3.14):

File "/home/pi/.local/lib/python3.9/site-packages/picamera2/__init__.py", line 9, in <module>
    from .picamera2 import Picamera2, Preview
  File "/home/pi/.local/lib/python3.9/site-packages/picamera2/picamera2.py", line 24, in <module>
    import picamera2.utils as utils
  File "/home/pi/.local/lib/python3.9/site-packages/picamera2/utils.py", line 1, in <module>
    from libcamera import ColorSpace, Orientation, Rectangle, Size, Transform
ImportError: cannot import name 'Orientation' from 'libcamera' (/usr/lib/python3/dist-packages/libcamera/__init__.py)
lida2003 commented 5 months ago

Maybe we have to face the fact that there are difference between bookworm and bullseye. And write an install script to distinguish the OS version and install proper picamera2 version?

stephendade commented 5 months ago

Using sudo apt install python3-picamera2 python3-libcamera will install the correct version for your OS. That's what I'm usng in ./deploy/RasPi2-3-4-5-deploy.sh

pip install method is not recommended.

lida2003 commented 5 months ago

OK. Maybe I have used pip upgrade before. Thanks for the support.