goodrobots / maverick

UAV Autonomous Systems Management
https://goodrobots.github.io/maverick/
MIT License
171 stars 61 forks source link

webrtc stream fails first start #939

Closed fnoop closed 4 years ago

fnoop commented 4 years ago

See https://github.com/goodrobots/maverick-web/issues/172

fnoop commented 4 years ago

Changed boot order so webrtc starts after visiond in systemd startup, but needs more than that. visiond takes time to startup the video stream and create a client, webrtc is starting before the stream starts, fails and doesn't retry. Restarting webrtc later on correctly connects to the visiond rtsp stream and then works.

Need to somehow delay the webrtc startup until visiond rtsp stream is fully active.

SamuelDudley commented 4 years ago

https://github.com/facebookincubator/pystemd/blob/master/_docs/daemon.md

We can use pystemd to notify systemd when the stream is created. Webrtc will wait until the notification occurs.

fnoop commented 4 years ago

Think this is fixed now

SamuelDudley commented 4 years ago

Interesting, this since your patch I have not been able to get a stream running. I can see that webrtc is trying to start but visiond was totally failing to start. Perhaps it's related to the http get in visond (janus thread) being made with webrtc not yet running? Need to have a closer look.

fnoop commented 4 years ago

What is the visiond log saying?

SamuelDudley commented 4 years ago

Visond is getting a connection refused error on line 86 of janus.py (the get_info request) which causes it to exit and retry. At the same time webrtc is in a loop not able to connect to the websocket handler (as visond has not started).

SamuelDudley commented 4 years ago

Possibly we could delay the get_info request until after the websock server has started?

fnoop commented 4 years ago

Visond is getting a connection refused error on line 86 of janus.py (the get_info request) which causes it to exit and retry. At the same time webrtc is in a loop not able to connect to the websocket handler (as visond has not started).

Ah. Odd I'm not getting that, but that's only there to query janus if it's already up and running, so just wrap it in a try/except. visiond then starts up the websocket which janus connects to and fires events over. The initial query is only for if janus is already running.

SamuelDudley commented 4 years ago

Sure thing. Leave it with me.

fnoop commented 4 years ago

Updated visiond, should work now

SamuelDudley commented 4 years ago

This is odd... I reads like it cant find the config files

---------------------------------------------------
  Starting Meetecho Janus (WebRTC Server) v0.9.2
---------------------------------------------------

Checking command line arguments...
Debug/log level is 3
Debug/log timestamps are disabled
Debug/log colors are enabled
[WARN] Token based authentication disabled
[WARN] Janus is deployed on a private address (192.168.43.216) but you didn't specify any STUN server! Expect trouble if this is supposed to work over the internet and not just in a LAN...
[WARN] The libsrtp installation does not support AES-GCM profiles
[WARN] Data Channels support not compiled
[WARN] Disabling event handlers statistics period, no media statistics will be pushed to event handlers
[WARN] Event handler plugin 'libjanus_sampleevh.so' has been disabled, skipping...
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.recordplay.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.plugin.recordplay), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.recordplay.cfg'... error 2 (No such file or directory)
[FATAL] [plugins/janus_recordplay.c:janus_recordplay_init:723] No recordings path specified, giving up...
[WARN] The 'janus.plugin.recordplay' plugin could not be initialized
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.voicemail.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.plugin.voicemail), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.voicemail.cfg'... error 2 (No such file or directory)
[ERR] [utils.c:janus_mkdir:280] Error creating folder ./html
[ERR] [plugins/janus_voicemail.c:janus_voicemail_init:355] Permission denied[WARN] The 'janus.plugin.voicemail' plugin could not be initialized
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.audiobridge.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.plugin.audiobridge), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.audiobridge.cfg'... error 2 (No such file or directory)
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.videoroom.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.plugin.videoroom), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.videoroom.cfg'... error 2 (No such file or directory)
[WARN] Data channels support not compiled, disabling TextRoom plugin
[WARN] The 'janus.plugin.textroom' plugin could not be initialized
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.videocall.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.plugin.videocall), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.videocall.cfg'... error 2 (No such file or directory)
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.echotest.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.plugin.echotest), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.plugin.echotest.cfg'... error 2 (No such file or directory)
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.transport.pfunix.jcfg'... error 2 (No such file or directory)
[WARN] Couldn't find .jcfg configuration file (janus.transport.pfunix), trying .cfg
[ERR] [config.c:janus_config_parse:191]   -- Error reading configuration file 'janus.transport.pfunix.cfg'... error 2 (No such file or directory)
[WARN] No Unix Sockets server started, giving up...
[WARN] The 'janus.transport.pfunix' plugin could not be initialized
[WARN] HTTP webserver disabled
[WARN] Admin/monitor HTTP webserver disabled
[WARN] Admin/monitor HTTPS webserver disabled
[WARN] libwebsockets has been built without IPv6 support, will bind to IPv4 only
[WARN] WebSockets server disabled
[WARN] Admin WebSockets server disabled
[WARN] Secure Admin WebSockets server disabled
[WARN] libnice version outdated: 0.1.14 installed, at least 0.1.15 recommended
[WARN] [rtsp-stream] 5s passed with no media, trying to reconnect the RTSP stream
[WARN] No c-line or source for RTSP video address, resolving server address...
[WARN]    -- 127.0.0.1
Stopping server, please wait...
Bye!

The service says its trying to read files from the config folder...

● maverick-webrtc.service - WebRTC Janus Gateway
   Loaded: loaded (/etc/systemd/system/maverick-webrtc.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Tue 2020-03-31 23:56:54 BST; 6min ago
  Process: 2483 ExecStart=/srv/maverick/software/janus-gateway/bin/janus --configs-folder=/srv/maverick/config/web/janus (code=exited, status=0/SUCCESS)
 Main PID: 2483 (code=exited, status=0/SUCCESS)

and the files exist at that path

mav@maverick-raspberrylite:~/software/maverick-api$ ls /srv/maverick/config/web/janus
janus.eventhandler.wsevh.jcfg  janus.jcfg  janus.plugin.nosip.jcfg  janus.plugin.sip.jcfg  janus.plugin.streaming.jcfg  janus.transport.http.jcfg  janus.transport.websockets.jcfg
fnoop commented 4 years ago

It's looking for the other sample files that maverick is removing - ignore those warnings.

SamuelDudley commented 4 years ago

okay, turns out it was a mix of things.. input = v4l2 is no longer a valid option in visiond and that was causing a failure. That was only a recent problem. The bigger problem was that the video streaming didnt work on linux + firefox or ios + chrome (ipad) so I was second guessing if webrtc was working as expected. Works fine (after fixing the visiond config file) on linux + chrome.

fnoop commented 4 years ago

input = v4l2 is no longer a valid option in visiond and that was causing a failure

oops, sorry - I normalised the inputs and changed all the configs, but forgot it would affect existing configs files. The default now is to have a completely blank config and let everything autodetect.

SamuelDudley commented 4 years ago

I just wiped the config file and all was well 👍 I think the input = v4l2 option is still floating around in there, just commented out. Are there any other options that if uncommented would cause problems?

fnoop commented 4 years ago

Oh yes, updated thanks! I've been through all the other configs and changed all the defaults in visiond so I think they should be good. v4l2 was 'wrong' to start with which is why I corrected it.