search

meetecho / janus-gateway

Janus WebRTC Server
https://janus.conf.meetecho.com
GNU General Public License v3.0
8.11k stars 2.46k forks source link

h264 streaming #97

Closed gregd72002 closed 9 years ago

gregd72002 commented 9 years ago

Hi,

I've been trying to get h264 stream working as described in here: https://groups.google.com/forum/#!topic/meetecho-janus/XjjboCTY0Xc

using gstreamer1 and nightly firefox build

This fails with firefox showing the following error:

WebRTC error... {"name":"INVALID_SESSION_DESCRIPTION","message":"Could not negotiate media lines; cause = PAYLOAD_MISMATCH | SDP Parsing Error: Warning: Group attribute type unsupported (BUNDLE). | SDP Parsing Error: Warning: Unrecognized attribute (msid-semantic) | SDP Parsing Error: Warning: Numeric token for mid attribute not found | SDP Parsing Error: Warning: Invalid token ordering detected, token c= found after token a= | SDP Parsing Error: Warning: Unrecognized attribute (ice-options) | SDP Parsing Error: Warning: Unrecognized attribute (ssrc) | SDP Parsing Error: Warning: Unrecognized attribute (ssrc) | SDP Parsing Error: Warning: Unrecognized attribute (ssrc) | SDP Parsing Error: Warning: Unrecognized attribute (ssrc) "}
---------------------------------------------------
  Starting Meetecho Janus (WebRTC Gateway) v0.0.6
---------------------------------------------------

Reading configuration from /etc/janus/janus.cfg
Checking command line arguments...
Debug/log level is 4
Adding 'vmnet' to the ICE ignore list...
Using 192.168.1.74 as local IP...
Initializing ICE stuff (IPv6 candidates disabled)
Fingerprint of our certificate: C5:5F:DA:7D:84:47:B1:BF:6B:55:16:62:48:31:3E:D3:F1:7B:25:89:92:4A:4B:4D:4D:D9:D5:AF:EA:D8:15:44
WARNING: no real random source present!
[WARN] Data Channels support not compiled
Plugins folder: /usr/lib/janus/plugins
Loading plugin 'libjanus_echotest.so'...
[ERR] [config.c:janus_config_parse:72:]   -- Error reading configuration file... error 2 (No such file or directory)
JANUS EchoTest plugin initialized!
Loading plugin 'libjanus_recordplay.so'...
Echotest watchdog started
[ERR] [config.c:janus_config_parse:72:]   -- Error reading configuration file... error 2 (No such file or directory)
[WARN] No recordings path specified, using /tmp...
JANUS Record&Play plugin initialized!
Loading plugin 'libjanus_videoroom.so'...
Record&Play watchdog started
[ERR] [config.c:janus_config_parse:72:]   -- Error reading configuration file... error 2 (No such file or directory)
JANUS VideoRoom plugin initialized!
Loading plugin 'libjanus_streaming.so'...
VideoRoom watchdog started
JANUS Streaming plugin initialized!
Loading plugin 'libjanus_sip.so'...
Streaming watchdog started
[h264-sample] New video stream! (ssrc=2269432660)
[ERR] [config.c:janus_config_parse:72:]   -- Error reading configuration file... error 2 (No such file or directory)
JANUS SIP plugin initialized!
Loading plugin 'libjanus_videocall.so'...
SIP watchdog started
[ERR] [config.c:janus_config_parse:72:]   -- Error reading configuration file... error 2 (No such file or directory)
JANUS VideoCall plugin initialized!
VideoCall watchdog started
HTTP webserver started (port 8088, /janus path listener)...
[WARN] HTTPS webserver disabled
WebSockets server started (port 8188)...
[WARN] Secure WebSockets server disabled
[WARN] RabbitMQ support not compiled
[WARN] Admin/monitor HTTP webserver disabled
[WARN] Admin/monitor HTTPS webserver disabled
Sessions watchdog started
Creating new session: 1451593739
Creating new handle in session 1451593739: 870209035
[870209035] There's a message for JANUS Streaming plugin
[870209035] There's a message for JANUS Streaming plugin
[870209035] ICE send thread started...
[870209035] Creating ICE agent (controlling mode)
lminiero commented 9 years ago

Can you paste the SDP that is getting to Firefox, please?

gregd72002 commented 9 years ago

I will need to get some additional tools for debugging this in Firefox. Will do it late on today.

Here is the SDP that the gateway sends:

We have a message to serve...
    {
   "janus": "event",
   "session_id": 329885277,
   "sender": 3528371946,
   "transaction": "vwJUKD5BrnVI",
   "plugindata": {
      "plugin": "janus.plugin.streaming",
      "data": {
         "streaming": "event",
         "result": {
            "status": "preparing"
         }
      }
   },
   "jsep": {
      "type": "offer",
      "sdp": "v=0\r\no=- 327542603 327542603 IN IP4 127.0.0.1\r\ns=Streaming Test\r\nt=0 0\r\na=group:BUNDLE video\r\na=msid-semantic: WMS janus\r\nm=video 1 RTP/SAVPF 126\r\na=mid:video\r\nc=IN IP4 192.168.1.167\r\na=sendonly\r\na=rtcp-mux\na=ice-ufrag:BriN\r\na=ice-pwd:pqAh71lk5laJ9VXPxk7SST\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C5:5F:DA:7D:84:47:B1:BF:6B:55:16:62:48:31:3E:D3:F1:7B:25:89:92:4A:4B:4D:4D:D9:D5:AF:EA:D8:15:44\r\na=setup:actpass\r\na=connection:new\r\na=rtpmap:126 H264/90000\r\na=fmtp:126 profile-level-id=42e01f;packetization-mode=1\r\na=ssrc:1960860913 cname:janusvideo\r\na=ssrc:1960860913 msid:janus janusv0\r\na=ssrc:1960860913 mslabel:janus\r\na=ssrc:1960860913 label:janusv0\r\na=candidate:1 1 udp 2013266431 192.168.1.167 34083 typ host\r\na=candidate:2 1 udp 1019216383 192.168.1.167 0 typ host\r\na=candidate:3 1 udp 1015022079 192.168.1.167 39373 typ host\r\na=candidate:1 2 udp 2013266430 192.168.1.167 51913 typ host\r\na=candidate:2 2 udp 1019216382 192.168.1.167 0 typ host\r\na=candidate:3 2 udp 1015022078 192.168.1.167 40825 typ host\r\n"
   }
}

And here the full debug:

Session: 329885277
Processing POST data (application/json; charset=UTF-8)...
Done getting payload, we can answer
{"janus":"attach","plugin":"janus.plugin.streaming","transaction":"6n2aRFdh7bQq"}
Creating new handle in session 329885277: 3528371946
Request completed, freeing data
Got a HTTP OPTIONS request on /janus/329885277/3528371946...
 ... Just parsing headers for now...
Host: 192.168.1.167:8088
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Origin: http://192.168.1.167
Access-Control-Request-Method: POST
Access-Control-Request-Headers: content-type
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Request completed, freeing data
Got a HTTP POST request on /janus/329885277/3528371946...
 ... Just parsing headers for now...
Host: 192.168.1.167:8088
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://192.168.1.167/demos/streamingtest.html
Content-Type: application/json; charset=UTF-8
Content-Length: 74
Origin: http://192.168.1.167
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Got a HTTP POST request on /janus/329885277/3528371946...
 ... parsing request...
Session: 329885277
Handle: 3528371946
Processing POST data (application/json; charset=UTF-8)...
  -- Uploaded data (74 bytes)
  -- Data we have now (74 bytes)
Got a HTTP POST request on /janus/329885277/3528371946...
 ... parsing request...
Session: 329885277
Handle: 3528371946
Processing POST data (application/json; charset=UTF-8)...
Done getting payload, we can answer
{"janus":"message","body":{"request":"list"},"transaction":"LbMP49xmgsLJ"}
[3528371946] There's a message for JANUS Streaming plugin
{
   "request": "list"
}
Handling message: {
   "request": "list"
}
Request for the list of mountpoints
Creating plugin result...
Destroying plugin result...
Request completed, freeing data
Got a HTTP OPTIONS request on /janus/329885277/3528371946...
 ... Just parsing headers for now...
Host: 192.168.1.167:8088
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Origin: http://192.168.1.167
Access-Control-Request-Method: POST
Access-Control-Request-Headers: content-type
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Request completed, freeing data
Got a HTTP POST request on /janus/329885277/3528371946...
 ... Just parsing headers for now...
Host: 192.168.1.167:8088
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://192.168.1.167/demos/streamingtest.html
Content-Type: application/json; charset=UTF-8
Content-Length: 83
Origin: http://192.168.1.167
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Got a HTTP POST request on /janus/329885277/3528371946...
 ... parsing request...
Session: 329885277
Handle: 3528371946
Processing POST data (application/json; charset=UTF-8)...
  -- Uploaded data (83 bytes)
  -- Data we have now (83 bytes)
Got a HTTP POST request on /janus/329885277/3528371946...
 ... parsing request...
Session: 329885277
Handle: 3528371946
Processing POST data (application/json; charset=UTF-8)...
Done getting payload, we can answer
{"janus":"message","body":{"request":"watch","id":10},"transaction":"vwJUKD5BrnVI"}
[3528371946] There's a message for JANUS Streaming plugin
{
   "request": "watch",
   "id": 10
}
Handling message: {
   "request": "watch",
   "id": 10
}
Creating plugin result...
Destroying plugin result...
Request to watch mountpoint/stream 10
Going to offer this SDP:
v=0
o=- 327542603 327542603 IN IP4 127.0.0.1
s=Streaming Test
t=0 0
m=video 1 RTP/SAVPF 126
c=IN IP4 1.1.1.1
a=rtpmap:126 H264/90000
a=fmtp:126 profile-level-id=42e01f;packetization-mode=1
a=sendonly

Pushing event: {
   "streaming": "event",
   "result": {
      "status": "preparing"
   }
}
[3528371946] Audio has NOT been negotiated
[3528371946] Video has been negotiated
[3528371946] SCTP/DataChannels have NOT been negotiated
[3528371946] Setting ICE locally: got ANSWER (0 audios, 1 videos)
Request completed, freeing data
[3528371946] ICE thread started, looping...
[3528371946] ICE send thread started...
[3528371946] Creating ICE agent (controlling mode)
Adding 192.168.1.167 to the addresses to gather candidates for
[3528371946] Gathering done for stream 1
 -------------------------------------------
  >> Anonymized (209 --> 209 bytes)
 -------------------------------------------
v=0
o=- 327542603 327542603 IN IP4 127.0.0.1
s=Streaming Test
t=0 0
m=video 1 RTP/SAVPF 126
c=IN IP4 1.1.1.1
a=sendonly
a=rtpmap:126 H264/90000
a=fmtp:126 profile-level-id=42e01f;packetization-mode=1

[3528371946] No RTP maps?? trying formats...
[3528371946] We have 3 candidates for Stream #1, Component #1
[3528371946] Stream #1, Component #1
[3528371946]   Address:    192.168.1.167:34083
[3528371946]   Priority:   2013266431
[3528371946]   Foundation: 1
[3528371946]     a=candidate:1 1 udp 2013266431 192.168.1.167 34083 typ host

[3528371946] Stream #1, Component #1
[3528371946]   Address:    192.168.1.167:0
[3528371946]   Priority:   1019216383
[3528371946]   Foundation: 2
[3528371946]     a=candidate:2 1 udp 1019216383 192.168.1.167 0 typ host

[3528371946] Stream #1, Component #1
[3528371946]   Address:    192.168.1.167:39373
[3528371946]   Priority:   1015022079
[3528371946]   Foundation: 3
[3528371946]     a=candidate:3 1 udp 1015022079 192.168.1.167 39373 typ host

[3528371946] We have 3 candidates for Stream #1, Component #2
[3528371946] Stream #1, Component #2
[3528371946]   Address:    192.168.1.167:51913
[3528371946]   Priority:   2013266430
[3528371946]   Foundation: 1
[3528371946]     a=candidate:1 2 udp 2013266430 192.168.1.167 51913 typ host

[3528371946] Stream #1, Component #2
[3528371946]   Address:    192.168.1.167:0
[3528371946]   Priority:   1019216382
[3528371946]   Foundation: 2
[3528371946]     a=candidate:2 2 udp 1019216382 192.168.1.167 0 typ host

[3528371946] Stream #1, Component #2
[3528371946]   Address:    192.168.1.167:40825
[3528371946]   Priority:   1015022078
[3528371946]   Foundation: 3
[3528371946]     a=candidate:3 2 udp 1015022078 192.168.1.167 40825 typ host

 -------------------------------------------
  >> Merged (209 --> 1016 bytes)
 -------------------------------------------
v=0
o=- 327542603 327542603 IN IP4 127.0.0.1
s=Streaming Test
t=0 0
a=group:BUNDLE video
a=msid-semantic: WMS janus
m=video 1 RTP/SAVPF 126
a=mid:video
c=IN IP4 192.168.1.167
a=sendonly
a=rtcp-mux
a=ice-ufrag:BriN
a=ice-pwd:pqAh71lk5laJ9VXPxk7SST
a=ice-options:trickle
a=fingerprint:sha-256 C5:5F:DA:7D:84:47:B1:BF:6B:55:16:62:48:31:3E:D3:F1:7B:25:89:92:4A:4B:4D:4D:D9:D5:AF:EA:D8:15:44
a=setup:actpass
a=connection:new
a=rtpmap:126 H264/90000
a=fmtp:126 profile-level-id=42e01f;packetization-mode=1
a=ssrc:1960860913 cname:janusvideo
a=ssrc:1960860913 msid:janus janusv0
a=ssrc:1960860913 mslabel:janus
a=ssrc:1960860913 label:janusv0
a=candidate:1 1 udp 2013266431 192.168.1.167 34083 typ host
a=candidate:2 1 udp 1019216383 192.168.1.167 0 typ host
a=candidate:3 1 udp 1015022079 192.168.1.167 39373 typ host
a=candidate:1 2 udp 2013266430 192.168.1.167 51913 typ host
a=candidate:2 2 udp 1019216382 192.168.1.167 0 typ host
a=candidate:3 2 udp 1015022078 192.168.1.167 40825 typ host

[3528371946] Adding event to queue of messages...
  >> 0 (Success)
We have a message to serve...
    {
   "janus": "event",
   "session_id": 329885277,
   "sender": 3528371946,
   "transaction": "vwJUKD5BrnVI",
   "plugindata": {
      "plugin": "janus.plugin.streaming",
      "data": {
         "streaming": "event",
         "result": {
            "status": "preparing"
         }
      }
   },
   "jsep": {
      "type": "offer",
      "sdp": "v=0\r\no=- 327542603 327542603 IN IP4 127.0.0.1\r\ns=Streaming Test\r\nt=0 0\r\na=group:BUNDLE video\r\na=msid-semantic: WMS janus\r\nm=video 1 RTP/SAVPF 126\r\na=mid:video\r\nc=IN IP4 192.168.1.167\r\na=sendonly\r\na=rtcp-mux\na=ice-ufrag:BriN\r\na=ice-pwd:pqAh71lk5laJ9VXPxk7SST\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C5:5F:DA:7D:84:47:B1:BF:6B:55:16:62:48:31:3E:D3:F1:7B:25:89:92:4A:4B:4D:4D:D9:D5:AF:EA:D8:15:44\r\na=setup:actpass\r\na=connection:new\r\na=rtpmap:126 H264/90000\r\na=fmtp:126 profile-level-id=42e01f;packetization-mode=1\r\na=ssrc:1960860913 cname:janusvideo\r\na=ssrc:1960860913 msid:janus janusv0\r\na=ssrc:1960860913 mslabel:janus\r\na=ssrc:1960860913 label:janusv0\r\na=candidate:1 1 udp 2013266431 192.168.1.167 34083 typ host\r\na=candidate:2 1 udp 1019216383 192.168.1.167 0 typ host\r\na=candidate:3 1 udp 1015022079 192.168.1.167 39373 typ host\r\na=candidate:1 2 udp 2013266430 192.168.1.167 51913 typ host\r\na=candidate:2 2 udp 1019216382 192.168.1.167 0 typ host\r\na=candidate:3 2 udp 1015022078 192.168.1.167 40825 typ host\r\n"
   }
}
Request completed, freeing data
Got a HTTP GET request on /janus/329885277...
 ... Just parsing headers for now...
Host: 192.168.1.167:8088
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://192.168.1.167/demos/streamingtest.html
Origin: http://192.168.1.167
Connection: keep-alive
Got a HTTP GET request on /janus/329885277...
 ... parsing request...
Session: 329885277
Session 329885277 found... returning up to 1 messages
... handling long poll...
Long poll time out for session 329885277...
We have a message to serve...
    {"janus" : "keepalive"}
Request completed, freeing data
Got a HTTP GET request on /janus/329885277...
 ... Just parsing headers for now...
Host: 192.168.1.167:8088
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://192.168.1.167/demos/streamingtest.html
Origin: http://192.168.1.167
Connection: keep-alive
Got a HTTP GET request on /janus/329885277...
 ... parsing request...
Session: 329885277
Session 329885277 found... returning up to 1 messages
... handling long poll...
Long poll time out for session 329885277...
We have a message to serve...
    {"janus" : "keepalive"}
lminiero commented 9 years ago

Please don't paste huge blocks of debug/log on threads, they make it very unreadable for me... use a service like pastebin or similar.

lminiero commented 9 years ago

I'm not sure where the PAYLOAD_MISMATCH error comes from, as I can see from the SDP that 126 is being used everywhere: in the m-line, in the rtpmap and the fmtp attributes. Which version of Firefox is this? Is H.264 support enabled in about:config? IIRC by default it isn't.

lminiero commented 9 years ago

I've just tried on my Nightly and indeed I'm getting the same error, but it seems to be caused by the fact that H.264 is not being negotiated/supported, although it's enabled in the settings. I verified this by checking with the Echo Test, from which I could see only VP8 negotiated in the SDP. This might explain why the payload is being rejected, as H.264 is not supported. As such, I guess it's an issue with Firefox.

gregd72002 commented 9 years ago

Is there a version of Firefox that is known to be working with H.264?

There are reports of having this working in firefox so would be nice to know which version was it in order have a reference point.

lminiero commented 9 years ago

Not sure as I never cared much for H.264. You may want to ask the user who opened the post on the Google group to see if he kept track of that. I think it was a version subsequent to FF34, not sure which one. Another idea might be asking on the mozilla.dev.media Google group why H.264 is not being negotiated anymore, as there may be some additional about:config setting that needs to be enabled.

lminiero commented 9 years ago

Just tried with Firefox 34 and it works for me.

gregd72002 commented 9 years ago

Right, I move d to FF34 and do not have the problem above but neither I can see video. The website says Stream started but nothing appears.

janus reports 

[h264-sample] New video stream! (ssrc=2446637362)
Creating new session: 3228518075
Creating new handle in session 3228518075: 1396910359
[1396910359] There's a message for JANUS Streaming plugin
[1396910359] There's a message for JANUS Streaming plugin
[1396910359] ICE send thread started...
[1396910359] Creating ICE agent (controlling mode)
[1396910359] There's a message for JANUS Streaming plugin
[1396910359]   -- ICE Trickling is supported by the browser, waiting for remote candidates...
ICE started and trickling, sending connectivity checks for candidates retrieved so far...
[ERR] [ice.c:janus_ice_cb_nice_recv:688:] Still waiting for the DTLS stack for component 1 in stream 1...
No more remote candidates for handle 1396910359!
[1396910359] The DTLS handshake has been completed
WebRTC media is now available
lminiero commented 9 years ago

What are you using to generate the stream?

gregd72002 commented 9 years ago

using my camera with gstreamer1:

raspivid -n -w 640 -h 480 -b 2000000 -fps 30 -vf -hf -t 0 -g 10 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=10 pt=126 ! udpsink host=127.0.0.1 port=8004
lminiero commented 9 years ago

I think it only works with baseline profile video. Not sure what your raspberry generates. I tried with a baseline video generated by x264enc in gstreamer and it worked for me.

gregd72002 commented 9 years ago

looks like this is one of the problems... I changed the h264 level to 4.0 (this is the only one supported by my camera) and now it 'occasionally' produces a 'video'. It is more like the first frame rather then video.

the profile-level-id I used is '42e028' (28 standing for 4.0 in hex)

Had success with one of these two commands:

raspivid -n -w 1280  -h 720 -b 2000000 -fps 30 -vf -hf -t 0 -pf baseline -o - | gst-launch-1.0 -v fdsrc fd=0 ! h264parse !  rtph264pay config-interval=2 pt=126 ! udpsink host=127.0.0.1 port=8004

raspivid -n -w 640 -h 480 -b 2000000 -fps 30 -vf -hf -t 0 -pf baseline -o - | gst-launch-1.0 -v fdsrc fd=0 ! h264parse !  rtph264pay config-interval=2 pt=126 ! udpsink host=127.0.0.1 port=8004

However very often the gateway produces entire stream of errors:

[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 1410)
[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 1410)
[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 1410)
[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 1410)
[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 833)
[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 1410)
[ERR] [ice.c:janus_ice_send_thread:1511:] [3658661208] ... only sent -1 bytes? (was 1410)
lminiero commented 9 years ago

It may be an MTU issue. Try having rtph264pay split packets in smaller frames.

gregd72002 commented 9 years ago

I tried it again today and it seems to be working much better using FF 34.0.5. Though, now I got a lot of artifacts and something is not quite right with color scheme. But I guess this is something unrelated to janus-gateway.

Thanks a lot!

lminiero commented 9 years ago

Good to know, thanks for the update! Closing the issue then.