Segmentation fault on arm

[9a4gl]

On intel i7 ubuntu 16 everything fine, but when running on arm (orange pi zero with armbian) I get crash:

Stacktrace:

#0  0xb6b01728 in free () from /lib/arm-linux-gnueabihf/libc.so.6
#1  0x0002c97c in std::default_delete<cricket::VideoFormat>::operator() (this=0xb1b1f128, __ptr=0xd312e00) at /usr/include/c++/5/bits/unique_ptr.h:76
#2  0x0002c35e in std::unique_ptr<cricket::VideoFormat, std::default_delete<cricket::VideoFormat> >::reset (this=0xb1b1f128, __p=0xd312e00) at /usr/include/c++/5/bits/unique_ptr.h:344
#3  0x0002be50 in cricket::VideoCapturer::SetCaptureFormat (this=0xb1b1f0c0, format=0xbec16f88) at /usr/local/src/webrtc/src/webrtc/media/base/videocapturer.h:241
#4  0x0002b7a4 in RTSPVideoCapturer::Start (this=0xb1b1f0c0, format=...) at src/rtspvideocapturer.cpp:144
#5  0x000c7ba6 in cricket::VideoCapturer::StartCapturing(cricket::VideoFormat const&) ()
#6  0x0006c0f0 in rtc::FunctorMessageHandler<bool, rtc::MethodFunctor<cricket::VideoCapturer, bool (cricket::VideoCapturer::*)(cricket::VideoFormat const&), bool, cricket::VideoFormat const&> >::OnMessage(rtc::Message*) ()
#7  0x00059b6e in rtc::Thread::ReceiveSendsFromThread(rtc::Thread const*) ()
#8  0x00056380 in rtc::MessageQueue::Get(rtc::Message*, int, bool) ()
#9  0x00059872 in rtc::Thread::Run() ()
#10 0x000595dc in rtc::Thread::PreRun(void*) ()
#11 0xb6ba05b4 in start_thread () from /lib/arm-linux-gnueabihf/libpthread.so.0
#12 0xb6b41aac in ?? () from /lib/arm-linux-gnueabihf/libc.so.6

Last thing on stdout (running with -vvv):

[032:355] [12486] (webrtcsession.cc:1156): ProcessIceMessage: Not ready to use candidate.
answer:1
[032:355] [12486] (webrtcsessiondescriptionfactory.cc:492): Setting new certificate.

*** 1490226120.096057524      1723814 2920281152 process_new_connection:14132: handle_request done
*** 1490226120.096158981       101457 2920281152 worker_thread_run:14442: Done processing connection from 10.130.133.12 (0.000000 sec)
*** 1490226120.096417395       258414 2920281152 worker_thread_run:14446: Connection closed
*** 1490226120.096498644        81249 2920281152 consume_socket:14248: going idle
[032:363] [12486] (PeerConnectionManager.h:39): virtual void PeerConnectionManager::SetSessionDescriptionObserver::OnSuccess() Remote SDP:v=0
o=- 1705043383676279282 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:54023 IN IP4 44.170.1.2
a=candidate:842163049 2 udp 1677729534 44.170.1.2 54023 typ srflx raddr 10.130.133.12 rport 54023 generation 0 network-cost 50
a=ice-ufrag:fJ8C
a=ice-pwd:+LbXikJkqtiGTv/m2ELk1JgH
a=fingerprint:sha-256 7D:79:96:3A:66:29:30:9C:81:CC:B5:95:F0:45:AD:6F:B5:F7:52:89:5B:F0:8F:0C:F2:09:A3:B0:E9:CA:7F:0D
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 isac/16000
a=rtpmap:104 isac/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 cn/32000
a=rtpmap:105 cn/16000
a=rtpmap:13 cn/8000
a=rtpmap:126 telephone-event/8000
m=video 49412 UDP/TLS/RTP/SAVPF 100 101 107 116 117 96 97 99 98
c=IN IP4 44.170.1.2
a=rtcp:60602 IN IP4 44.170.1.2
a=candidate:842163049 1 udp 1677729535 44.170.1.2 49412 typ srflx raddr 10.130.133.12 rport 49412 generation 0 network-cost 50
a=candidate:842163049 2 udp 1677729534 44.170.1.2 60602 typ srflx raddr 10.130.133.12 rport 60602 generation 0 network-cost 50
a=ice-ufrag:fJ8C
a=ice-pwd:+LbXikJkqtiGTv/m2ELk1JgH
a=fingerprint:sha-256 7D:79:96:3A:66:29:30:9C:81:CC:B5:95:F0:45:AD:6F:B5:F7:52:89:5B:F0:8F:0C:F2:09:A3:B0:E9:CA:7F:0D
a=setup:actpass
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtpmap:101 VP9/90000
a=rtcp-fb:101 ccm fir
a=rtcp-fb:101 nack
a=rtcp-fb:101 nack pli
a=rtcp-fb:101 goog-remb
a=rtcp-fb:101 transport-cc
a=rtpmap:107 H264/90000
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=rtcp-fb:107 goog-remb
a=rtcp-fb:107 transport-cc
a=fmtp:107 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
a=rtpmap:96 rtx/90000
a=fmtp:96 apt=100
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=101
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=107
a=rtpmap:98 rtx/90000
a=fmtp:98 apt=116

I didn't use raspberry pi crosscompile instructions on wiki, but I crosscompiled with: gn gen out/Release --args='target_cpu="arm" is_debug=false rtc_use_h264=true ffmpeg_branding="Chrome"'

and then webrtc-streamer was build on arm cpu with crosscompiled webrtc used from cifs share.

any idea ?

BR, Tihomir 9a4gl

[mpromonet]

Hi Tihomir, This is probably because std used compiling webrtc is different from the one used to compile webrtc-streamer. It could be also some compilation options. I put an awful option in order to link between clang & g++, maybe you should try to remove it ? Regards, Michel.

[9a4gl]

Yup, after writing an issue it came to my mind that stdlib might have different implementation. Unfortunatelly webrtc cannot be built on arm, it complains that it is not supported and apparently chromium guys are guilty for that. So I have to crosscompile.

To solve this, I would need also crosscompile webrtc-stream using same sysroot as for webrtc, and one that comes with webrtc is missing some libraries you use. Ah, either I will have to install that stuff in sysroot or I will just compile and then link on arm, don't know which way is bigger pain.

Anyone running webrtc-streamer on arm, how it performs ?

BR, Tihomir

[9a4gl]

Finally got it working. I added required libs to sysroot and created ugly Makefile.arm.txt that use clang to compile using extended sysroot. Now it works on my orange pi zero. Running 1920x1080 I got cca 250% cpu usage (have 4 cores), so it got heavy loaded.

I noticed on one stream (640x352) I got crash:

[023:572] [15215] (h264_decoder_impl.cc:330): avcodec_decode_video2 error: -1094995529
*** Error in `./webrtc_streamer': free(): invalid next size (fast): 0xb0a9f418 ***

This does not happen on amd64. The first error message is probably something wrong in rtsp stream, but second one indicate memory corruption, would be good to check that with valgrind.

[9a4gl]

giving up arm, we can reopen this and continue once we found someone made a stable arm build