search

cyoung / stratux

Aviation weather and traffic receiver based on RTL-SDR.
BSD 3-Clause "New" or "Revised" License
1.06k stars 362 forks source link

RTL error messages when no UAT traffic ? - "failed with -4": also crash scenario #174

Closed bradanlane closed 8 years ago

bradanlane commented 8 years ago

Are the following messages "normal" with the new power savings mode (when not receiving UAT) ?

The timestamped lines make sense but all of the "failed with -4" are worrisome.

2015/12/31 15:58:07  - GPS vertical velocity: 0.00 ft/sec; GPS vertical accuracy: 0 m
2015/12/31 15:58:07 Pausing UAT listening for 50 seconds - none received.
2015/12/31 15:58:07 Entered UAT shutdown() ...
2015/12/31 15:58:07 UAT shutdown(): calling uat_wg.Wait() ...
2015/12/31 15:58:07 UAT read(): shutdown msg received...
2015/12/31 15:58:07 UAT shutdown(): uat_wg.Wait() returned...
2015/12/31 15:58:07 UAT shutdown(): closing device ...
rtlsdr_demod_write_reg failed with -4
rtlsdr_demod_read_reg failed with -4
r82xx_write: i2c wr failed=-4 reg=06 len=1
rtlsdr_demod_write_reg failed with -4
rtlsdr_demod_read_reg failed with -4
rtlsdr_write_reg failed with -4
cyoung commented 8 years ago

Not normal, no. So .Close() on the device is throwing these errors, previous to 15:58:07 were any of these errors appearing?

bradanlane commented 8 years ago

I just fetched code from github (probable two weeks worth of changes); build; install; restart Stratux. That is when I saw the errors.

cyoung commented 8 years ago

I mean that do the errors show up on opening/initializing the SDR or just after "closing device ..."?

bradanlane commented 8 years ago

(sorry) - startup all looks good. Then, I get that block of messages each time it cycles the SDR.

cyoung commented 8 years ago

Hmm, do you have another dongle that you can use for UAT to see if it does this uniformly with different SDRs in your case?

bradanlane commented 8 years ago

I don't know what is going on with that code but its hammering the CPU. I see a 10C temp spike.

jpoirier commented 8 years ago

I just noticed the uat duty cycle code changes. Any idea if the problem existed before those changes were made?

On Thu, Dec 31, 2015 at 10:34 AM, bradanlane notifications@github.com wrote:

I don't know what is going on with that code but its hammering the CPU. I see a 10C temp spike.

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168217320.

bradanlane commented 8 years ago

@jpoirier - I cant say for certain if the problem occurred "just prior" to the duty cylce change. I can say I was fine as of about two weeks ago (the last time I had time to work on the Stratux UI).

Something from then to now chanced. It could be code or it could be hardware. I'm going to start with a clean image and work forward from there.

bradanlane commented 8 years ago

I'm still seeing the "failed with -4" errors.

The starting and stopping of the SDR is CPU intensive. I have an enclosure on my Stratux which is not well vented. I see the CPU temp just 10C when the SDR is restarted. This was enough to cause a number of problems and ultimately throw a fault code. I added a heatsink which lowered the initial temperature so the spike was within limits.

My only guess is that with the old code, the spike happened when the system was relatively cold and thus the 10C peak was tolerated.

I'll comment out the SDR power-down and see where my system temp stabilizes.

bradanlane commented 8 years ago

If it helps, the "failed with -4" errors do not appear in the logs. I only see them when I run "gen_gdl90" directly on the command prompt.

bradanlane commented 8 years ago

it would appear I have trashed one SDR. I swapped in an SDR from another setup and I no longer get the error.

cyoung commented 8 years ago

Interested to know if the newest code caused the error to surface somehow? Would you be able to test with v0.4r4? git checkoout tags/v0.4r4

bradanlane commented 8 years ago

I'll have to setup my system to pull the old code. While the "failed with -4" errors seem to have disappeared with the replacement SDR, I'm still crashing out of gen_gdl90 after 15-30 minutes.

cyoung commented 8 years ago

Do you debug output for the crash?

bradanlane commented 8 years ago

I have the crash trace dump. I first verified my build. Next I verified the hardware. When I have time I will look at 0.4r4 and I will look at the dump. Unfortunately, at the moment, I have limited time each day to poke the bear.

bradanlane commented 8 years ago

I am running V0.4r4 and get the same issue as with the current code (albeit the timing of the crash is different which may be related to cycling the tuner). I was able to get Stratux V0.4r4 to start up if I unplugged the SDR, started Stratux and then plugged the SDR in.

Here is one dump of the crash - in case any of this makes sense to someone:

root@stratuxpi:~# /usr/bin/gen_gdl90
1970/01/01 00:02:54 Stratux v0.4r4 (d0d466896ee0b0efbfb66868e695327dd3021553) starting.
1970/01/01 00:02:54 can't read settings /etc/stratux.conf: open /etc/stratux.conf: no such file or directory
1970/01/01 00:02:55 ===== UAT Device name: Generic RTL2832U OEM =====
Found Rafael Micro R820T tuner
1970/01/01 00:02:56     GetUsbStrings - Realtek RTL2838UHIDIR 00000001
1970/01/01 00:02:56     GetTunerType: RTLSDR_TUNER_R820T
1970/01/01 00:02:56     SetTunerGainMode Successful
1970/01/01 00:02:56     SetTunerGain Successful
Exact sample rate is: 2083334.141630 Hz
[R82XX] PLL not locked!
1970/01/01 00:02:56     SetSampleRate - rate: 2083334
1970/01/01 00:02:56     GetSampleRate: 2083334
1970/01/01 00:02:56     GetXtalFreq - Rtl: 28800000, Tuner: 28800000
1970/01/01 00:02:56     SetXtalFreq - Center freq: 28800000, Tuner freq: 28800000
1970/01/01 00:02:56     SetCenterFreq 978MHz Successful
1970/01/01 00:02:56     GetCenterFreq: 978000000
1970/01/01 00:02:56     Setting Bandwidth: 1000000
1970/01/01 00:02:57     SetTunerBw 1000000 Successful
1970/01/01 00:02:57     ResetBuffer Successful
1970/01/01 00:02:57     GetFreqCorrection: 0
1970/01/01 00:02:57     SetFreqCorrection 0 Failed, error: invalid parameter(s)
1970/01/01 00:02:57     ReadSync Failed - error: overflow
SIGILL: illegal instruction
PC=0x6e386 m=5
signal arrived during cgo execution

goroutine 36 [syscall, locked to thread]:
runtime.cgocall(0x13338, 0x10800ce4, 0x0)
    /root/go/src/runtime/cgocall.go:120 +0x11c fp=0x10800cc8 sp=0x10800cb0
github.com/jpoirier/gortlsdr._Cfunc_rtlsdr_close(0xb3000468, 0x0)
    github.com/jpoirier/gortlsdr/_obj/_cgo_gotypes.go:128 +0x34 fp=0x10800ce0 sp=0x10800cc8
github.com/jpoirier/gortlsdr.(*Context).Close(0x1076a2b8, 0x0, 0x0)
    /root/go_path/src/github.com/jpoirier/gortlsdr/rtlsdr.go:228 +0x30 fp=0x10800d1c sp=0x10800ce0
main.sdrReader()
    /root/stratux/main/sdr.go:127 +0xeb4 fp=0x10800fd4 sp=0x10800d1c
runtime.goexit()
    /root/go/src/runtime/asm_arm.s:1036 +0x4 fp=0x10800fd4 sp=0x10800fd4
created by main.sdrWatcher
    /root/stratux/main/sdr.go:154 +0xd8

goroutine 1 [syscall]:
syscall.Syscall(0x3, 0x0, 0x107e8000, 0x1000, 0x0, 0x0, 0x0)
    /root/go/src/syscall/asm_linux_arm.s:17 +0x8
syscall.read(0x0, 0x107e8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/go/src/syscall/zsyscall_linux_arm.go:783 +0x78
syscall.Read(0x0, 0x107e8000, 0x1000, 0x1000, 0xc81a0, 0x0, 0x0)
    /root/go/src/syscall/syscall_unix.go:160 +0x4c
os.(*File).read(0x1076a0e8, 0x107e8000, 0x1000, 0x1000, 0xc5dd8, 0x0, 0x0)
    /root/go/src/os/file_unix.go:211 +0x54
os.(*File).Read(0x1076a0e8, 0x107e8000, 0x1000, 0x1000, 0xc7284, 0x0, 0x0)
    /root/go/src/os/file.go:95 +0x7c
bufio.(*Reader).fill(0x10745f44)
    /root/go/src/bufio/bufio.go:97 +0x1c4
bufio.(*Reader).ReadSlice(0x10745f44, 0x3b00a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/bufio/bufio.go:328 +0x264
bufio.(*Reader).ReadBytes(0x10745f44, 0x2f310a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/bufio/bufio.go:406 +0x7c
bufio.(*Reader).ReadString(0x10745f44, 0x10745f0a, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/bufio/bufio.go:446 +0x44
main.main()
    /root/stratux/main/gen_gdl90.go:906 +0x5e8

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /root/go/src/runtime/asm_arm.s:1036 +0x4

goroutine 20 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x51b590)
    /root/go_path/src/github.com/golang/glog/glog.go:882 +0x60
created by github.com/golang/glog.init.1
    /root/go_path/src/github.com/golang/glog/glog.go:410 +0x2cc

goroutine 3 [chan receive]:
main.sdrWatcher()
    /root/stratux/main/sdr.go:144 +0x4c
created by main.sdrInit
    /root/stratux/main/sdr.go:168 +0x3c

goroutine 4 [chan receive]:
main.uatReader()
    /root/stratux/main/sdr.go:132 +0x3c
created by main.sdrInit
    /root/stratux/main/sdr.go:169 +0x50

goroutine 5 [chan receive]:
_/root/stratux/godump978.ProcessDataFromChannel()
    /root/stratux/godump978/godump978.go:59 +0x40
created by main.sdrInit
    /root/stratux/main/sdr.go:171 +0x68

goroutine 22 [sleep]:
time.Sleep(0x3b9aca00, 0x0)
    /root/go/src/runtime/time.go:59 +0x104
main.esListen()
    /root/stratux/main/traffic.go:380 +0x48
created by main.initTraffic
    /root/stratux/main/traffic.go:539 +0xe8

goroutine 23 [chan receive]:
main.pollRY835AI()
    /root/stratux/main/ry835ai.go:607 +0x58
created by main.initRY835AI
    /root/stratux/main/ry835ai.go:630 +0x8c

goroutine 24 [select]:
main.heartBeatSender()
    /root/stratux/main/gen_gdl90.go:356 +0x170
created by main.main
    /root/stratux/main/gen_gdl90.go:890 +0x4d8

goroutine 25 [IO wait]:
net.runtime_pollWait(0xb4188040, 0x72, 0x10778000)
    /root/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0x107bc138, 0x72, 0x0, 0x0)
    /root/go/src/net/fd_poll_runtime.go:73 +0x34
net.(*pollDesc).WaitRead(0x107bc138, 0x0, 0x0)
    /root/go/src/net/fd_poll_runtime.go:78 +0x30
net.(*netFD).accept(0x107bc100, 0x0, 0xb41880d8, 0x107be070)
    /root/go/src/net/fd_unix.go:408 +0x21c
net.(*TCPListener).AcceptTCP(0x107ea028, 0x7f67c, 0x0, 0x0)
    /root/go/src/net/tcpsock_posix.go:254 +0x4c
net/http.tcpKeepAliveListener.Accept(0x107ea028, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/net/http/server.go:2135 +0x3c
net/http.(*Server).Serve(0x107bc0c0, 0xb41880b8, 0x107ea028, 0x0, 0x0)
    /root/go/src/net/http/server.go:1887 +0x88
net/http.(*Server).ListenAndServe(0x107bc0c0, 0x0, 0x0)
    /root/go/src/net/http/server.go:1877 +0x124
net/http.ListenAndServe(0x381b90, 0x3, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/net/http/server.go:1967 +0x90
main.managementInterface()
    /root/stratux/main/managementinterface.go:247 +0x38c
created by main.main
    /root/stratux/main/gen_gdl90.go:892 +0x4ec

goroutine 26 [chan receive]:
main.monitorDHCPLeases()
    /root/stratux/main/network.go:224 +0x60
created by main.initNetwork
    /root/stratux/main/network.go:328 +0x124

goroutine 34 [chan receive]:
main.(*uibroadcaster).writer(0x107be010)
    /root/stratux/main/uibroadcast.go:32 +0x48
created by main.NewUIBroadcaster
    /root/stratux/main/uibroadcast.go:18 +0xf4

goroutine 27 [runnable]:
main.messageQueueSender()
    /root/stratux/main/network.go:190 +0x380
created by main.initNetwork
    /root/stratux/main/network.go:329 +0x138

goroutine 28 [IO wait]:
net.runtime_pollWait(0xb4187fc8, 0x72, 0x10778000)
    /root/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0x107d6638, 0x72, 0x0, 0x0)
    /root/go/src/net/fd_poll_runtime.go:73 +0x34
net.(*pollDesc).WaitRead(0x107d6638, 0x0, 0x0)
    /root/go/src/net/fd_poll_runtime.go:78 +0x30
net.(*netFD).readFrom(0x107d6600, 0x1081e000, 0x5dc, 0x5dc, 0x0, 0x0, 0x0, 0xb532e018, 0x10778000)
    /root/go/src/net/fd_unix.go:259 +0x20c
net.(*IPConn).ReadFromIP(0x10806058, 0x1081e000, 0x5dc, 0x5dc, 0x3b023, 0x1, 0x0, 0x0)
    /root/go/src/net/iprawsock_posix.go:75 +0xec
net.(*IPConn).ReadFrom(0x10806058, 0x1081e000, 0x5dc, 0x5dc, 0x2f3180, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/net/iprawsock_posix.go:109 +0xe4
golang.org/x/net/icmp.(*PacketConn).ReadFrom(0x107cc080, 0x1081e000, 0x5dc, 0x5dc, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go_path/src/golang.org/x/net/icmp/endpoint.go:59 +0xfc
main.sleepMonitor()
    /root/stratux/main/network.go:274 +0x1c0
created by main.initNetwork
    /root/stratux/main/network.go:330 +0x14c

goroutine 29 [chan receive]:
main.printStats()
    /root/stratux/main/gen_gdl90.go:814 +0x8c
created by main.main
    /root/stratux/main/gen_gdl90.go:898 +0x504

goroutine 30 [chan receive]:
main.cpuTempMonitor()
    /root/stratux/main/gen_gdl90.go:442 +0x4c
created by main.main
    /root/stratux/main/gen_gdl90.go:901 +0x518

goroutine 35 [chan receive]:
main.(*uibroadcaster).writer(0x107be020)
    /root/stratux/main/uibroadcast.go:32 +0x48
created by main.NewUIBroadcaster
    /root/stratux/main/uibroadcast.go:18 +0xf4

goroutine 50 [chan receive]:
main.icmpEchoSender(0x107cc080)
    /root/stratux/main/network.go:233 +0x70
created by main.sleepMonitor
    /root/stratux/main/network.go:270 +0x134
jpoirier commented 8 years ago

The error below indicates that an old version of gortlsdr and/or librtsdr is being used. librtlsdr's rtlsdr_set_center_freq function returns an error when you send a freq correct that's already set. gortlsdr was initial changed to intercept the returned error and change it to a non-error return then we forked the librtlsdr code and made the changes their.

"SetFreqCorrection 0 Failed, error: invalid parameter(s)

On Sat, Jan 2, 2016 at 10:33 AM, bradanlane notifications@github.com wrote:

We may be picking up changes from external sources as well (such as the various Go repositories) and possibly changes from dump1090, dump978, and rtlsdr ?

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168404777.

jpoirier commented 8 years ago

Actually, it's the two parts GetFreqCorrection 0 then SetFreqCorrection 0 which is the indicator...

On Sat, Jan 2, 2016 at 12:22 PM, Joseph Poirier jdpoirier@gmail.com wrote:

The error below indicates that an old version of gortlsdr and/or librtsdr is being used. librtlsdr's rtlsdr_set_center_freq function returns an error when you send a freq correct that's already set. gortlsdr was initial changed to intercept the returned error and change it to a non-error return then we forked the librtlsdr code and made the changes their.

"SetFreqCorrection 0 Failed, error: invalid parameter(s)

On Sat, Jan 2, 2016 at 10:33 AM, bradanlane notifications@github.com wrote:

We may be picking up changes from external sources as well (such as the various Go repositories) and possibly changes from dump1090, dump978, and rtlsdr ?

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168404777.

bradanlane commented 8 years ago

This is deliberately older code: V0.4r4. What we are trying to track down is the crash.

jpoirier commented 8 years ago

Sorry, didn't catch that. Then obviously you can ignore that error message.

On Sat, Jan 2, 2016 at 12:27 PM, bradanlane notifications@github.com wrote:

This is deliberately older code: V0.4r4. What we are trying to track down is the crash.

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168414616.

jpoirier commented 8 years ago

btw, what version of Go are you compiling with?

On Sat, Jan 2, 2016 at 12:42 PM, Joseph Poirier jdpoirier@gmail.com wrote:

Sorry, didn't catch that. Then obviously you can ignore that error message.

On Sat, Jan 2, 2016 at 12:27 PM, bradanlane notifications@github.com wrote:

This is deliberately older code: V0.4r4. What we are trying to track down is the crash.

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168414616.

bradanlane commented 8 years ago

Since this is a backup image from November it has GO 1.5.1 (IIRC)

Ergonomicmike commented 8 years ago

Don't know if this is related: am flying v0.5b3 now w/ the MalcolmRobb 1090dump. Had to reboot the Stratux four times in the Phx area when on 978. Kept losing heartbeat. Even had to pull SDR out 'cause it wouldn't boot. After out of Phx and no traffic, didn't have to reboot anymore. Wasn't logging - Sorry.

cyoung commented 8 years ago

@Ergonomicmike - I assume you mean v0.5b3 plus those commands to update to MalcolmRobb, or the most current master branch?

@jpoirier - do you think we should use submodules for critical parts like gortlsdr and keep it at a specific commit?

Ergonomicmike commented 8 years ago

@cyoung Yes, the special MR code you talked me thru. Otherwise stock v0.5b3.

I'll turn on logging for next two flights. Meantime looks like someone opened a similar issue.

skypuppy commented 8 years ago

Have you by chance done a lot of power-offs without shutdown? Missing files points in that direction. Try a fresh image with a fresh SD Card. If that still happens, then I would suspect hardware failure on the Pi2.

On 01/02/2016 09:10 AM, bradanlane wrote:

It would appear to be a timing issue. Possibly an protected common resource? I am running V0.4r4 and get the same issue as with the current code. I was able to get Stratux to start up if I unplugged the SDR, started Stratux and then plugged the SDR in.

Here is one dump of the crash - in case any of this makes sense to somone:

|root@stratuxpi:~# /usr/bin/gen_gdl90 1970/01/01 00:02:54 Stratux v0.4r4 (d0d466896ee0b0efbfb66868e695327dd3021553) starting. 1970/01/01 00:02:54 can't read settings /etc/stratux.conf: open /etc/stratux.conf: no such file or directory 1970/01/01 00:02:55 ===== UAT Device name: Generic RTL2832U OEM ===== Found Rafael Micro R820T tuner 1970/01/01 00:02:56 GetUsbStrings - Realtek RTL2838UHIDIR 00000001 1970/01/01 00:02:56 GetTunerType: RTLSDR_TUNER_R820T 1970/01/01 00:02:56 SetTunerGainMode Successful 1970/01/01 00:02:56 SetTunerGain Successful Exact sample rate is: 2083334.141630 Hz [R82XX] PLL not locked! 1970/01/01 00:02:56 SetSampleRate - rate: 2083334 1970/01/01 00:02:56 GetSampleRate: 2083334 1970/01/01 00:02:56 GetXtalFreq - Rtl: 28800000, Tuner: 28800000 1970/01/01 00:02:56 SetXtalFreq - Center freq: 28800000, Tuner freq: 28800000 1970/01/01 00:02:56 SetCenterFreq 978MHz Successful 1970/01/01 00:02:56 GetCenterFreq: 978000000 1970/01/01 00:02:56 Setting Bandwidth: 1000000 1970/01/01 00:02:57 SetTunerBw 1000000 Successful 1970/01/01 00:02:57 ResetBuffer Successful 1970/01/01 00:02:57 GetFreqCorrection: 0 1970/01/01 00:02:57 SetFreqCorrection 0 Failed, error: invalid parameter(s) 1970/01/01 00:02:57 ReadSync Failed - error: overflow SIGILL: illegal instruction PC=0x6e386 m=5 signal arrived during cgo execution goroutine 36 [syscall, locked to thread]: runtime.cgocall(0x13338, 0x10800ce4, 0x0) /root/go/src/runtime/cgocall.go:120 +0x11c fp=0x10800cc8 sp=0x10800cb0 github.com/jpoirier/gortlsdr._Cfunc_rtlsdr_close(0xb3000468, 0x0) github.com/jpoirier/gortlsdr/_obj/_cgo_gotypes.go:128 +0x34 fp=0x10800ce0 sp=0x10800cc8 github.com/jpoirier/gortlsdr.(_Context).Close(0x1076a2b8, 0x0, 0x0) /root/go_path/src/github.com/jpoirier/gortlsdr/rtlsdr.go:228 +0x30 fp=0x10800d1c sp=0x10800ce0 main.sdrReader() /root/stratux/main/sdr.go:127 +0xeb4 fp=0x10800fd4 sp=0x10800d1c runtime.goexit() /root/go/src/runtime/asm_arm.s:1036 +0x4 fp=0x10800fd4 sp=0x10800fd4 created by main.sdrWatcher /root/stratux/main/sdr.go:154 +0xd8 goroutine 1 [syscall]: syscall.Syscall(0x3, 0x0, 0x107e8000, 0x1000, 0x0, 0x0, 0x0) /root/go/src/syscall/asm_linux_arm.s:17 +0x8 syscall.read(0x0, 0x107e8000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /root/go/src/syscall/zsyscall_linux_arm.go:783 +0x78 syscall.Read(0x0, 0x107e8000, 0x1000, 0x1000, 0xc81a0, 0x0, 0x0) /root/go/src/syscall/syscall_unix.go:160 +0x4c os.(_File).read(0x1076a0e8, 0x107e8000, 0x1000, 0x1000, 0xc5dd8, 0x0, 0x0) /root/go/src/os/file_unix.go:211 +0x54 os.(_File).Read(0x1076a0e8, 0x107e8000, 0x1000, 0x1000, 0xc7284, 0x0, 0x0) /root/go/src/os/file.go:95 +0x7c bufio.(_Reader).fill(0x10745f44) /root/go/src/bufio/bufio.go:97 +0x1c4 bufio.(_Reader).ReadSlice(0x10745f44, 0x3b00a, 0x0, 0x0, 0x0, 0x0, 0x0) /root/go/src/bufio/bufio.go:328 +0x264 bufio.(_Reader).ReadBytes(0x10745f44, 0x2f310a, 0x0, 0x0, 0x0, 0x0, 0x0) /root/go/src/bufio/bufio.go:406 +0x7c bufio.(_Reader).ReadString(0x10745f44, 0x10745f0a, 0x0, 0x0, 0x0, 0x0) /root/go/src/bufio/bufio.go:446 +0x44 main.main() /root/stratux/main/gen_gdl90.go:906 +0x5e8 goroutine 17 [syscall, locked to thread]: runtime.goexit() /root/go/src/runtime/asm_arm.s:1036 +0x4 goroutine 20 chan receive: github.com/golang/glog.(_loggingT).flushDaemon(0x51b590) /root/go_path/src/github.com/golang/glog/glog.go:882 +0x60 created by github.com/golang/glog.init.1 /root/gopath/src/github.com/golang/glog/glog.go:410 +0x2cc goroutine 3 chan receive: main.sdrWatcher() /root/stratux/main/sdr.go:144 +0x4c created by main.sdrInit /root/stratux/main/sdr.go:168 +0x3c goroutine 4 chan receive: main.uatReader() /root/stratux/main/sdr.go:132 +0x3c created by main.sdrInit /root/stratux/main/sdr.go:169 +0x50 goroutine 5 chan receive: /root/stratux/godump978.ProcessDataFromChannel() /root/stratux/godump978/godump978.go:59 +0x40 created by main.sdrInit /root/stratux/main/sdr.go:171 +0x68 goroutine 22 [sleep]: time.Sleep(0x3b9aca00, 0x0) /root/go/src/runtime/time.go:59 +0x104 main.esListen() /root/stratux/main/traffic.go:380 +0x48 created by main.initTraffic /root/stratux/main/traffic.go:539 +0xe8 goroutine 23 chan receive: main.pollRY835AI() /root/stratux/main/ry835ai.go:607 +0x58 created by main.initRY835AI /root/stratux/main/ry835ai.go:630 +0x8c goroutine 24 [select]: main.heartBeatSender() /root/stratux/main/gen_gdl90.go:356 +0x170 created by main.main /root/stratux/main/gen_gdl90.go:890 +0x4d8 goroutine 25 [IO wait]: net.runtime_pollWait(0xb4188040, 0x72, 0x10778000) /root/go/src/runtime/netpoll.go:157 +0x60 net.(_pollDesc).Wait(0x107bc138, 0x72, 0x0, 0x0) /root/go/src/net/fd_poll_runtime.go:73 +0x34 net.(_pollDesc).WaitRead(0x107bc138, 0x0, 0x0) /root/go/src/net/fd_poll_runtime.go:78 +0x30 net.(_netFD).accept(0x107bc100, 0x0, 0xb41880d8, 0x107be070) /root/go/src/net/fd_unix.go:408 +0x21c net.(_TCPListener).AcceptTCP(0x107ea028, 0x7f67c, 0x0, 0x0) /root/go/src/net/tcpsock_posix.go:254 +0x4c net/http.tcpKeepAliveListener.Accept(0x107ea028, 0x0, 0x0, 0x0, 0x0) /root/go/src/net/http/server.go:2135 +0x3c net/http.(_Server).Serve(0x107bc0c0, 0xb41880b8, 0x107ea028, 0x0, 0x0) /root/go/src/net/http/server.go:1887 +0x88 net/http.(_Server).ListenAndServe(0x107bc0c0, 0x0, 0x0) /root/go/src/net/http/server.go:1877 +0x124 net/http.ListenAndServe(0x381b90, 0x3, 0x0, 0x0, 0x0, 0x0) /root/go/src/net/http/server.go:1967 +0x90 main.managementInterface() /root/stratux/main/managementinterface.go:247 +0x38c created by main.main /root/stratux/main/gen_gdl90.go:892 +0x4ec goroutine 26

/root/stratux/main/network.go:224 +0x60 created by main.initNetwork /root/stratux/main/network.go:328 +0x124 goroutine 34 chan receive: main.(_uibroadcaster).writer(0x107be010) /root/stratux/main/uibroadcast.go:32 +0x48 created by main.NewUIBroadcaster /root/stratux/main/uibroadcast.go:18 +0xf4 goroutine 27 [runnable]: main.messageQueueSender() /root/stratux/main/network.go:190 +0x380 created by main.initNetwork /root/stratux/main/network.go:329 +0x138 goroutine 28 [IO wait]: net.runtime_pollWait(0xb4187fc8, 0x72, 0x10778000) /root/go/src/runtime/netpoll.go:157 +0x60 net.(_pollDesc).Wait(0x107d6638, 0x72, 0x0, 0x0) /root/go/src/net/fd_poll_runtime.go:73 +0x34 net.(_pollDesc).WaitRead(0x107d6638, 0x0, 0x0) /root/go/src/net/fd_poll_runtime.go:78 +0x30 net.(_netFD).readFrom(0x107d6600, 0x1081e000, 0x5dc, 0x5dc, 0x0, 0x0, 0x0, 0xb532e018, 0x10778000) /root/go/src/net/fd_unix.go:259 +0x20c net.(_IPConn).ReadFromIP(0x10806058, 0x1081e000, 0x5dc, 0x5dc, 0x3b023, 0x1, 0x0, 0x0) /root/go/src/net/iprawsock_posix.go:75 +0xec net.(_IPConn).ReadFrom(0x10806058, 0x1081e000, 0x5dc, 0x5dc, 0x2f3180, 0x0, 0x0, 0x0, 0x0) /root/go/src/net/iprawsock_posix.go:109 +0xe4 golang.org/x/net/icmp.(_PacketConn).ReadFrom(0x107cc080, 0x1081e000, 0x5dc, 0x5dc, 0x0, 0x0, 0x0, 0x0, 0x0) /root/go_path/src/golang.org/x/net/icmp/endpoint.go:59 +0xfc main.sleepMonitor() /root/stratux/main/network.go:274 +0x1c0 created by main.initNetwork /root/stratux/main/network.go:330 +0x14c goroutine 29 chan receive: main.printStats() /root/stratux/main/gen_gdl90.go:814 +0x8c created by main.main /root/stratux/main/gen_gdl90.go:898 +0x504 goroutine 30 [chan receive]: main.cpuTempMonitor() /root/stratux/main/gen_gdl90.go:442 +0x4c created by main.main /root/stratux/main/gen_gdl90.go:901 +0x518 goroutine 35 chan receive: main.(_uibroadcaster).writer(0x107be020) /root/stratux/main/uibroadcast.go:32 +0x48 created by main.NewUIBroadcaster /root/stratux/main/uibroadcast.go:18 +0xf4 goroutine 50 chan receive: main.icmpEchoSender(0x107cc080) /root/stratux/main/network.go:233 +0x70 created by main.sleepMonitor /root/stratux/main/network.go:270 +0x134 |

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168399586.

bradanlane commented 8 years ago

This is my development system so it always gets a systematic shutdown via SSH. This unit had never been flown.

When I am able to sit down again with the system, I'll swap out the hardware.

jpoirier commented 8 years ago

It seems the targets went from mostly stationary to erratically moving in an uncomfortably short period of time. :(

@cyoung - as far as Go code is concerned, the Go compiler version 1.6 (in beta at the moment but usable) supports vendoring natively and that's what I'd recommend as opposed to submodules.

On Sat, Jan 2, 2016 at 3:57 PM, cyoung notifications@github.com wrote:

@Ergonomicmike https://github.com/Ergonomicmike - I assume you mean v0.5b3 plus those commands to update to MalcolmRobb, or the most current master branch?

@jpoirier https://github.com/jpoirier - do you think we should use submodules for critical parts like gortlsdr and keep it at a specific commit?

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168431041.

jpoirier commented 8 years ago

Was looking for justification to buy either a BladeRF (http://nuand.com/) or a USRP B200 (http://www.ettus.com/product/details/UB200-KIT) board. I may have found one; it'd be really handy to have a test bench setup that can transmit ads-b out messages.

And to preempt any concerns, yes I have access to rf shielded test enclosures and labs.

On Sat, Jan 2, 2016 at 4:31 PM, Joseph Poirier jdpoirier@gmail.com wrote:

It seems the targets went from mostly stationary to erratically moving in an uncomfortably short period of time. :(

@cyoung - as far as Go code is concerned, the Go compiler version 1.6 (in beta at the moment but usable) supports vendoring natively and that's what I'd recommend as opposed to submodules.

On Sat, Jan 2, 2016 at 3:57 PM, cyoung notifications@github.com wrote:

@Ergonomicmike https://github.com/Ergonomicmike - I assume you mean v0.5b3 plus those commands to update to MalcolmRobb, or the most current master branch?

@jpoirier https://github.com/jpoirier - do you think we should use submodules for critical parts like gortlsdr and keep it at a specific commit?

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168431041.

cyoung commented 8 years ago

Charge it to the company card!

bradanlane commented 8 years ago

Here's one data point to consider:

I loaded V0.4r4 and had the long trace crash dump I posted above. To get past the crash on initial start, I had to remove the SDR and then execute /etc/init.d/stratux start

This allowed Stratux to start and then I plugged the SDR in.

This setup has been running for 9.5 hours and counting.

In that time the USB GPS managed a lock (sitting in my office window) and it received the occasional UAT traffic passing above.

bradanlane commented 8 years ago

FYI - I just rebooted and Stratux crashed immediately. I unplugged the SDR and was able to start Stratux. Then I plugged the SDR back in.

I will let it run for as long as it is willing (or when I wake up, which ever is longer).

BTW: the logs won't show the crash. They will likely show some RTL error and that's it. The crash dump goes to "screen". In my recent test with V0.4r4, the only error is the "ReadSync failed" error which bubbled up from the SDR.

cyoung commented 8 years ago

You could modify /usr/bin/start_uat and add to the end ">/var/log/templog 2>&1"

jpoirier commented 8 years ago

Wish I could but since the purchase of alcatel-lucent any excess expenditure approvals are difficult to come by! We did have a couple B200 boards in-house that I was able to play with for a bit but they ended up in Germany after the project that was using them got killed.

On Sat, Jan 2, 2016 at 7:46 PM, cyoung notifications@github.com wrote:

Charge it to the company card!

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168449452.

cyoung commented 8 years ago

I mean the stratux company card :)

Well, I've been thinking about getting a HackRF - do you think BladeRF is more appropriate for our purposes?

skypuppy commented 8 years ago

For me, if I got a HackRF, the temptation would be TOO GREAT to turn it into ADS-B in and OUT. Which is not only dangerous to me and others but also somewhat illegal to use as an OUT. It is very, very tempting, though.

Be velly, velly careful with transmitters in the aviation band.

Skypuppy On 01/02/2016 09:26 PM, cyoung wrote: > I mean the stratux company card :) > > Well, I've been thinking about getting a HackRF - do you think BladeRF > is more appropriate for our purposes? > > — > Reply to this email directly or view it on GitHub > https://github.com/cyoung/stratux/issues/174#issuecomment-168456220.
jpoirier commented 8 years ago

I think HackRF would work fine but look here for a comparison: http://www.taylorkillian.com/2013/08/sdr-showdown-hackrf-vs-bladerf-vs-usrp.html

8-bit IQs, half duplex, slower interface speeds, and lower sample rate kind of limit its use.

On Sat, Jan 2, 2016 at 9:26 PM, cyoung notifications@github.com wrote:

I mean the stratux company card :)

Well, I've been thinking about getting a HackRF - do you think BladeRF is more appropriate for our purposes?

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168456220.

jpoirier commented 8 years ago

You could use it as a test bench to transmit ads-b provided the tx power is low enough such that it doesn't radiate out of the confines of your home.

On Sat, Jan 2, 2016 at 9:51 PM, skypuppy notifications@github.com wrote:

For me, if I got a HackRF, the temptation would be TOO GREAT to turn it into ADS-B in and OUT. Which is not only dangerous to me and others but also somewhat illegal to use as an OUT. It is very, very tempting, though.

Be velly, velly careful with transmitters in the aviation band.

Skypuppy On 01/02/2016 09:26 PM, cyoung wrote: > I mean the stratux company card :) > > Well, I've been thinking about getting a HackRF - do you think BladeRF > is more appropriate for our purposes? > > — > Reply to this email directly or view it on GitHub > https://github.com/cyoung/stratux/issues/174#issuecomment-168456220. — Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168457219.
skypuppy commented 8 years ago

What, you think I got that kind of self-control and willpower? Hahahahahaahahaha.

(Just trying to be funny here, guys. No worries.)

On 01/02/2016 10:00 PM, Joseph Poirier wrote:

You could use it as a test bench to transmit ads-b provided the tx power is low enough such that it doesn't radiate out of the confines of your home.

On Sat, Jan 2, 2016 at 9:51 PM, skypuppy notifications@github.com wrote:

For me, if I got a HackRF, the temptation would be TOO GREAT to turn it into ADS-B in and OUT. Which is not only dangerous to me and others but also somewhat illegal to use as an OUT. It is very, very tempting, though.

Be velly, velly careful with transmitters in the aviation band.

Skypuppy On 01/02/2016 09:26 PM, cyoung wrote: > I mean the stratux company card :) > > Well, I've been thinking about getting a HackRF - do you think BladeRF > is more appropriate for our purposes? > > — > Reply to this email directly or view it on GitHub > https://github.com/cyoung/stratux/issues/174#issuecomment-168456220. — Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168457219.

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168457428.

jpoirier commented 8 years ago

On Sat, Jan 2, 2016 at 10:04 PM, skypuppy notifications@github.com wrote:

What, you think I got that kind of self-control and willpower? Hahahahahaahahaha.

(Just trying to be funny here, guys. No worries.)

Mischievous ADS-B transmissions would most probably be looked at as domestic terrorism and likely get said individual an unlimited free stay at a domestic rendition center sans due process most riki tik.

bradanlane commented 8 years ago

Back to the issue (smirk) :-)

Once I started Stratux without the SDR and then plugged it in, the V0.4r4 has been running an additional 9.5 hours. This means I've had two runs - each in excess of 8 hours.

Now I need to figure out what is causing the startup crash with ReadSync failed - error: overflow. My suspicion is some change in the RTL code that gets pulled in during build.

As a parallel test, I'll take the current code, disable the SDR power savings "cycling" and see if I get similar successful run durations.

bradanlane commented 8 years ago

I swapped in a pristine new virgin Pi with V0.4r4 and a single SDR and it booted and ran without error. Upon the next reboot, the crash problem returned.

This has me concerned that something I am doing is stressing the hardware somehow (or just really bad luck).

bradanlane commented 8 years ago

I may be crazy but I tried a third pristine new virgin Pi with a fresh image of my V0.4r4 and pristine new virgin SDR.

It failed to boot. Manually starting /usr/bin/gen_gdl90 showed the same crash with the ReadSync Failed - error: overflow

Manually starting /usr/bin/gen_gdl90 with the SDR not inserted, starts. Then inserting the SDR and it remains operating.

bradanlane commented 8 years ago

I'm back on the current code stream. Either by luck or using the "start w/o SDR" trick, I can get Stratux to run. However, with the power saver code active, it will crash sooner or later. With that code disabled, it runs longer - how long is being tested now.

cyoung commented 8 years ago

This isn't the OrangePi hardware is it?

bradanlane commented 8 years ago

No. I assume any orange issues are mine to resolve in quiet desperation :-)

mooneyguy commented 8 years ago

I just bought a new RPi2 because my old one died running the V0.5b3 (worked fine in the plane on the ground but at start up on the first flight, it died). What does that mean? The wifi light and both antennas did not light up but the red and green lights on the RPi2 board were solid red and green). The gps module did not light up at all. The configuration did work fine in the previous version. I put in the new RPi2 last night and everything worked except the gps would only have a power light on solid so I just disconnected the gps for now since I am mostly interested in traffic and weather. However, while on the ground with the new board installed and using V0.5b3 I still get 0 feet altitude for some planes but the other information looks good.

From: bradanlane [mailto:notifications@github.com] Sent: Sunday, January 03, 2016 5:59 AM To: cyoung/stratux Subject: Re: [stratux] RTL error messages when no UAT traffic ? - "failed with -4" (#174)

I swapped in a pristine new virgin Pi with V0.4r4 and a single SDR and it booted and ran without error.

This has me concerned that something we are doing is stressing the hardware somehow (or just really bad luck).

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/174#issuecomment-168500387 . https://github.com/notifications/beacon/APEqprjGfYxYqTKxSjDrIVZ02mX_ZuIdks5pWSCrgaJpZM4G9P6d.gif

bradanlane commented 8 years ago

should we merge issue #174 and issue #180 ?

Ergonomicmike commented 8 years ago

I had to pull the SDR to get the Pi to reboot again today, this time with v0.5b4. (On the ground at home, after a soft-reboot.)

Since it's new firmware but since everything else is the same, I'm beginning to wonder if my particular new problem of having the pull the SDR out before booting isn't a power (a.k.a. KMASHI) issue? Regardless of why my Pi started requiring reboots in flight (although I suppose that could also be a KMASHI issue), it might be that, during initialization, with warm components, there's a power demand when the SDR comes on line that causes the KMASHI to drop too much and stalls the boot process? I say this because I noticed that the green light on the Pi stopped flickering when the SDR's LED came on.

I will see if I have a 5v AC P/S that works with the Pi and will test later to see if I can get thru reboots using that with the SDR connected.

bradanlane commented 8 years ago

I'm running with a stable regulated power supply (measuring both voltage and current) and I am seeing the same issue:

had to pull the SDR to get the Pi to reboot again today, this time with v0.5b4