cyoung / stratux

Aviation weather and traffic receiver based on RTL-SDR.
BSD 3-Clause "New" or "Revised" License
1.06k stars 363 forks source link

Loss of Foreflight and UI connection in flight #420

Closed cbjohnsonjr closed 7 years ago

cbjohnsonjr commented 8 years ago
  1. Stratux version: 0.8r2
  2. Stratux config: Raspberry Pi 3, 1 SDR Nano 2( 978 only), no GPS, no AHRS, Battery - Anker® 2nd Gen Astro E5 16000mAh, short micro usb cable
  3. EFB app and version : Foreflight v7.6.1 and Flightplan same issue
  4. Description of your issue : Loss of Forelight connection and UI connection occurs every flight with duration approximately 1.0 - 1.5 hours. Jet aircraft operating FL 350-450. I unplug power then repower and device operates normally for a few minutes then problem reoccurs. If I leave power off a for longer period then last operates for 1.0+ hour then problem starts again. I've tried different batteries, full sized receiver, placing receiver in cooler spot (typical reporting temp 50 degrees C), all no change. Wifi never goes away only EFB and UI connection. On the ground with no ADSB tower connection continues to operate without problems. Everything else works
peepsnet commented 8 years ago

Did you turn AHRS off in the webUI???

https://github.com/cyoung/stratux/issues/332

Ergonomicmike commented 8 years ago

Sounds like a classic overheating problem. But since you put it in a cooler spot ... Maybe try a different SD card in case it's failing with a little heat? (No ADS-B on ground, things don't get as hot.)

idriss-ophawk commented 8 years ago

I have the same issue with:

It started to appear after I bought a more powerful antenna. I now get 6-10 UAT towers but my stratux freezes after 7-10 minutes. By freeze I mean that my ipad is still connected to startux in wifi but the control center and all other operations are not available (GPS, UAT, etc.). I have to unplug/plug it to restart it and it works again. I can't get it work longer than 10min in the air. On the ground, it lasts longer (30min+).

I enabled the logs in flight, the file is attached without the "Invalid NMEA string: Invalid NMEA message" entries which are cluttering the file but don't seem to be the issue.

stratux-clean.txt

I also have the UAT and GPS log if needed.

cbjohnsonjr commented 8 years ago

I purchased another Raspberry Pi 3 and SD card, same exact performance. I've tried multiple batteries, multiple receivers, multiple SD cards, larger SD cards, smaller SD cards, multiple Raspberry Pi's, warmer, cooler, shutdown through UI, unplugged no shutdown the only variable to make a difference is received data. No data no problems. Lots of data problem. I believe this is a software issue related to the traffic. I typically have a half dozen or more stations and cover a lot of stations over an hour period. Attached is my log from the last few flights. stratux.pdf

Ergonomicmike commented 8 years ago

Yes, it is sounding like a software issue. I only see a few towers here in the desert SW, so don't stress the system. Questions: 1) iOS only? (I don't know why Android would make a difference, but you both seem to be reporting the same problem with iPads.) 2) What kind of message rate numbers are you seeing on UAT at altitude? 3) Do you both fly in the same region? (Perhaps a bad tower in the mix?) 4) Flying so high in jets, I wonder if there's an inherent problem with receiving FIS-B data from disparate regions? (Probably not, since others fly high with a Stratus and aren't reporting problems.) 5) Also, FF seems to be common to you both. What happens if you run the Sratux w/o FF running and just watch the web UI Status page? Does the Stratux freeze then?

idriss-ophawk commented 8 years ago

For me the issue is independent from Foreflight since it fails even without my ipad connected. I fly in the NYC class B area usually low altitude in a cessna 150. I see 6-10 towers. During my log capture when the stratux failed, I was 10 nm outside class B at 4,000ft MSL. The total number of traffic targets tracked was around 173 when it failed.

Ergonomicmike commented 8 years ago

Interesting. I've never seen that much traffic. The next step might be to isolate whether it's 978 traffic or 1090 traffic that causes the freeze. (Guessing 978, since that appears to be more compute intensive.) The Stratux is throttled back to 600 MHz by way of Powersave governor. Can you either run htop or Raspi or change the governor to Performance to see if the CPU is the bottleneck? (If Performance solves the problem, we could try the stock OnDemand to see if that gets us by.)

cbjohnsonjr commented 8 years ago

I am only receiving 978 so I think your guess is correct. I will research how to run stop or Raspi or change the governor to further research. If you could provide a link to guide me that would help. Thanks

Ergonomicmike commented 8 years ago

Having said that it's a 978 traffic problem, there's an impressive graphic posted in one of the closed issues where either @AvSquirrel or @jpoirier created a replay of, like, 1000 targets. (They made an interesting spiral design.) Stratux handled that okay, although there's been some changes made since then, I think. Am looking for the screen shot now.

monorailorange commented 8 years ago

How many times has this issue occurred? Are there any predictable variables to see if this happens again? I have seen RAM fail on a Pi2 and it took the whole board down once the failed address was reached. Could be that.

cbjohnsonjr commented 8 years ago

This happens every flight for me. It's very predictable. An hour or so into a flight the connection to the UI and Foreflight and Fltplan app quits. The wifi signal continues and the green activity light continues to flicker. Resetting the power clears the problem for a short period unless the power is removed for a long period.

Ergonomicmike commented 8 years ago

@cbjohnsonjr There were instuctions in both this github and the reddit forum on how to change the governor. Unfortunately a search in the reddit doesn't find it anymore. And I haven't found a way to search for text in the github. The change involves SSH'ing into the Pi and editing a line in a config file.

Since you're running an iPad, Raspi isn't an option, since that's an Android app. To install htop, you also need to SSH in. (To install it and to run it in flight.)

Ergonomicmike commented 8 years ago

@monorailorange cb says, in an earlier comment, that he's tried different Pi's. So, although the bad RAM theory is a good one, it doesn't seem like two differnt Pi's would have the same bad RAM. Plus, one other guy is having a similar problem. Perhaps they're both reaching a stack overflow (WAG) or some memory or address space limit in RAM? Or could they be saturating a buffer?

monorailorange commented 8 years ago

My apologies. I overlooked the two-board statement. It's possible it could be a stack overflow, or a saturated buffer... I need to see the logs to look into this more.

Ergonomicmike commented 8 years ago

@cbjohnsonjr Since you seem to have two of everything, it would be an interesting experiment to fly both Strautux's at the same time to see if they both peter out at the same time. That would definitely point to a software issue. (Would need to change the WiFi channel on one of the Stratux's along with its SSID to make life easier.)

idriss-ophawk commented 8 years ago

I am going to fly tonight, and should be able to reproduce this issue - it is also very predictable for me and occurs every 7-10 minutes on my RPi2. I will disable the GPS in the settings to make sure only UAT is activated and logged. Will revert with the logs.

Is there anything else I should try while I am up there?

Ergonomicmike commented 8 years ago

I suggest reimaging the sd card (to wipe it to make sure there's room for logs) with v0.8r2 and then .sh'ing to to latest build. http://updates.stratux.me/builds/

Test it before you get to the airport and configure settings ahead of time too. Looking forward to what happens.

Ergonomicmike commented 8 years ago

Also, if you plan to land in the middle of your flight, perhaps you could swap in an SD card that has the governor set for Performance to see what happens coming back home. (Although I'm wondering aloud how many towers the software is written for? Is there an upper bound?)

ssokol commented 8 years ago

I've been working with a user to diagnose an issue that result in a similar state: the wifi remains connected but the data stops flowing to both the web UI and EFB. I had him SSH in and it turns out that gen_gdl90 is dying.

His configuration includes a Raspberry PI 2 with Edimax and one NooElec NESDR Nano 2 doing UAT. Testing is on the ground with CPU temp stable at 40° C.

He's located within line-of-sight of an ADS-B tower and has some suspicions that the volume of UAT data or something in the UAT messages received is the cause. Apparently the system stops responding when the message rate reaches ~500 UAT messages per minute. That doesn't sound like all that much, so I wonder if there was a format change in the data that's causing the problem.

Attached is a picture of the stratux.log file. I had him tail it until the process died. Notice the odd -99.00 temp that appears a couple of places in the log, including the last entry.

On Tue, May 24, 2016 at 11:48 AM, Ergonomicmike notifications@github.com wrote:

Also, if you plan to land in the middle of your flight, perhaps you could swap in an SD card that has the governor set for Performance to see what happens coming back home. (Although I'm wondering aloud how many towers the software is written for? Is there an upper bound?)

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-221333537

Steven Sokol 408 Camelot Drive Liberty, MO 64068

mobile: +1 816-806-8844 fax: +1 816-817-0441

idriss-ophawk commented 8 years ago

@Ergonomicmike Thanks vm - just updated to 47271df764. I only have 1 sdcard, will try to change the governor to "performance" in flight with ssh on the return leg.

Ergonomicmike commented 8 years ago

@ssokol Yeah, 500 UAT's sound low for failure. I found an old screenshot showing 3700 for me going strong.

I plan to go flying tomorrow. We will see what we will see.

P.S. I saw a commit recently where Chris fixed the -99 degree CPU temp thing. OTOH, he told me once that there's a known hardware issue in the Pi2 responsible for this. Wondering aloud if the fix broke anything? ( Since you said that the -99 popped up toward the failure. A clue there?)

ghost commented 8 years ago

In summary, it seems like the common threads are:

I'm on board with the idea that this could be a software problem. One thing that came to mind is the handling of UAT tower messages in gen_gdl90.go.

Stratux uses a map data structure (ADSBTowers) to store information about each of the towers it's receiving, including location, signal strength, and number of messages received. Maps aren't thread safe for writes, so reads and writes both need to be mutex protected. We do this for our traffic data structure, but somehow ADSBTowers was missed.

The commits below (.sh package and Github comparison) are my first go at fixing it. It compiles and runs, but I haven't had a chance to test it on UAT traffic yet. There's an outside chance it will completely deadlock and not receive anything. I hope to get out to our local tower and give it a run this evening.

update-stratux-v0.8r2-5be9ec80ef.sh.zip

https://github.com/cyoung/stratux/compare/master...AvSquirrel:uat-mutex

Ergonomicmike commented 8 years ago

Glad that the Squirrel is on it. If you bless it tonight, I'll plan to fly with tomorrow.

skypuppy commented 8 years ago

More data points: it happens to me at home, with very little 978 radio traffic. It occurs repeatedly during the time tests using the Pi3 and dmurray equipment along with nano 2's, while the Pi 2 with the big -ole sdr radios would go a week without crashing. I don't have a length of time when the crashes occur, but it is always less than 24 hours and seems to be less than 12 hours. This is all inside an air-conditioned house with 2.5A power supply. I will do it again but can't ssh into it to see if gen_gdl90 is still running or not because all outside comms are shut off.

jpoirier commented 8 years ago

On Tue, May 24, 2016 at 4:09 PM, AvSquirrel notifications@github.com wrote:

Stratux uses a map data structure (ADSBTowers) to store information about each of the towers it's receiving, including location, signal strength, and number of messages received. Maps aren't thread safe for writes, so reads and writes both need to be mutex protected. We do this for our traffic data structure, but somehow ADSBTowers was missed.

I hate to be that guy but these are the kinds of things that are eliminated when the project work flow includes/requires a code review prior to a merge.

-joe

Ergonomicmike commented 8 years ago

Since skypuppy is having freezes w/o 978 traffic, I wonder if the problem there is with the new Nano 2's? As Chris has shown with his FLIR, those dongles run very hot.

idriss-ophawk commented 8 years ago

ok so I have the results from my flight tonight around NYC class bravo with 6-14 UAT towers. did the test without and with an EFB, it didn't make any difference. only enabled 978 - no gps, no 1090, no AHRS. tests resulted in no GDL90 messages and web UI quit but wifi still active:

1- the version v0.8r2-47271df764 fails a lot faster than the tagged release v0.8r2 - after 1-2 minutes of activity. it even fails without 978 (i.e. no hardware enabled). 2 - AvSquirrell version v0.8r2-5be9ec80ef.sh works a lot better but still fails when I reached 10 UAT towers. It works fine on the ground without any UAT tower. 3 - changed the cpu governor to performance with AvSquirrell version v0.8r2-5be9ec80ef on my way back but no luck there, it still failed after a few minutes.

unfortunately, can't find much in the logs (attached), btw beginner question, the timestamps in the logs are not ordered sequentially sometimes, I guess it is linked to GPS clock sync vs RPi clock, but logs entries lines are added sequentially (append), aren't they?

stratux-logs-5be9ec80ef.txt

Thanks very much for looking into it and let me know if there is anything I can do to help.

Ergonomicmike commented 8 years ago

Hmmm... I ran f764 last night on the couch and it ran okay. I'm gonna run it again now to make sure.

It's strange that yours failed w/o any hardware hooked up. Mine's got 5 min now w/ everything hooked up. 10 min now, 22% w/ some 1090 coming in. You might have a hardware issue in addition to whatever else is going on.

Interesting about 10 towers being a trigger. Still probably software as opposed to CPU, especially since Performance didn't make a difference.

The Stratux shouldn't need a GPS to run, but I wonder if it needs it to match any time stamps with UAT? (Since there's Pi time and GPS time.) I've never noticed whether the last line in my log is an out of chronological order startup notice. Some strange SD card caching?

They might ask you for your uat log file.

I will update to AvSquirrel's now. There is no way I can get 10 towers around here, but I made a test card and will report back after flying tomorrow afternoon.

P.S. 20 min and still okay.

Ergonomicmike commented 8 years ago

@idriss-ophawk The easiest thing to try is a new SD card. 8GB is cheap enough. BTW, any errors on yours w/ fsck?

Ergonomicmike commented 8 years ago

I'm comparing his log to mine. I don't have any watchdog loops in a month long log.

ghost commented 8 years ago

@idriss-ophawk --

Thanks for the updates. This is unfortunate -- there's still obviously something else that isn't playing nice at high UAT message rates / tower counts.

Do you have the ability to connect a laptop to your Pi in flight and run Stratux over ssh? I'd like to capture a stack trace when it crashes so I can see which line is actually triggering the crash.

service stratux stop
/usr/bin/gen_gdl90

If / when it crashes, you'll see something like this:

2016/05/25 11:52:43 client connected: 192.168.10.15:49002 ().
2016/05/25 11:52:43 client connected: 192.168.10.13:4000 ().
2016/05/25 11:52:43 client connected: 192.168.10.13:49002 ().
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0xd1bd8]

goroutine 9 [running]:
main.ffMonitor()
        /root/stratux-dev/main/network.go:469 +0xf0
created by main.initNetwork
        /root/stratux-dev/main/network.go:522 +0x174

goroutine 1 [select (no cases)]:
main.main()
        /root/stratux-dev/main/gen_gdl90.go:1332 +0xdbc

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /root/go-1.5.1/src/runtime/asm_arm.s:1036 +0x4

goroutine 20 [syscall]:
os/signal.loop()
        /root/go-1.5.1/src/os/signal/signal_unix.go:22 +0x14
created by os/signal.init.1
        /root/go-1.5.1/src/os/signal/signal_unix.go:28 +0x30

goroutine 21 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x694c60)
        /root/go_path/src/github.com/golang/glog/glog.go:882 +0x60
created by github.com/golang/glog.init.1
        /root/go_path/src/github.com/golang/glog/glog.go:410 +0x2cc
...

Copy the whole shebang into a text file to upload it when you get on the ground. The most important bit are going to be the line starting "panic: runtime error" and the first few blocks of line following it. (For reference, this particular crash was triggered intentionally by trying to run a second instance of gen_gdl90 when one was already running... hence the service stratux stop in my instructions.) I'll also need to know the exact build number, e.g. 'Stratux v0.8r2 (65566816a40c74ad648d09a0dcfebd0b9a9e097b)`

idriss-ophawk commented 8 years ago

Thanks @AvSquirrel - will bring a laptop on my flight tonight and send you the output of stdout and stderr from /usr/bin/gen_gdl90.

Which version would you like me to test v0.8r2- 5be9ec80ef or the release v0.8r2- db130aab76 ?

Thanks

jpoirier commented 8 years ago

I'd recommend building stratux with the latest Go compiler (1.7 beta is out but the tip of master should work as well) and testing it. To build the go compiler and assuming the prebuilt binaries are at /root/go1.6.2 and the stratux folder is sitting in root then from within the same shell window and as root it'd look something like this:

reboot

To create a race detection build use the race flag in the build command, e.g. in the stratux/makefile add -race to the BUILDINFO variable, e.g. BUILDINFO=-race -ldflags ...

Then set the GORACE flag in /root/.bashrc, e.g. GORACE="log_path=/root/ strip_path_prefix=/stratux/main/" See the following link for detailed info https://golang.org/doc/articles/race_detector.html

skypuppy commented 8 years ago

For "race detection" are you referring to mutex issues? (multiple writers to a single location simultaneously?)

ghost commented 8 years ago

@idriss-ophawk -- Let's start with the v0.8r2 db130aa release. This should force a failure more quickly, and as @Ergonomicmike said,

They might ask you for your uat log file.

I'd like to see your UAT log file. Specifically, I'd like to see a high message rate log from multiple towers for my testing purposes, and the log format from the "older" (pre-sqlite) versions is easier for me to feed into UAT replay. So, turn on "Record Logs" and capture the resulting nnnn-uat.log file.

If you have a separate image with 5be9ec8 and can capture a crash on it, I'd like to see that stack trace as well.

ssokol commented 8 years ago

I now have two customers reporting this issue. One is running on a Pi 2, the other a Pi 3. I believe both are running v0.8r2-db130aab76. Both have recently added the Dan Murray antennas and both live in areas with many towers.

One of them happens to live within line of site of two towers in the NYC area, so receives a ton of traffic. Running only UAT, his system was consistently dying off after a few minutes. I had him switch the SDR over to 1090 and the crashes stopped. Seems to indicate:

  1. The issue is not new since db130aab76
  2. The issue is almost definitely in the UAT code

Neither of these is "news" but it's always good to get confirmation.

jpoirier commented 8 years ago

On May 25, 2016, at 9:50 AM, skypuppy notifications@github.com wrote: For "race detection" are you referring to mutex issues? (multiple writers to a single location simultaneously?) See the link to the Go race detector page I provided in my previous post...

idriss-ophawk commented 8 years ago

@AvSquirrel - for the UAT log under v0.8r2-db130aab76 I still have my logs from last Sunday:

0006-uat.log.txt

I will use that version tonight for the test. I am not sure I'll be able to get the GO race detector set up by then though.

Thanks

Ergonomicmike commented 8 years ago

Wow - just did a one hour flight using AvSquirrel's latest. Had to pull the power on the Stratux about every 7 minutes. (Remind you of anyone? And I was getting only 5 towers.) These problems seemed to have started sometime around sqlite. Anyway, I'm going to fly home with an older version. I might even down grade to the sh for 0.8r2.

skypuppy commented 8 years ago

I seem to remember sqlite having memory leakage problems but that was several years ago.

idriss-ophawk commented 8 years ago

@AvSquirrel - just finished my night flight around NYC bravo and got the traces:

both versions failed again after a 5-10 minutes

1- v0.8r2 - db130aab76 gen_gdl90 trace at 3,000 agl v0.8r2-db130aab76 gen_gdl90 logs at 3,000 AGL.txt


2016/02/26 01:21:44 27443B => {"Icao_addr":2573371,"Tail":"u","Emitter_category":0,"OnGround":false,"Addr_type":3,"TargetType":4,"SignalLevel":-24.436974992327126,"Position_valid":true,"Lat":40.470306,"Lng":-73.54434,"Alt":3900,"GnssDiffFromBaroAlt":0,"AltIsGNSS":false,"NIC":6,"NACp":8,"Track":30,"Speed":278,"Speed_valid":true,"Vvel":0,"Timestamp":"2016-02-26T01:21:43.33442314Z","Age":1.9,"Last_seen":"0001-01-01T00:01:14.15Z","Last_alt":"0001-01-01T00:01:14.15Z","Last_GnssDiff":"0001-01-01T00:00:00Z","Last_GnssDiffAlt":0,"Last_speed":"0001-01-01T00:01:14.15Z","Last_source":2,"ExtrapolatedPosition":false,"Bearing":0,"Distance":0}
panic: runtime error: index out of range

goroutine 4 [running]:
_/root/stratux/uatparse.(*UATFrame).decodeAirmet(0x1078eb00)
    /root/stratux/uatparse/uatparse.go:429 +0x33a0
_/root/stratux/uatparse.(*UATFrame).decodeInfoFrame(0x1078eb00)
    /root/stratux/uatparse/uatparse.go:480 +0xb4
_/root/stratux/uatparse.(*UATMsg).DecodeUplink(0x10bce540, 0x0, 0x0)
    /root/stratux/uatparse/uatparse.go:542 +0x45c
main.parseInput(0x10f57c00, 0x36e, 0x0, 0x0, 0x0, 0x7)
    /root/stratux/main/gen_gdl90.go:915 +0x640
main.uatReader()
    /root/stratux/main/sdr.go:302 +0xfc
created by main.sdrInit
    /root/stratux/main/sdr.go:523 +0x38

stratux logs: stratux.log.txt

UAT logs: 0002-uat.log.txt 0003-uat.log.txt 0004-uat.log.txt 0005-uat.log.txt 0006-uat.log.txt

2- v0.8r2- 5be9ec8 gen_gdl90 trace from ground to 3,000 msl v0.8r2- 5be9ec8 gen_gdl90 logs.txt

If you need any other log file, let me know

Thanks

Ergonomicmike commented 8 years ago

I also had repeatable failures with ab76 coming home just now. About every 10 minutes. (I had this problem last wek with an intermediate .sh build, but since the flight lasted only 24 minutes, it failed in flight, but I didn't have time to see a pattern.) I'm beginning to like the theory that the FAA is sending a "Stratux" bit.

jpoirier commented 8 years ago

On Wed, May 25, 2016 at 9:17 PM, idriss-ophawk notifications@github.com wrote:

@AvSquirrel https://github.com/AvSquirrel - just finished by night flight and got the traces:

1- v0.8r2 - db130aa https://github.com/cyoung/stratux/commit/db130aab76e9acde26b95c14014ced8fadbfeb80 gen_gdl90 trace at 3,000 agl v0.8r2-db130aab76 gen_gdl90 logs at 3,000 AGL.txt https://github.com/cyoung/stratux/files/283335/v0.8r2-db130aab76.gen_gdl90.logs.at.3.000.AGL.txt

"panic: runtime error: index out of range" kills gen_gdl90

So an invalid array/slice (zero length?) access in the decodeAirmet method

jpoirier commented 8 years ago

On Wed, May 25, 2016 at 9:17 PM, idriss-ophawk notifications@github.com wrote:

2- v0.8r2- 5be9ec8 https://github.com/cyoung/stratux/commit/5be9ec80efe62b9196ee8c50f28fac29062d383e gen_gdl90 trace from ground to 3,000 msl v0.8r2- 5be9ec8 gen_gdl90 logs.txt https://github.com/cyoung/stratux/files/283334/v0.8r2-.5be9ec8.gen_gdl90.logs.txt

And the same "panic: runtime error: index out of range" message in this log as well with the invalid access at uatparse/uatparse.go:429

edit: apologies for the redundant info, I didn't get @idriss-ophawk original message highlighting the panic via gmail

Ergonomicmike commented 8 years ago

The AIRMET failure explains the peroidicity. Had an AIRMET today for Mod Turb. Gets rebroadcast about every 10 minutes.

jpoirier commented 8 years ago

fyi - the logs also show that a fairly old version of the Go compiler (1.5.1) was used for the build, version 1.6.2 is current.

edit: remove quoted reply info

ghost commented 8 years ago

Right. Both panics had a common mode: the decodeAirmet() function in uatparse.go attempted to read the 11th index of an array that was only 10 indexes long.

panic: runtime error: index out of range

goroutine 4 [running]:
_/root/stratux/uatparse.(*UATFrame).decodeAirmet(0x1078eb00)
    /root/stratux/uatparse/uatparse.go:429 +0x33a0
_/root/stratux/uatparse.(*UATFrame).decodeInfoFrame(0x1078eb00)
    /root/stratux/uatparse/uatparse.go:480 +0xb4
_/root/stratux/uatparse.(*UATMsg).DecodeUplink(0x10bce540, 0x0, 0x0)
    /root/stratux/uatparse/uatparse.go:542 +0x45c
main.parseInput(0x10f57c00, 0x36e, 0x0, 0x0, 0x0, 0x7)
    /root/stratux/main/gen_gdl90.go:915 +0x640
main.uatReader()
    /root/stratux/main/sdr.go:302 +0xfc
created by main.sdrInit
    /root/stratux/main/sdr.go:523 +0x38

This is what actually generated the panic (line 429):

alt_top_raw := ((int32(record_data[9]) & 0x01) << 6) | ((int32(record_data[10]) & 0xFC) >> 2

The 10th byte (record_data[9]) was also read on L428 and didn't generate the panic there, so it follows that the reference to record_data[10] did.

This function, and a few others within uatparse.go, did not have any bounds checks on array / slice lengths. They were written with the assumption that data would be structured correctly. I've added checks for those functions, and committed them as https://github.com/AvSquirrel/stratux/commit/72aa6b55831bb59be14e41da53a8d99f5dc35215. The .sh update is below.

update-stratux-v0.8r2-72aa6b5583.sh.zip

There may be more that I've missed, but this should be a step in the right direction.

ghost commented 8 years ago

One other comment -- this decoding of graphical AIRMET decoding is essentially useless in the current incarnation of Stratux, since the raw UAT messages are passed as GDL90 messages, and are decoded by the EFB.

I'd consider seriously paring back the untested and/or unneeded portions of uatparse.go until there's a need to use them in the core Stratux code.

Ergonomicmike commented 8 years ago

FWIW, iFly depicts graphical AIRMETs and attributes them to ADS-B. I don't know if they are taking the GDL90 data and graphing it on their own, or using graphical AIRMET data directly. But I thought I should mention this.

Ergonomicmike commented 8 years ago

"They were written with the assumption that data would be structured correctly."

I've been sharing snippets from this thread w/ my airplane partner, since she's interested in knowing why I have to fuss with the Stratux lately and is intested in seeing how the solution converges.

She designed the spec for the FMC for Boeing's 757, 767. This probably goes without saying. But she said "You never assume that the input data will be correct." (That is, she always checked data for valid range.)

Corrected the aircraft she designed for.