Loss of Foreflight and UI connection in flight

[cbjohnsonjr]

1. Stratux version: 0.8r2
2. Stratux config: Raspberry Pi 3, 1 SDR Nano 2( 978 only), no GPS, no AHRS, Battery - Anker® 2nd Gen Astro E5 16000mAh, short micro usb cable
3. EFB app and version : Foreflight v7.6.1 and Flightplan same issue
4. Description of your issue : Loss of Forelight connection and UI connection occurs every flight with duration approximately 1.0 - 1.5 hours. Jet aircraft operating FL 350-450. I unplug power then repower and device operates normally for a few minutes then problem reoccurs. If I leave power off a for longer period then last operates for 1.0+ hour then problem starts again. I've tried different batteries, full sized receiver, placing receiver in cooler spot (typical reporting temp 50 degrees C), all no change. Wifi never goes away only EFB and UI connection. On the ground with no ADSB tower connection continues to operate without problems. Everything else works

[skypuppy]

Valgrind is your friend. IF it works with golang.

[idriss-ophawk]

Thanks @AvSquirrel will try to load your fix and fly tonight to re-test.

[ssokol]

Checked with a contact at the FAA. Apparently the contract for weather data is in the process of switching over from WSI to Harris Corp. right now. It is supposed to be seamless, but... It's quite possible that the WSI data was a bit cleaner.

[idriss-ophawk]

@AvSquirrel - it works now! didn't get the issue on my flight tonight. Thanks for the quick turnaround, if you happen to be around NYC one day, let me know and we'll celebrate with a fly in the hudson sfra and over manhattan!

[Ergonomicmike]

Yea!

[ssokol]

+1 I sent an update to my customer in New York. @AvSquirrel fixed it! He ran for 3.5 hours, received multiple AIRMETs and never crashed.

[Ergonomicmike]

It's strange that the reddit isn't flooded with people reporting this problem. Either there aren't that many AIRMETS out there or perhaps this is a software transition problem - although it extends from Phx to NYC?

[kdknigga]

Mike,

I've had the problem (I think). The problem is easily fixed with a power cycle and I really didn't have time to pull together a useful bug report (something more than "I had to reboot a couple of times in flight.").

On Fri, May 27, 2016, 10:39 Ergonomicmike notifications@github.com wrote:

It's strange that the reddit isn't flooded with people reporting this problem. Either there aren't that many AIRMETS out there or perhaps this is a software transition problem - although it extends from Phx to NYC?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-222180033, or mute the thread https://github.com/notifications/unsubscribe/ANN_wFArMZZUM1NX8PcwCgZAfpjYQ0GYks5qFxA1gaJpZM4Iiyu6 .

[JohnOCFII]

It's strange that the reddit isn't flooded with people reporting this problem. Either there aren't that many AIRMETS out there or perhaps this is a software transition problem - although it extends from Phx to NYC?

It is strange. I'm still on the last image. I flew about 2 hours two weeks ago with no issues. Of course, I don't think I've ever seen more than 6 UAT towers, and perhaps the message rate here in the MSP area never gets high enough to trigger the problem. I also wonder if there was some software change upstream in the UAT ADS-B system software that might be rolling out station by station (or region by region) such that some have seen the problem, and others have not.

[cyoung]

Thanks, everyone.

decodeAirmet() wasn't supposed to be in there. It's a miracle that it worked at all.

[GrayFlyer]

How far back does this bug go? Just wondering because I've had this problem since 0.4r1.

[cyoung]

v0.5b2 is when it was introduced

[cyoung]

Give v0.9b1 a try

[idriss-ophawk]

sure - will do! thx

[pp4life]

I wonder if this is/was the problem I posted on Reddit about two months ago. I'll try v0.9b1 next time I fly and see if things are more stable.

You are some super sleuths! I wish I could have dug into this more back then. I just don't get the seat time to be able to troubleshoot on a regular basis.

[cbjohnsonjr]

I flew from Boston to Memphis @430 2.6 hours and Memphis to Dallas @280 1.2 hours with v0.9b1. Much of the time more than 6 stations and high UAT traffic. No problems. I have several long flights the next two days, So far so good. Great work guys.

[cyoung]

@cbjohnsonjr - were you using Stratux in the cockpit? Which antennas?

[cbjohnsonjr]

Yes in the cockpit Citation 650. Only 978 receiver and antenna.

[idriss-ophawk]

@cyoung - flew with v0.9b1 yesterday and today in NYC & Boston area for around 6 hours between 1,500agl to 7,500 agl in a cessna 150. 4-9 UAT towers with skyguardtwx antenna (Remote Mount 978mhz Antenna with 4 ft. coax cable, http://adsb.skyguardtwx.com/store/), which is a very nice antenna btw. It worked very well. Had a few issues sometimes joining the wifi ssid (both ipad and iphone) and had to recycle but it was happening from time to time before. Thanks again

[Ergonomicmike]

@idriss-ophawk On WiFi dropping out, did you notice if the blue light in the Edimax was flashing when the connection dropped? And what kind of power are you running? We were chasing the "Blue Light of Death" problem for a while, but haven't seen it for months after some code changes changed the way that data went over the network. But insufficient power can still cause drop outs.

[GrayFlyer]

@idriss-ophawk I'm having the same problem since I went to 0.9b1. I have to reboot at least once to get either of my devices (iFly 720 and a Android tablet) to connect with WiFi. I don't recall ever having to reboot to connect before this version. The devices see the WiFi but just won't connect until I reboot.

[GrayFlyer]

Flying 0.9b1 with dual SDR's, and on two flights now I have had poor 978 reception. I traced the problem to the SDR's which have tagged eeproms not being on their assigned freq. The SDR tagged for 978 has it's tuned antenna on the belly of my plane, and the SDR for 1090 has a tuned antenna on the panel. With 978 connected to the panel mounted antenna the reception is nowhere near as good as with the belly mounted antenna which is tuned for 978. I verified by disconnecting the antenna's in turn. I have not seen this problem before this version.

Also my GPS (a RY835AI) constantly shows disconnects and reconnects on the WebUI, while having excellent stats reported? This has not been the norm before this version either.

Now the good news: No loss of heartbeat in about two hours of flying. I have never had a run like this before the version!

[kdknigga]

I logged 3.3 on the Hobbs today and had to pull power twice. Same symptoms: wifi still works, but the EFB no longer gets updates and the web interface is dead.

I was solo, so I couldn't ssh in and take a look, but I had verbose logging turned on.

https://www.dropbox.com/s/ug498uc22nju9us/stratux_logs.kdknigga.20160530.tar.gz?dl=0

[cbjohnsonjr]

Over 4 hours Dallas to Gunnison Co and back today @ 380 and 370 Citation 501 running in cockpit no problems with 978 only.

[kdknigga]

I've poked through my logs from yesterday and I don't see any smoking gun (on a side note, what's MessageClass in messages? I'm only seeing 0s.). I've had stratux running for 27 hours in my living room now with no issues. I'd venture a guess that UAT handling still isn't rock solid.

(This is all on v0.9b1 (89107818b821c603fe5353e2f0f3ca5c28ac6118))

[dbarrington1]

Just reading on this issue and wanted to add user level input. I just got home from a trip from Stockton CA to as far east as Minneapolis MN. All the way there I had No problems with Stratux.. Using just the released 8.2 image with the only changes being I added my Smode Hex code and set GPS on AHRS off. I made a power supply using a 3A 5v regulator that is NON switching (for a while I had issues with switching supplies and thought that was were my lockups were) My power is hard wired to the PI2. I use 2 nano's and the first RY AHRS chip wired in directly for AHRS (though shut off because of FF) anyway... Dmurry High gain antenna's All the way there over 3 days (we stopped to ride horses) in my Mooney M20C at 13-14K feet MSL it worked GREAT... then on the way home I started having this issue where WiFi was connected but I could not get in with the WEB UI and FF would loose connection. Power cycle fixed it... I have a fan and was never hotter than 121F on the CPU. I assumed one of the power shut offs during plane shutdown was the problem but reading this thread it may be something else. Anyway I will try the Beta Version 9 release. Oh also when it started failing I would only get 5 to 15 min of use before it would fail and I would fail with only one tower.... I was getting ARMET's for Moderate turbulence though......

[kdknigga]

gen_gdl90 (v0.9b1 (8910781)) crashed once on me on a 3.1 hour flight Friday night. The error message was "panic: runtime error: index out of range". The full (insanely long) traceback is here.

The sqlite log is very, very large. I can work on trimming that down and making it available if it'll help.

[cyoung]

@kdknigga - that's a new one, thanks for sending it.

I'm curious why we're getting (what appear to be) invalidly formatted UAT packets now?

If you want to select from the sqlite db, try SELECT * FROM messages WHERE MessageClass = 0. In 8910781 it won't actually give us replayable UAT packets, so now it's not useful - if you upgrade to the latest it actually logs the UAT packet.

[kdknigga]

If you want to select from the sqlite db...

Yeah, I've got about 60 hours of verbose logs in the database, or 11 GB. I need to cull a lot of data and to remember to rotate the database. Perhaps I'll put in a script to rotate the database on boot for future purposes.

if you upgrade to the latest it actually logs the UAT packet

I'll update to the latest version, but I probably won't be able to fly for a while. I'll post back when I've got something interesting.

On Tue, Jun 21, 2016 at 11:42 AM, cyoung notifications@github.com wrote:

@kdknigga https://github.com/kdknigga - that's a new one, thanks for sending it.

I'm curious why we're getting (what appear to be) invalidly formatted UAT packets now?

If you want to select from the sqlite db, try SELECT * FROM messages WHERE MessageClass = 0. In 8910781 https://github.com/cyoung/stratux/commit/89107818b821c603fe5353e2f0f3ca5c28ac6118 it won't actually give us replayable UAT packets, so now it's not useful - if you upgrade to the latest it actually logs the UAT packet.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-227498940, or mute the thread https://github.com/notifications/unsubscribe/ANN_wNFLLMesFFCzvQyFfg_J9TQsMR0fks5qOBRzgaJpZM4Iiyu6 .

[cyoung]

@kdknigga - the restart setup was integrated into v0.9b2. By the way, did you have something special set up to capture the stack trace?

[kdknigga]

Nope, I think systemd just does it and dumps it to syslog.

On Thu, Jun 23, 2016, 18:42 cyoung notifications@github.com wrote:

@kdknigga https://github.com/kdknigga - the restart setup was integrated into v0.9b2. By the way, did you have something special set up to capture the stack trace?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-228217791, or mute the thread https://github.com/notifications/unsubscribe/ANN_wDBjdLf--SMTaXDU3y2oxoeT7IO3ks5qOxnIgaJpZM4Iiyu6 .

[cyoung]

Perfect.

[joeysweet]

I have the same issue.... I originally thought it was a hardware issue due to a loose power connection turning the pi off during flight - but after putting some constant down pressure on the micro usb cable it stays connected and on throughout the flight....

I notice sometimes the green wifi light blinks faster at times than others, but the red power light stays on consistently....

The device connects to foreflight if I unplug and plug the power back in - but then moments later the foreflight device pages shows no devices connected.... When i check the ipad wifi, however, the stratux never loses a connection.... Any help for this bug?

[kdknigga]

Which version of stratux are you running?

On Tue, Jul 5, 2016, 19:22 joeysweet notifications@github.com wrote:

I have the same issue.... I originally thought it was a hardware issue due to a loose power connection turning the pi off during flight - but after putting some constant down pressure on the micro usb cable it stays connected and on throughout the flight....

I notice sometimes the green wifi light blinks faster at times than others, but the red power light stays on consistently....

The device connects to foreflight if I unplug and plug the power back in - but then moments later the foreflight device pages shows no devices connected.... When i check the ipad wifi, however, the stratux never loses a connection.... Any help for this bug?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230640388, or mute the thread https://github.com/notifications/unsubscribe/ANN_wMlKuBStd2VjQnECLUliy1ytwLZRks5qSvVBgaJpZM4Iiyu6 .

[joeysweet]

Kris,

First off, you don't fly at Illinois Aviation, do you? Name looks familiar

• I instruct there.

I have foreflight version 7.6.1 ios 9.2 Stratux v0.8r2

On Tue, Jul 5, 2016 at 7:34 PM, Kris Knigga notifications@github.com wrote:

Which version of stratux are you running?

On Tue, Jul 5, 2016, 19:22 joeysweet notifications@github.com wrote:

I have the same issue.... I originally thought it was a hardware issue due to a loose power connection turning the pi off during flight - but after putting some constant down pressure on the micro usb cable it stays connected and on throughout the flight....

I notice sometimes the green wifi light blinks faster at times than others, but the red power light stays on consistently....

The device connects to foreflight if I unplug and plug the power back in

but then moments later the foreflight device pages shows no devices connected.... When i check the ipad wifi, however, the stratux never loses a connection.... Any help for this bug?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230640388, or mute the thread < https://github.com/notifications/unsubscribe/ANN_wMlKuBStd2VjQnECLUliy1ytwLZRks5qSvVBgaJpZM4Iiyu6

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230642058, or mute the thread https://github.com/notifications/unsubscribe/ATXmvOYTOESJ-9bKkjyKxFD6O81KkfSIks5qSvgggaJpZM4Iiyu6 .

[chazchandler]

I the same thing flying near DC using v0.8r2 on a pi zero and foreflight.

Ps, Though it wouldn't solve this issue, I wonder if it would make sense to decouple the web ui rendering from what provides it's data. That way when/if some parts of the system fail other parts could go on operating and independent healthchecks from the ui could disambiguate failures.

[kdknigga]

Yes, I do fly at Illinois Aviation.

Stability enhancements have been made since 0.8r2. I suggest giving 0.9b3 a go.

https://github.com/cyoung/stratux/releases/tag/v0.9b3

On Tue, Jul 5, 2016, 19:46 joeysweet notifications@github.com wrote:

Kris,

First off, you don't fly at Illinois Aviation, do you? Name looks familiar

• I instruct there.

I have foreflight version 7.6.1 ios 9.2 Stratux v0.8r2

On Tue, Jul 5, 2016 at 7:34 PM, Kris Knigga notifications@github.com wrote:

Which version of stratux are you running?

On Tue, Jul 5, 2016, 19:22 joeysweet notifications@github.com wrote:

I have the same issue.... I originally thought it was a hardware issue due to a loose power connection turning the pi off during flight - but after putting some constant down pressure on the micro usb cable it stays connected and on throughout the flight....

I notice sometimes the green wifi light blinks faster at times than others, but the red power light stays on consistently....

The device connects to foreflight if I unplug and plug the power back

in

but then moments later the foreflight device pages shows no devices connected.... When i check the ipad wifi, however, the stratux never loses a connection.... Any help for this bug?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230640388, or mute the thread <

https://github.com/notifications/unsubscribe/ANN_wMlKuBStd2VjQnECLUliy1ytwLZRks5qSvVBgaJpZM4Iiyu6

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230642058, or mute the thread < https://github.com/notifications/unsubscribe/ATXmvOYTOESJ-9bKkjyKxFD6O81KkfSIks5qSvgggaJpZM4Iiyu6

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230643599, or mute the thread https://github.com/notifications/unsubscribe/ANN_wCspnxVOHWzSK7nOgA5w4nGK2erkks5qSvrggaJpZM4Iiyu6 .

[joeysweet]

Small world! I will reimage the card and give the newer version a go. Thanks!

On Tue, Jul 5, 2016 at 7:55 PM, Kris Knigga notifications@github.com wrote:

Yes, I do fly at Illinois Aviation.

Stability enhancements have been made since 0.8r2. I suggest giving 0.9b3 a go.

https://github.com/cyoung/stratux/releases/tag/v0.9b3

On Tue, Jul 5, 2016, 19:46 joeysweet notifications@github.com wrote:

Kris,

First off, you don't fly at Illinois Aviation, do you? Name looks familiar

• I instruct there.

I have foreflight version 7.6.1 ios 9.2 Stratux v0.8r2

On Tue, Jul 5, 2016 at 7:34 PM, Kris Knigga notifications@github.com wrote:

Which version of stratux are you running?

On Tue, Jul 5, 2016, 19:22 joeysweet notifications@github.com wrote:

I have the same issue.... I originally thought it was a hardware issue due to a loose power connection turning the pi off during flight - but after putting some constant down pressure on the micro usb cable it stays connected and on throughout the flight....

I notice sometimes the green wifi light blinks faster at times than others, but the red power light stays on consistently....

The device connects to foreflight if I unplug and plug the power back

in

but then moments later the foreflight device pages shows no devices connected.... When i check the ipad wifi, however, the stratux never loses a connection.... Any help for this bug?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/cyoung/stratux/issues/420#issuecomment-230640388 , or mute the thread <

https://github.com/notifications/unsubscribe/ANN_wMlKuBStd2VjQnECLUliy1ytwLZRks5qSvVBgaJpZM4Iiyu6

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230642058, or mute the thread <

https://github.com/notifications/unsubscribe/ATXmvOYTOESJ-9bKkjyKxFD6O81KkfSIks5qSvgggaJpZM4Iiyu6

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230643599, or mute the thread < https://github.com/notifications/unsubscribe/ANN_wCspnxVOHWzSK7nOgA5w4nGK2erkks5qSvrggaJpZM4Iiyu6

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230644721, or mute the thread https://github.com/notifications/unsubscribe/ATXmvJdpdMd83I7qyi3pkojwD8tfKLXjks5qSv0QgaJpZM4Iiyu6 .

[kdknigga]

@joeysweet, do you think anyone would be interested in an IAA stratux build night? If so, email me: kris at knigga dot com

On Tue, Jul 5, 2016, 20:05 joeysweet notifications@github.com wrote:

Small world! I will reimage the card and give the newer version a go. Thanks!

On Tue, Jul 5, 2016 at 7:55 PM, Kris Knigga notifications@github.com wrote:

Yes, I do fly at Illinois Aviation.

Stability enhancements have been made since 0.8r2. I suggest giving 0.9b3 a go.

https://github.com/cyoung/stratux/releases/tag/v0.9b3

On Tue, Jul 5, 2016, 19:46 joeysweet notifications@github.com wrote:

Kris,

First off, you don't fly at Illinois Aviation, do you? Name looks familiar

• I instruct there.

I have foreflight version 7.6.1 ios 9.2 Stratux v0.8r2

On Tue, Jul 5, 2016 at 7:34 PM, Kris Knigga notifications@github.com wrote:

Which version of stratux are you running?

On Tue, Jul 5, 2016, 19:22 joeysweet notifications@github.com wrote:

I have the same issue.... I originally thought it was a hardware issue due to a loose power connection turning the pi off during flight - but after putting some constant down pressure on the micro usb cable it stays connected and on throughout the flight....

I notice sometimes the green wifi light blinks faster at times than others, but the red power light stays on consistently....

The device connects to foreflight if I unplug and plug the power back

in

but then moments later the foreflight device pages shows no devices connected.... When i check the ipad wifi, however, the stratux never loses a connection.... Any help for this bug?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/cyoung/stratux/issues/420#issuecomment-230640388 , or mute the thread <

https://github.com/notifications/unsubscribe/ANN_wMlKuBStd2VjQnECLUliy1ytwLZRks5qSvVBgaJpZM4Iiyu6

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/cyoung/stratux/issues/420#issuecomment-230642058 , or mute the thread <

https://github.com/notifications/unsubscribe/ATXmvOYTOESJ-9bKkjyKxFD6O81KkfSIks5qSvgggaJpZM4Iiyu6

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230643599, or mute the thread <

https://github.com/notifications/unsubscribe/ANN_wCspnxVOHWzSK7nOgA5w4nGK2erkks5qSvrggaJpZM4Iiyu6

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230644721, or mute the thread < https://github.com/notifications/unsubscribe/ATXmvJdpdMd83I7qyi3pkojwD8tfKLXjks5qSv0QgaJpZM4Iiyu6

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cyoung/stratux/issues/420#issuecomment-230645999, or mute the thread https://github.com/notifications/unsubscribe/ANN_wO44kqrhzHLu34L5bgIlMfqb_-h5ks5qSv9PgaJpZM4Iiyu6 .

[cyoung]

568. @cbjohnsonjr - give v1.2r2 a try. Thanks.