Stratux -failure mode analysis-, root partition fails overtime

[TomTravis]

1. Stratux version: 1.4r5 current version

2. Stratux config:

   SDR

   • [] single
   • [x ] dual

   GPS

   • [x] yes
   • [ ] no type: vk172 vk162

   AHRS

   • [ x] yes
   • [ ] no

   power source: stepdown Power supply 5.5vdc at 3amp

3. EFB app and version: Avare current version

   EFB platform: android

   EFB hardware: Fire 7 Tablet

4. Description of your issue:

SD card failure after ~6 to 8 weeks runtime

It appears the default logging function from stratux 1.4r5 writes to the following /var/log files: -rw-r--r-- 1 root root 0 Jan 4 21:50 fancontrol.err -rw-r--r-- 1 root root 0 Jan 4 21:50 stratux-screen.log root@raspberrypi:/var/log# tail stratux.log 2019/01/04 22:03:18 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/04 22:03:19 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/04 22:03:20 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/04 22:03:20 stats [started: 1 day ago] 2019/01/04 22:03:20 - Disk bytes used = 2.9 GB (37.5 %), Disk bytes free = 4.8 GB (62.5 %) 2019/01/04 22:03:20 - CPUTemp=36.48 deg C, MemStats.Alloc=15 MB, MemStats.Sys=39 MB, totalNetworkMessagesSent=39,719 2019/01/04 22:03:20 - UAT/min 0/0 [maxSS=0.00%], ES/min 0/0, Total traffic targets tracked=0 2019/01/04 22:03:20 - Network data messages sent: 0 total, 0 nonqueueable. Network data bytes sent: 0 total, 0 nonqueueable. 2019/01/04 22:03:20 - Last GPS fix: now, GPS solution type: 2 using 9 satellites (9/16 seen/tracked), NACp: 10, est accuracy 4.00 m 2019/01/04 22:03:20 - GPS vertical velocity: 0.08 ft/sec; GPS vertical accuracy: 8 m

root@raspberrypi:/var/log# tail syslog Jan 4 22:05:05 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:05 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:06 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:06 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:07 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:07 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:08 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:08 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:09 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:09 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:10 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:10 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:11 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:11 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:12 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:12 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:13 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:13 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 4 22:05:14 raspberrypi gen_gdl90[692]: 2019/01/04 22:05:14 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006

while syslog uses standard service logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

Stratux.log fills the sdcard to a point of failure without management, this is the issue.

I now boot my Rpi3 from a root partition /dev/sda2 on a usb flash memory and use the sd card as boot on /dev/mmcblk0p1 root@raspberrypi:/var/log# uname -a Linux raspberrypi 4.14.90-v7+ #1183 SMP Fri Dec 21 14:03:50 GMT 2018 armv7l GNU/Linux current Rpi OS image via rpi-update root@raspberrypi:/etc# cat fstab proc /proc proc defaults 0 0 /dev/mmcblk0p1 /boot vfat defaults 0 2 /dev/sda2 / ext4 defaults,noatime 0 1 /dev/sda3 /home ext4 defaults,noatime 0 1

I have added the following to crontab for root 00,10,20,30,40,50 truncate -s 0 /var/log/ 00,10,20,30,40,50 truncate -s 0 /var/log/syslog 00,10,20,30,40,50 * truncate -s 0 /var/log/stratux.log

This allows the SDcard to remain static(boot only) and the usbflash(root) memory not to be filled by log files which appears to kill the SDcard to a readonly state.

I have started to run the duration test and will see how long stratux lives without failure.

as noted one of the test articles is just a GPS VK172 and no SDR or AHRS are used. GPS hardware:USB u-blox 7 GNSS receiver (NMEA-UBX protocol) GPS solution:GPS + SBAS (WAAS), 4.0 m GPS satellites:9 in solution; 11 seen; 15 tracked

also the systems with 2 SDR and GPS(remote VK162) with AHRS the GPS performance is degraded; GPS hardware:USB u-blox 7 GNSS receiver (NMEA-UBX protocol) GPS solution:3D GPS, 10.0 m GPS satellites:7 in solution; 8 seen; 12 tracked

both systems are in the same location with clear view of sky.

[TomTravis]

Stratux system logging (In settings) is off while failure takes place SDcard readonly. will not boot.

[TomTravis]

For any interested party. It appears my EFB software does not fail over time, Avare. https://www.hf.faa.gov/Workbenchtools/default.aspx?rPage=ToolList&subCatID=43 I do not use stratux in flight as it is not reliable for use in its current configuration.

I have moved forward with this; https://uavionix.com/products/echo-uat/ no issues to date. Part 43 install and Part 91 operation. Part 23 certified aircraft.

[TomTravis]

Interesting; such as it is this cannot be a good practice for keeping a SDcard healthy; the system without a AHRS device installed; writes to 3 files every second reading IMU calibrations that is turned off(in setting) and not installed. The same information set. root@raspberrypi:/var/log# tail stratux.log 2019/01/06 00:27:32 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:33 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:34 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:35 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:36 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:37 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:38 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:39 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:40 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/06 00:27:41 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 root@raspberrypi:/var/log# tail syslog Jan 6 00:28:02 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:02 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:03 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:03 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:04 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:04 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:05 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:05 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:06 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:06 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:07 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:07 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:08 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:08 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:08 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:08 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:09 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:09 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:10 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:10 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 root@raspberrypi:/var/log# tail daemon.log Jan 6 00:28:25 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:25 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:26 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:26 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:27 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:27 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:28 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:28 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:28 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:28 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:29 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:29 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:30 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:30 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:31 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:31 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:32 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:32 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 6 00:28:33 raspberrypi gen_gdl90[692]: 2019/01/06 00:28:33 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 root@raspberrypi:/var/log#

Crontab truncate appears to be working for the USB flash device, keeping the files in check. The sdcard is static for this evaluation

[TomTravis]

Looks like the test terminated all by itself;

GPS only configGPS. System uptime 217720 seconds. Daemon was restarted. NO boot device failure yet.

The GPS/AHRS/SDR config Lost GPS fix and GPS would not lock, reboot did not recover the GPS, Power cycle did recover the GPS. NO boot device failure yet.

This is a Human Factors error, I cannot be playing with the Stratux box while flying the aircraft PIC

It stay home on the bench such as it is.

[Helno]

I'm curious to see the same sort of test done on the latest beta.

http://updates.stratux.me/builds/ Try the newest file out of there and see if it has the same files being written to.

It should be easy to find out where in the software it is writing and stop it. Looks like it is mostly AHRS related.

The IMU in syslog appears to be coming from this line.

https://github.com/cyoung/stratux/blob/e5916be47226e385bcc892a2d78eb5208d882a26/main/sensors.go#L165

[TomTravis]

Since I now Truncate the log files I really do not know. The daemon restarts after time has elapsed and the test terminates by itself. The USB flash-drive seem to be unaffected.

Stratux.log is not managed by the standard logrotate service. Daemon.log and syslog.log are managed by logrotate.

Startux SD cards only last so long before the file system fails or becomes full and then become read only and fail to boot. Linux itself does not do this. This not a "new" issue. one that has been called reflash a new sd card.

Stratux is not a flight avionics equipment for myself, at least not now. I have to assume flightbox has the same failure. It makes boot partition readonly, not root to the best of my knowledge, I do not own one. https://www.openflightsolutions.com/

[Helno]

It would be nice if you could see if you get the same files being written to with the latest version.

Not much sense chasing an issue that might have been fixed.

There should be no files being written to in a typical installation. Identifying that it is writing to the log file when there is no GPS leads us to a bit of code that is expecting a GPS.

If you are running without a GPS or AHRS board perhaps going into the developer settings and turning off GPS, BARO, and AHRS would let us know if turning that off turns off the log files you are seeing. If the software config doesn't match the hardware then errors can be expected but we should be able to identify that situation to avoid writing.

Any help is finding why it is writing to anything is appreciated.

[TomTravis]

I running the release Latest release v1.4r5 ddbd3d5 I have one system running GPS only, in Developer mode, SDR and AHRS turned off. here is the log root@raspberrypi:/var/log# tail syslog Jan 17 02:10:54 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:54 - CPUTemp=38.63 deg C, MemStats.Alloc=2.9 MB, MemStats.Sys=13 MB, totalNetworkMessagesSent=8,410 Jan 17 02:10:54 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:54 - UAT/min 0/0 [maxSS=0.00%], ES/min 0/0, Total traffic targets tracked=0 Jan 17 02:10:54 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:54 - Network data messages sent: 0 total, 0 nonqueueable. Network data bytes sent: 0 total, 0 nonqueueable. Jan 17 02:10:54 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:54 - Last GPS fix: 1 second ago, GPS solution type: 1 using 9 satellites (13/16 seen/tracked), NACp: 10, est accuracy 8.00 m Jan 17 02:10:54 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:54 - GPS vertical velocity: 0.54 ft/sec; GPS vertical accuracy: 16 m Jan 17 02:10:55 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:55 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:10:56 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:56 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:10:57 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:57 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:10:58 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:58 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:10:59 raspberrypi gen_gdl90[614]: 2019/01/17 02:10:59 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006

root@raspberrypi:/var/log# tail stratux.log 2019/01/17 02:11:39 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:40 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:41 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:41 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:42 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:43 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:44 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:45 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:46 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 2019/01/17 02:11:47 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 root@raspberrypi:/var/log#

root@raspberrypi:/var/log# tail daemon.log Jan 17 02:12:08 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:08 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:09 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:09 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:10 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:10 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:11 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:11 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:12 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:12 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:13 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:13 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:14 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:14 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:15 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:15 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:16 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:16 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 17 02:12:17 raspberrypi gen_gdl90[614]: 2019/01/17 02:12:17 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 root@raspberrypi:/var/log#

[TomTravis]

here is the system running two sdr, one gps, ahrs. root@raspberrypi:/var/log# tail stratux.log 2019/01/17 02:20:19 stats [started: 14 hours ago] 2019/01/17 02:20:19 - Disk bytes used = 4.2 GB (54.1 %), Disk bytes free = 3.6 GB (45.9 %) 2019/01/17 02:20:19 - CPUTemp=46.16 deg C, MemStats.Alloc=8.0 MB, MemStats.Sys=22 MB, totalNetworkMessagesSent=1,628,511 2019/01/17 02:20:19 - UAT/min 0/0 [maxSS=0.00%], ES/min 0/0, Total traffic targets tracked=0 2019/01/17 02:20:19 - Network data messages sent: 1606673 total, 1606673 nonqueueable. Network data bytes sent: 40174035 total, 40174035 nonqueueable. 2019/01/17 02:20:19 - Last GPS fix: now, GPS solution type: 1 using 4 satellites (11/15 seen/tracked), NACp: 8, est accuracy 54.00 m 2019/01/17 02:20:19 - GPS vertical velocity: -0.91 ft/sec; GPS vertical accuracy: 108 m 2019/01/17 02:20:19 - Last IMU read: now, Last BMP read: now

root@raspberrypi:/var/log# tail daemon.log Jan 17 02:21:27 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:27 AHRS Gyro/Accel Error: MPU9250 Error: No new accel/gyro values Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 stats [started: 14 hours ago] Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - Disk bytes used = 4.2 GB (54.1 %), Disk bytes free = 3.6 GB (45.9 %) Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - CPUTemp=46.16 deg C, MemStats.Alloc=8.3 MB, MemStats.Sys=22 MB, totalNetworkMessagesSent=1,631,296 Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - UAT/min 0/0 [maxSS=0.00%], ES/min 0/0, Total traffic targets tracked=0 Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - Network data messages sent: 1609422 total, 1609422 nonqueueable. Network data bytes sent: 40242955 total, 40242955 nonqueueable. Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - Last GPS fix: now, GPS solution type: 1 using 3 satellites (7/14 seen/tracked), NACp: 7, est accuracy 100.00 m Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - GPS vertical velocity: 6.55 ft/sec; GPS vertical accuracy: 200 m Jan 17 02:21:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:49 - Last IMU read: now, Last BMP read: now Jan 17 02:21:54 raspberrypi gen_gdl90[27530]: 2019/01/17 02:21:54 AHRS Gyro/Accel Error: MPU9250 Error: No new accel/gyro values

root@raspberrypi:/var/log# tail syslog Jan 17 02:29:19 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:19 - GPS vertical velocity: 0.85 ft/sec; GPS vertical accuracy: 26 m Jan 17 02:29:19 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:19 - Last IMU read: now, Last BMP read: now Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 stats [started: 14 hours ago] Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - Disk bytes used = 4.2 GB (54.1 %), Disk bytes free = 3.6 GB (45.9 %) Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - CPUTemp=46.16 deg C, MemStats.Alloc=8.6 MB, MemStats.Sys=22 MB, totalNetworkMessagesSent=1,646,087 Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - UAT/min 0/0 [maxSS=0.00%], ES/min 0/0, Total traffic targets tracked=0 Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - Network data messages sent: 1624021 total, 1624021 nonqueueable. Network data bytes sent: 40608951 total, 40608951 nonqueueable. Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - Last GPS fix: now, GPS solution type: 1 using 4 satellites (9/14 seen/tracked), NACp: 9, est accuracy 11.80 m Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - GPS vertical velocity: 2.85 ft/sec; GPS vertical accuracy: 23.6 m Jan 17 02:29:49 raspberrypi gen_gdl90[27530]: 2019/01/17 02:29:49 - Last IMU read: now, Last BMP read: now root@raspberrypi:/var/log#

[Helno]

So it appears that the AHRS code is doing something strange when there is no AHRS installed or enabled.

@westphae Got any ideas why it is happening? Looks like this line of code is to blame.

https://github.com/cyoung/stratux/blob/ddbd3d5ef8f0f8b967c9f96450f3e432ea5b006c/main/sensors.go#L165

[TomTravis]

Yes Stratux Code, it kills the sdcard to: readonly. System will not boot. running SDcard

on both test system with the crontab truncate commands to root /var/log running boot static SDcard /dev/mmcblk0p1 /boot vfat defaults 0 2 and root on/ RWX USB flashdrive dev/sda2 / ext4 defaults,noatime 0 1 it is still alive two weeks in many restart daemons

[TomTravis]

GPS On the 2sdr GPS(VK162) and AHRS GY91 GPS was in and out of lock. restart application did not change anything. Power cycle reboot worked GPS now locked again. AHRS status is green. System ran 19hrs and needed restart, power cycle. when Stratux does a daemon resets the Uptime clock is zeroed and count-up starts again

[TomTravis]

Unknown time. ~5hrs ago to now. System with 2sdr GPS(VK162) and AHRS GY91 Network connection failed, Stratux appeared to be running, green light. Power cycle required to get network back up.

[TomTravis]

System with 2sdr GPS(VK162) and AHRS GY91 stratux task failed, would not restart, via stxstop, stxstop, system is unstable. ssh terminal commands would not finish and abort. power cycle and the system is back up.

It may be the beginning of the end for this setup. I can remove the USB flash drive and fsck the partitions on my linux device. next

GPS system standalone is running without issue

[cyoung]

Couple of problems I see:

1. gen_gdl90 duplicating logs in both the syslog general log and stratux.log.
2. stratux.log not being rotated.

[TomTravis]

crontab was trying to email root on the unstable system, error. Command would not run.

Will fix messaging problem if there is one.

[cyoung]

/var/log/syslog should no longer have gen_gdl90 messages and logrotate should keep rotate /var/log/stratux.log daily, keeping 10 days.

Try this update: https://stratux.s3-us-west-1.amazonaws.com/builds/update-stratux-v1.5b1-659e7860a2.sh

[TomTravis]

Thank you Daemon.log on the GPS only system still get updated 1/sec with AHRS info for no reason. AHRS is turned off. root@raspberrypi:/var/log# tail daemon.log Jan 18 02:45:08 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:08 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:09 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:09 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:10 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:10 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:11 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:11 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:12 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:12 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:13 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:13 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:14 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:14 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:15 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:15 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:16 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:16 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006 Jan 18 02:45:16 raspberrypi gen_gdl90[657]: 2019/01/18 02:45:16 AHRS Info: IMU Calibrations read from settings: accel 0.014745 -0.094416 -0.832047; gyro -0.812403 1.476791 -1.151006

I did not update with the patch yet. however the daemon.log is no longer receiving gdl90 messages with the update?

Crontab truncates -s 0 *.log & syslog every 10min on my test systems. Which appears to keep the system alive and running for the test period this far.

[TomTravis]

System with 2sdr GPS(VK162) and AHRS GY91 Thank you

ran the update .sh file. The logfiles are no longer filling up the storage device. The update was to the SDcard, it is now the boot /root device.

I think a message to the user should be a "please update" to avoid system boot failures.

I'll get the USB flash device updated via dd offline

[cyoung]

Yes, daemon.log should also not be receiving these messages.

Good find, thanks for posting this. I'll close the issue now unless you find further unresolved related issues.