Closed RobertGary1 closed 8 years ago
Do you have "replay" logging turned on? If so, once Stratux AHRS appears to have stopped working, and without rebooting Stratux, please log in via SSH and see how much space is left on the SD card.
Check heat buildup, maybe.
On 12/01/2015 01:55 PM, RobertGary1 wrote:
This has happened twice in a row now. I've left my set up on my bench running overnight with the RY835AI. Throughout the evening I'd run the Stratux GPS/AHRS page and found reasonable values. By morning all values aside from altitude are zero and moving the ahrs around shows no change. Reloading the browser etc makes no difference. I'm running v5 now but I also saw this with v4r4. Any suggested debugging?
-Robert
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131.
At this point I just have it all sitting exposed on my work bench without any case so it should cool ok. My eventual plan is to just mount everything to a piece of abs plastic and not enclose it.
I'll check the storage space tonight.
-Robert
On Dec 1, 2015, at 12:36 PM, skypuppy notifications@github.com wrote:
Check heat buildup, maybe.
On 12/01/2015 01:55 PM, RobertGary1 wrote:
This has happened twice in a row now. I've left my set up on my bench running overnight with the RY835AI. Throughout the evening I'd run the Stratux GPS/AHRS page and found reasonable values. By morning all values aside from altitude are zero and moving the ahrs around shows no change. Reloading the browser etc makes no difference. I'm running v5 now but I also saw this with v4r4. Any suggested debugging?
-Robert
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131.
— Reply to this email directly or view it on GitHub.
I already rebooted the stratux so I don't have it to test. At least now there is 40% free space on the drive. But I'm not sure if the logs clear on reboot. I'll try to reproduce it again tonight.
http://192.168.10.1/logs/ , will show you how large the log files are (see the stratux* files)
Ok. I don't think I have any logs turned on.
-Robert
On Dec 1, 2015, at 9:42 PM, cyoung notifications@github.com wrote:
http://192.168.10.1/logs/ , will show you how large the log files are (see the stratux* files)
— Reply to this email directly or view it on GitHub.
On 12/01/2015 02:55 PM, RobertGary1 wrote:
This has happened twice in a row now. I've left my set up on my bench running overnight with the RY835AI. Throughout the evening I'd run the Stratux GPS/AHRS page and found reasonable values. By morning all values aside from altitude are zero and moving the ahrs around shows no change. Reloading the browser etc makes no difference. I'm running v5 now but I also saw this with v4r4. Any suggested debugging?
-Robert
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131.
I put this little script in mine to reduce the log file sizes at startup. It is called from the init.d stratux script under the case for start. If you cycle stratux in cron nightly it will let you keep the end of the previous log as well as append the new information
MAX_SIZE=10000
LINES_TO_SAVE=100
LOGFILE_DIR=/var/log
for fname in cd $LOGFILE_DIR;ls stratux*
do
echo $fname
size=ls -l $LOGFILE_DIR/$fname | cut -d' ' -f5
if [ $size -gt $MAX_SIZE ]
then
first_line=head -1 $LOGFILE_DIR/$fname
last_100_lines=tail -n$LINES_TO_SAVE $LOGFILE_DIR/$fname
echo "Abbreviated:" > $LOGFILE_DIR/$fname
echo date
>> $LOGFILE_DIR/$fname
echo $first_line >> $LOGFILE_DIR/$fname
echo $last_lines >> $LOGFILE_DIR/$fname
fi
done
Happened again last night. Disk space still 40% free so no decrease in disk space.
Does the AHRS chip need to be reinitialized periodically? Looking at address 0x65 no longer has values...
pi@raspberrypi ~/Pi2D2 $ sudo i2cdump -y 1 0X68 No size specified (using byte-data access) 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 00: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 10: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 20: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 30: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 40: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 50: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 60: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 70: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 80: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 90: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX a0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX b0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX c0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX d0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX e0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX f0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX
Has anyone else tried running their Stratux overnight? -Robert
The chip is still detected...
pi@raspberrypi ~/Pi2D2 $ i2cdetect -l i2c-1 unknown 3f804000.i2c N/A pi@raspberrypi ~/Pi2D2 $
But address 0x68 is not reachable..
pi@raspberrypi ~/Pi2D2 $ !143 sudo i2cget -y 1 0X68 Error: Read failed
root@raspberrypi:~/stratux# i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- 77
root@raspberrypi:~/stratux# i2cget -y 1 0X68
0xff
root@raspberrypi:~/stratux#
Both of these are not matching? If you reboot, yes?
I can reboot now unless there is any further debugging you'd suggest
-Robert
On Dec 2, 2015, at 9:04 AM, cyoung notifications@github.com wrote:
root@raspberrypi:~/stratux# i2cdetect -y 1 0 1 2 3 4 5 6 7 8 9 a b c d e f 00: -- -- -- -- -- -- -- -- -- -- -- -- -- 10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- -- 70: -- -- -- -- -- -- -- 77
root@raspberrypi:~/stratux# i2cget -y 1 0X68 0xff root@raspberrypi:~/stratux# Both of these are not matching? If you reboot, yes?— Reply to this email directly or view it on GitHub.
Can you paste dmesg output?
I have run it 72 hours, without fan or heatsink on the cpu or graphics engine. No problems seen. This is with two sdr's. However I'm not using the "standard" IMU chip; it's an MPU6050 chip (no onboard GPS.) GPS chip is separate and comms to and from it for stratux are still in the development phase.
On 12/02/2015 10:57 AM, RobertGary1 wrote:
Happened again last night. Disk space still 40% free so no decrease in disk space.
Does the AHRS chip need to be reinitialized periodically? Looking at address 0x65 no longer has values...
pi@raspberrypi ~/Pi2D2 $ sudo i2cdump -y 1 0X68 No size specified (using byte-data access) 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 00: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 10: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 20: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 30: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 40: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 50: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 60: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 70: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 80: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 90: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX a0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX b0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX c0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX d0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX e0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX f0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX
Has anyone else tried running their Stratux overnight? -Robert
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-161363792.
When you say "it" you are referring to the ahrs chip I assume?
-Robert
On Dec 2, 2015, at 11:21 AM, skypuppy notifications@github.com wrote:
I have run it 72 hours, without fan or heatsink on the cpu or graphics engine. No problems seen. This is with two sdr's. However I'm not using the "standard" IMU chip; it's an MPU6050 chip (no onboard GPS.) GPS chip is separate and comms to and from it for stratux are still in the development phase.
On 12/02/2015 10:57 AM, RobertGary1 wrote:
Happened again last night. Disk space still 40% free so no decrease in disk space.
Does the AHRS chip need to be reinitialized periodically? Looking at address 0x65 no longer has values...
pi@raspberrypi ~/Pi2D2 $ sudo i2cdump -y 1 0X68 No size specified (using byte-data access) 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 00: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 10: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 20: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 30: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 40: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 50: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 60: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 70: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 80: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 90: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX a0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX b0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX c0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX d0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX e0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX f0: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX
Has anyone else tried running their Stratux overnight? -Robert
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-161363792.
— Reply to this email directly or view it on GitHub.
Yes.
On 12/02/2015 01:32 PM, RobertGary1 wrote:
When you say "it" you are referring to the ahrs chip I assume?
-Robert
On Dec 2, 2015, at 11:21 AM, skypuppy notifications@github.com wrote:
I have run it 72 hours, without fan or heatsink on the cpu or graphics engine. No problems seen. This is with two sdr's. However I'm not using the "standard" IMU chip; it's an MPU6050 chip (no onboard GPS.) GPS chip is separate and comms to and from it for stratux are still in the development phase.
On 12/02/2015 10:57 AM, RobertGary1 wrote:
Happened again last night. Disk space still 40% free so no decrease in disk space.
<<
>>
@RobertGary1 - do you see anything in dmesg after the failure that points to an issue?
Sorry about that. I flew late last night and had to work early. I'll take a look tonight.
-Robert
On Dec 3, 2015, at 8:53 AM, cyoung notifications@github.com wrote:
@RobertGary1 - do you see anything in dmesg after the failure that points to an issue?
— Reply to this email directly or view it on GitHub.
Mine is on it's 5th day of continuous running. I hope to make it to 7 days. 64 gb sd so IF the logs are turned on, I hope it's big enough.
On 12/03/2015 10:53 AM, cyoung wrote:
@RobertGary1 https://github.com/RobertGary1 - do you see anything in dmesg after the failure that points to an issue?
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-161713187.
My Stratux continues to work great. Just not getting values for pitch and roll.
-Robert
On Dec 3, 2015, at 10:45 AM, skypuppy notifications@github.com wrote:
Mine is on it's 5th day of continuous running. I hope to make it to 7 days. 64 gb sd so IF the logs are turned on, I hope it's big enough.
On 12/03/2015 10:53 AM, cyoung wrote:
@RobertGary1 https://github.com/RobertGary1 - do you see anything in dmesg after the failure that points to an issue?
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-161713187.
— Reply to this email directly or view it on GitHub.
Stratux uptime contest ?
Sounds like fun! :)
On Dec 3, 2015, at 1:06 PM, cyoung notifications@github.com wrote:
Stratux uptime contest ?
— Reply to this email directly or view it on GitHub.
At work call them CHO (continuous hours of operations ) tests :)
Sent from my iPhone
On Dec 3, 2015, at 1:14 PM, KingoL notifications@github.com wrote:
Sounds like fun! :)
On Dec 3, 2015, at 1:06 PM, cyoung notifications@github.com wrote:
Stratux uptime contest ?
— Reply to this email directly or view it on GitHub.
— Reply to this email directly or view it on GitHub.
No. Torture test. (minus the heat parameter.)
On 12/03/2015 03:06 PM, cyoung wrote:
Stratux uptime contest ?
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-161784397.
I don't see anything
7.957678] i2c /dev entries driver [ 11.769663] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup [ 13.310513] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1 [ 14.871750] Adding 102396k swap on /var/swap. Priority:-1 extents:1 across:102396k SSFS
The swap entry was the last on dmesg output after AHRS stopped working?
I'm not sure if the time correlates between the chip not responding and that error. Is it possible we sent something to the chip that would cause it to stop responding?
It doesn't correlate, the swap message is from 14 seconds after boot. Did you retrieve the dmesg AFTER the chip stopped responding, and confirm this swap message is the last one showing in dmesg after reproducing the error.
I ran into a problem when working with the BMP180 on another design. I was reading the IC at 10 times a second and occurring about 1-2 times an hour, the BMP180 would take down the I2C bus by pulling SDA low and keeping it low. In order to get it to release the bus I had to pulse the SCL line.
That would do it - going to look into that
I ordered a sensor board "0DOF MPU6050 HMC5883L BMP180 Gyroscope Acceleration Compass Module" on eBay for $9 shipped in the US, so I should have it in about a week. When I get it I'll take a look as well
@jamez70 - I'll be very interested in what success you have with the low cost board.
@bradanlane you got it. I may hook my 10hz gps recorder board that I designed to the Raspberry Pi, since it records all of those parameters at 10hz, I will just dump them out to the serial port.
Jamez, at our speeds, 10 samples per second yields: 60 min per hour and 60 seconds per minute = 3600 samples per second, and at a sample rate of 10 per second, you're working on 36,000 samples per hour.
Translating that into distance: at 60 mph, 5280 ft/hr = 5280 ft / 60 min/hr = 88 feet per minute or fpm, or /60 again = 1.47 fps, so divide by 10 samples per second, you are measuring 0.147 feet per sample, while increasing computational overhead (for formatting, transmitting, function calls, and etc.) by quite a large percentage.
Even at 120 mph, that is still 2.94 feet per sample at 10 Hz, and at those lucky enough to cruise at 180 mph, that is still only 4.41 feet per sample --- WAY beyond the resolution of GPS in it's current configuration. At private plane speeds, you're simply wasting processor cycles on data that is way, way below the resolution of GPS.
Even at 1 Hz, at 180 mph, you're still only 4.41 feet per sample. During cruise.
Even during T/O and landing, you're still somewhere around 60 mph during the highly critical phases of flight, you're far exceeding the GPS resolution even at the 1 Hz sample rate.
Please, show me where I have fouled up the math, or missed something important in the thoughts above.
Can we even do 10 Hz on an RPi 2? At 1 Hz, cyoung says we're pushing as much as 80% of the processor so if you increase the overhead for the GPS part of the calcs, we could easily be well beyond 100% capacity of the processor/system.
I like precision as much (or more) than the next guy, but you must factor in the relatively limited resolution of the GPS system, which even with WAAS can still be as much as 30 feet error boundaries. Without WAAS, 90 feet is seen often. And that is just horizontal. Vertical error is MUCH worse.
Skypuppy.
On 12/07/2015 07:43 PM, jamez70 wrote:
@bradanlane https://github.com/bradanlane you got it. I may hook my 10hz gps recorder board that I designed to the Raspberry Pi, since it records all of those parameters at 10hz, I will just dump them out to the serial port.
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-162731552.
I did GPS at 10hz, barometer at 10hz, accelerometer, compass and gyro at 10hz, all recording to an SD card, on a 72MHz STM32F103 processor.
Its actually fairly accurate if you substitute altitude using barometric pressure. Here is a screenshot of the data on Google Earth
Skypuppy, I'm no Albert Epstein but I think you've got some typo's above. For instance, 180mph = 15,840fpm = 264fps. So at 10Hz it would be 26.4fps instead of 4.41. Regardless, I agree that 1Hz is probably plenty for Stratux's purposes unless the Pi can handle more.
60 mph, 5280 ft/hr is 1mph, its 316800 feet an hour. We want 5280 feet a minute = 88 feet a second = 8.8 feet every 100 milliseconds
Oh, man! With such obvious mistakes in reasoning, it's a wonder I'm
still alive!!!
Yes, I forgot to expand the 5280/hour times 60 miles in that hour, and
the rest tumbled from there.
I had even double-checked it with unit analyses and STILL overlooked that one!
Sorry, guys, about the math errors.
Skypuppy (slinking away into the darkness)
On 12/07/2015 09:15 PM, jamez70 wrote:
60 mph, 5280 ft/hr is 1mph, its 316800 feet an hour. We want 5280 feet a minute = 88 feet a second = 8.8 feet every 100 milliseconds
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-162746803.
@skypuppy don't sweat it! You are right though, 10hz is pretty excessive. I built the recorder in order to get decent results when overlaying on a video and the results were impressive, but the sheer magnitude of data is overwhelming.
I was more on the train of thought for AHRS needing a fast update rate over GPS position information though.
If 10hz is possible, there is advantage to it. Over the past 10 years, more and more systems have switched to the faster data rate.
The challenge for Stratux is whether or not the advantages are part of its core purpose or are "new opportunities".
For navigation, weather, and traffic, 1hz is more than adequate.
For attitude related services, 10hz is important.
@bradanlane - agreed! @skypuppy - we still have a bit of room with the processor.
How does GPS data rate impact attitude? Is it paired with the AHRS data in the code every time?
On 12/08/2015 05:07 AM, bradanlane wrote:
If 10hz is possible, there is advantage to it. Over the past 10 years, more and more systems have switched to the faster data rate.
The challenge for Stratux is whether or not the advantages are part of its core purpose or are "new opportunities".
For navigation, weather, and traffic, 1hz is more than adequate.
For attitude related services, 10hz is important.
— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-162851597.
Re: GPS data rate and attitude
I started a separate topic to not detail this one. See #136
I received the board in the mail today. I'll hook it up and see if I can figure out what is happening.
AHRS running for a long time issue - v0.5b5. Please test.
Anyone ever test this?
I flew 4 hours this weekend with version 5.5 and all worked great with no issues! Thank you!
mooneyguy (Mark Miller)
From: cyoung [mailto:notifications@github.com] Sent: Saturday, December 05, 2015 3:12 PM To: cyoung/stratux Subject: Re: [stratux] Ahrs stops responding if run overnight (#131)
That would do it - going to look into that
Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/131#issuecomment-162255908 . https://github.com/notifications/beacon/APEqpqayhaV_6BDcD23mlgZDbr5PG_6iks5 pM2bZgaJpZM4Gskw_.gif
I ran my new LEADIY AHRS board (which has same chips as RY835AI (except no GPS)) overnight on Stratux v0.7b3+18de.sh alpha. No issue. (I didn't have it on a turn table tho.)
Old issue. Thanks for following up @Ergonomicmike.
This has happened twice in a row now. I've left my set up on my bench running overnight with the RY835AI. Throughout the evening I'd run the Stratux GPS/AHRS page and found reasonable values. By morning all values aside from altitude are zero and moving the ahrs around shows no change. Reloading the browser etc makes no difference. I'm running v5 now but I also saw this with v4r4. Any suggested debugging?
-Robert