Timestamp on Traffic WebUI row seems not right

[bkwny]

1. Stratux version: 6b1 plus the first 29 commits since it.
2. Stratux config: single/dual SDR, GPS yes/no (type?), AHRS yes/no, battery, battery cable. Dual SDR Nano 2. RY835AI via GPIO. AHRS is disabled. Line power via Turnigy 5.1V 3A battery eliminator powered by 12 VDC. From "tail -f /var/log/stratux.log" I can see the R Pi system time is set to UTC.
3. EFB app and version (e.g., WingX Pro7 8.6.2). EFB platform (e.g., iOS 9.2). EFB hardware (e.g., iPad Mini 2). N/A for this issue.
4. Description of your issue. If possible, enable "Replay Logs", reproduce the problem, and provide a copy of the logs in http://192.168.10.1/logs/stratux/ and http://192.168.10.1/logs/stratux.log. On the WebUI traffic display, for each traffic row, the last column seems to be a time stamp of some kind, for example, 04:31:38Z and increasing. But the value printed is not a UTC timestamp nor is it a local timestamp. There is some speculation that maybe it is the hh:mm:ss amount of time that has transpired since the R Pi was booted.

[cyoung]

Please use the issue template: https://github.com/cyoung/stratux/blob/master/CONTRIBUTING.md (linked at the top of the "New issue" page).

Some relevant information needed: version you're running and GPS or not (since the clock of the RPi is set off of the GPS when connected, not set at all if not connected).

[Ergonomicmike]

Not a big deal to me, but I've got the GlobalSat GPS. Internal Pi time synchs (per date command) to GPS. But the Traffic time stamp does not. My time stamps reset to 00:00Z on each reboot. Did it on v0.6r1 and still on v0.7b1.

[ghost]

As noted on reddit, this is time since boot, and is on my to-do list.

Stratux has used time since boot over the last few releases to track all time-critical activities. These include timing out old traffic, determining if GPS fix is recent, sending log messages, etc. This approach is used because the RPi lacks a realtime clock, and "jumping" the time forward or backward (as might happen with a GPS or UAT uplink providing time data) has caused odd problems in the past.

[cyoung]

We're not saving the incoming timestamps on traffic reports. Would you just use those?

[ghost]

Well, there are two ways I could go about this.

One is to change the presentation from timestamp to traffic age. That approach, I'd update traffic.js to convert the traffic timestamps to seconds, subtract from uptime, and display seconds since last seen.

The other would be as you suggest, and add a ti.Timestamp variable to show the actual time seen,

Any preference?

[Axtel4]

Whatever would make it easy to sync between the GPS, UAT, and 1090ES logs to aid in analysis. I currently use the "since boot" time stamp that is at the start of each line in the respective logs and convert the nanoseconds to minutes.

[cyoung]

I think most people are looking at the traffic page to see traffic flows, and the timestamp serves as an indirect indicator of how fresh each record is. Showing age directly both simplifies the problem and shows what is really needed. Parsing the timestamps from UAT packets has future value, but showing the age is easier and satisfies our current needs. Up to you - I'll assign this ticket to you and defer to you on it.

[Ergonomicmike]

If the "Zulu" reference in the time stamp were removed, it would eliminate most of the confusion .And perhaps change the title in the time column to "Rel. Time" or "Time since Boot."

[ghost]

Related issue with same root cause: #254

[ghost]

https://github.com/AvSquirrel/stratux/commit/367dedc5a6d316ecbf7efadc939cbd92d760ae5e

This commit restores the "old" behavior (traffic displayed with UTC time from the RPi clock) using ti.Timestamp, and adds a few minor cleanup / enhancements to GPS parsing and the traffic generation routine.

@cyoung - I still have a few things to do before I pull it into the master branch. I'm only starting to get up to speed with JS, but as I'm learning how to deal with its time functions, it seemed natural to add Stratux and local device time to the status page. I'm still planning to add a "freshness" of traffic data indicator.

[Axtel4]

Also, keep in mind while looking at the 1090ES Time Stamp UTC time IS NOT downlinked in any of the ADS-B DF messages from the target aircraft. The "UTC" time stamp comes from the OS and is appended to the dump1090 data during post processing. If the clock is off in the computing device, the Time Stamp will not reflect the actual time the target information as received.

[ghost]

@Axtel4 -- yes, thank you. I saw how time and date was recorded in the dump1090 code, and I should have known better, having previously looked at the DF messages. Time isn't sent with 978 traffic messages (ADS-B or TIS-B) either, so I'll pull out all of the time-related "to-dos" from traffic.go.

I'll poke around with FIS-B decoding this evening.

[bradanlane]

ironically, the traffic page initially showed "how old" traffic was - aka a delta. Users complained. So, the traffic page was changed to attempt to compute an actual time-stamp for traffic.

[Axtel4]

What I don't know is how long Stratux or a flight app "holds" on to a target. If compaired to TCAS it would put the targets in "coast" after approximately 4 interrogation misses (4 seconds). After 10 interrogation misses (10 seconds) it will drop the target if no interogations are received. I think what confuses people is Stratux / Flight App continues to display stale Targets for 10s of seconds.

Where is the balance? I don't know, maybe 10 seconds of not receiving a DF 17 message for a target. But, then the software would need a "track list" to know when to drop the target from the data stream.

Is the target held by Stratux or the Flight App? Maybe both and that adds to the length the target is displayed to the user.

[cyoung]

We do keep a track list and hold targets for 60s. After that it is held by the EFB, usually 10s or more (depends on the app).

[Ergonomicmike]

Yeah, iFly will show missed / lost traffic sliding down its Direction Vector for about a minute until it disappears. Not so good if Traffic is turning. Perhaps consider tweaking the Stratux timer to 30 seconds? (Or less?)

[Axtel4]

Yeah, iFly will show missed / lost traffic sliding down its Direction Vector for about a minute until it disappears.

That is how TCAS behaves except it drops the Intruder after 10 seconds instead of a minute. Once it puts an intruder in coast it will extrapolate the projected trajectory based in the last know trajectory. It will continue to do so until either it reacquires the target or the coast times out.

[cyoung]

Fixed with PR #259.

[ghost]

259 is in the master branch now. This switches the UI from showing a timestamp to traffic age.

Unlike previous attempts at using relative age, this is based solely on monotonic timer and is set within traffic.go as ti.Age. The JSON marshaller is now called once per second on all targets, whether valid or invalid, to make sure that their status is pushed to the webUI. By moving all of the time functions out of the client, this makes us insensitive to clock difference between Stratux and the connected device. Net result is that the UI refreshes cleanly and consistently -- if you see a 5 second age on a target, you can trust that it was updated five seconds ago.

Regarding comments by @Axtel4 , @cyoung , and @Ergonomicmike on traffic age, ForeFlight times out traffic after 20 seconds of no updates. It doesn't coast traffic either, based on testing with my traffic demo code. Even with the GDL90 timeout cut to 0.9 sec (guaranteeing I never sent "double" messages at the same location), targets stalled out exactly where I expected them to.

Anyway, I did cut the Stratux GDL90 timeout to 6 seconds to give other EFBs a fighting change in case they do trend traffic. The UI will now keep data in its table for 60 seconds before it's purged (down from 180 seconds), matching the purge timer in traffic.go.

That said, adding a traffic coast routine would be a worthwhile addition for a future commit.

[Ergonomicmike]

Thanks for all. I just updated to v0.7b2, both via a git pull, and then a .sh, just to see how it worked.

Interestingly, I am still seeing "Time" for a heading in the rightmost column, with the "Z" label. (I was surprised, 'cause I thought it was going to be Age. And so I was going to suggest "Z" to "Pi."

Also, since we're talking about time, any value in having GPS time (to the nearest minute) display on the GPS page, just under Track?

(Update: Just saw a bunch of commits for v0.7b2. Guess I'll pull those now.)

Edited: Heading to Track

[cyoung]

The changes above came in just after I made the cut for v0.7b2. So if you use the .sh to "update" after doing the git pull, you're actually rolling back. Sorry for the confusion.

[Ergonomicmike]

Okay, I did the git pull. The web UI still came up with Time. It took a shutdown of the browser (clear the cache? Reset fields?) for Age to show.

Wow. Everything looks cool. But I can see users bitching now about why their Stratux time doesn't match their device time. And they're going to think it's important that they match exactly, since the difference is shown. So I predict they're going to ask you all for a sync function. (Update: But they all probably have their tablets / phones set to time servers. I don't.)

And I see that Stratux time reports GPS time if one has GPS. So that kinda moots my suggestion for GPS time on the GPS page. Still ..

[cyoung]

Did you do a make ; make install ?

[skypuppy]

That would not be a bad thing, labeled as "real time" or "time of day" or somesuch. RPi2 system time should be set via GPS time, in more cases. However, there is that position lock time to consider... The MTK3339 GPS, and others, do have battery backup capability and I did discover that not only does it keep the ephemeris and last know position, it also maintains a real-time clock in itself! Yay. (Same GPS that Adafruit calls it's "Ultimate GPS." I like it a lot. Sadly, only an RS-232 serial interface to it.) It also give a one pulse per second on one of it's lines if that would be handy for anyone. Accurate to +- 10 nsec at each pulse. So there you have your GPS and RTC for the stratux to set it's time with during boot. Woohoo. Even without sats being visible or not lock yet. Call me crazy but I like having my files and other things labeled with the correct time of day, plus having the much more accurate system ticks for, among other things, traffic data. No real need to keep both but if there is only one, let it be the system ticks. :) Unless some EFB is not compatible. Will have to try with iFlyGPS when a stable release has it.

Thanks, all! Skypuppy

P.S. Going way out on a limb. Agreed to the Qt 30-day commercial evaluation. Monster download to the PC but where will I get $3,000 to pay for the dang thing if it comes to that?

On 02/17/2016 11:48 PM, Ergonomicmike wrote:

Thanks for all. I just updated to v0.7b2, both via a git pull, and then a .sh, just to see how it worked.

Interestingly, I am still seeing "Time" for a heading in the rightmost column, with the "Z" label. (I was surprised, 'cause I thought it was going to be Age. And so I was going to suggest "Z" to "Pi."

Also, since we're talking about time, any value in having GPS time (to the nearest minute) display on the GPS page, just under Heading?

— Reply to this email directly or view it on GitHub https://github.com/cyoung/stratux/issues/251#issuecomment-185554079.

[Ergonomicmike]

@cyoung, yes, I re-did the make, make install.

Also, here in Phx, where there's only one Air Transport flying tonight. I watched as he passed out of range. Age climbs to 60 seconds - and then holds indefinitely. In the "old days" the traffic page refreshed itself every now and then. Now I need to move to another page to refresh the traffic page. Not a big deal. But will it populate to infinity if a user doesn't clear it?

[Ergonomicmike]

@skypuppy - iFly is very near a public release of v9.4. (Like a week or two.) I'll post in the reddit when it's available for trying.

[ghost]

Age climbs to 60 seconds - and then holds indefinitely

Traffic older than 59 seconds and less than or equal to 60 seconds wasn't triggering it to clear. Try https://github.com/AvSquirrel/stratux/commit/9ad3a7e333768e41eb5bf41d6c15c604e68ed86e

[Ergonomicmike]

That was quick. I'll try it tomorrow. Midnight now.

[Ergonomicmike]

@AvSquirrel, the fix works. (Yeah, I shoulda gone to bed. Guess you can tell I'm an engineer at heart.)

Update: Guess I misunderstood. The Stratux time is not syncing with GPS time. But, now that it's after midnight, the Statux time is off by one hour. (And about 30 seconds too.) I thought it was supposed to lock to GPS time? (I did the make; make install on the Pi by SSH'ing in over Wi-Fi. So I disabled the GPS and the SDR's in the Settings page first. When I re-enabled everything, the Stratux time was hosed. Reboot/Shutdown don't fix it. Disabling/Enabling GPS didn't fix it. Will investigate more tomorrow.

[bradanlane]

Nice work @avsquirrel. Someone will ask so I'll ask - what are the units for "age"? I assume it's "seconds" but the UI does not make it obvious.

[bkwny]

On my Win 7 + Chrome machine this morning, the UI incremented two target rows' ages up to 59.x seconds and then quit updating them. The two rows are still sitting there. It's been at least 10 minutes since the rows stopped aging.

Edit: It's been about 15 more minutes. The two rows are still there, joined by three more. All five rows have in common the trait that the displayed age is 59.x seconds and they're no longer updating. On my ForeFlight map page no targets are visible.

[Axtel4]

Pull and make https://github.com/AvSquirrel/stratux/commit/9ad3a7e333768e41eb5bf41d6c15c604e68ed86e?

On February 18, 2016 7:18:41 AM CST, bkwny notifications@github.com wrote:

On my Win 7 + Chrome machine this morning, the UI incremented two target rows' ages up to 59.x seconds and then quit updating them. The two rows are still sitting there. It's been at least 10 minutes since the rows stopped aging.

---

Reply to this email directly or view it on GitHub: https://github.com/cyoung/stratux/issues/251#issuecomment-185713818

[bkwny]

I have it already. git pull says "Already up-to-date."

[bkwny]

One of my stuck rows is displaying age 60.0. I thought that might be relevant so I am posting it.

[ghost]

@bradanlane - units are seconds. Traffic help file is updated to state that, but units are not listed on the main traffic page.

@Ergonomicmike - can turn on logging (if it's not already on) and pull some recent stratux.log and nnnn-gps.log data from the latest commit? I don't have a BU-353 and would like to see exactly what output the @iposva configuration is giving us work with. I suspect that the GPRMC messages isn't formatted like I think it is, and that it's stumbing when parsing the time. I'll be able to look at it tonight.

@bkwny -- the fix isn't in the main branch yet; you'd have to pull it from the commit listed above, or manually edit or copy that traffic.js onto your Pi.

[Ergonomicmike]

I manually made the change to the .js file and thought that would work without doing a make. But it didn't. I had to do a make; make install.

@AvSquirrel, Wilco.

[Ergonomicmike]

@AvSquirrel, I tried to make it easy for you. I deleted the old stratux.log so that the new is short. And a short gps.log.

https://www.sendspace.com/filegroup/jAM9YcHsgKmnEahc%2Bw%2Fsbw

[Ergonomicmike]

iFly is painting Traffic motion a lot smoother since all these changes. UAT direct traffic that used to flicker between Call Sign and xpnder code on the Stratux status page (per a video I posted a while ago) are less flickery.

[iposva]

@AvSquirrel Logged into my Pi, stopped stratux and dumped the raw serial output from the BU-353. Log is below. Let me know if you need more...

root@raspberrypi:~# /etc/init.d/stratux stop [ ok ] Stopping Stratux ADS-B Receiver: stratux. root@raspberrypi:~# cat /dev/ttyUSB0 $GPGGA,170937.000,3725.0786,N,12204.3421,W,2,07,3.0,28.9,M,-25.6,M,2.0,0000_7A $GPGSA,M,3,22,11,01,08,27,03,48,,,,,,4.2,3.0,2.9_33 $GPRMC,170937.000,A,3725.0786,N,12204.3421,W,0.36,34.33,180216,,,D_47 $GPVTG,34.33,T,,M,0.36,N,0.7,K,D_3D $GPGGA,170937.20$GPGGA,170938.000,3725.0786,N,12204.3422,W,2,07,3.0,28.5,M,-25.6,M,2.0,0000_7A $GPGSA,M,3,22,11,01,08,27,03,48,,,,,,4.2,3.0,2.9_33 $GPRMC,170938.000,A,3725.0786,N,12204.3422,W,0.37,34.33,180216,,,D_4A $GPVTG,34.33,T,,M,0.37,N,0.7,K,D_3C $GPGGA,170938.200,3725.0786,N,12204.3423,W,2,07,3.0,28.5,M,-25.6,M,2.2,0000_7B $GPGSA,M,3,22,11,01,08,27,03,48,,,,,,4.2,3.0,2.9_33 $GPRMC,170938.200,A,3725.0786,N,12204.3423,W,0.43,34.33,180216,,,D_4A $GPVTG,34.33,T,,M,0.43,N,0.8,K,D_30 $GPGGA,170938.400,3725.0786,N,12204.3423,W,2,07,3.0,28.4,M,-25.6,M,1.4,0000_79 $GPGSA,M,3,22,11,01,08,27,03,48,,,,,,4.2,3.0,2.9_33 $GPRMC,170938.400,A,3725.0786,N,12204.3423,W,0.17,34.33,180216,,,D_4D $GPVTG,34.33,T,,M,0.17,N,0.3,K,D_3A $GPGGA,170938.600,3725.0786,N,12204.3423,W,2,07,3.0,28.4,M,-25.6,M,1.6,0000_79 $GPGSA,M,3,22,11,01,08,27,03,48,,,,,,4.2,3.0,2.9_33 $GPRMC,170938.600,A,3725.0786,N,12204.3423,W,0.12,34.33,180216,,,D_4A $GPVTG,34.33,T,,M,0.12,N,0.2,K,D_3E $GPGGA,170938.800,3725.0786,N,12204.3422,W,2,07,3.0,28.3,M,-25.6,M,1.8,0000_7F $GPGSA,M,3,22,11,01,08,27,03,48,,,,,,4.2,3.0,2.9_33 $GPRMC,170938.800,A,3725.0786,N,12204.3422,W,0.03,34.33,180216,,,D_45 $GPVTG,34.33,T,,M,0.03,N,0.1,K,D_3D

[bradanlane]

Since all the other data in the traffic list have appropriate units, we should add "s" for seconds. I realize it uses up space but having only one column sans units makes for an inconsistent display.

[ghost]

Fixes for the "sticky" 59.x second traffic and GPS time updates are in #264 .

@Ergonomicmike , @iposva -- thanks for the data. I missed a "%" in the string used to set GPS time after parsing an RMC message, and didn't catch it during debug.

[ghost]

@bradanlane - fine by me. Do you want to get that one, or should I pick it up in my next commit?

[bradanlane]

I'll add the units.

I should probably also do a test with that handy new traffic simulation code :-)

[Ergonomicmike]

Capital S or no? https://www.unc.edu/~rowlett/units/symbol.html

[bradanlane]

from that URL, I would say "lower case 's' is correct".

[Ergonomicmike]

I agree. I get confused because they suggest "A" for amps. But not "S" for seconds.

Even if cap S was (were?) correct, most Americans probably wouldn't recognize that as seconds. Whereas we see yymmddss, so know that s = seconds.

[Ergonomicmike]

@AvSquirrel Yea, it works! Pi time instantly updated to GPS time. Thanks.