AQ vs. DataOut vs. Burst table issue

[violetteta8]

AQ table is missing triggers to activate causing single point missing data. DataOut table is not missing the trigger. Investigate the flags to see if there is an error.

[violetteta8]

I'm working on this. Made a branch that contains a few debug variables to determine when the program is reaching the AQ table and whether the BurstCount has reached 30 by that point (the trigger to store AQ table when in burst mode).

[violetteta8]

I have left a board running in the office while I was gone on vacation. So far, it hasn't missed a single data point for either DataOut or Aquarius. Still, I may come up with some language that will protect against not writing the Aquarius table appropriately.

[violetteta8]

We (@USGSminc and I) think we have come up with a solution for this bug. Mark will work on this issue. Set a timer, if timer reaches 30sec AND BurstCount > MinBurstCount (or similar variable) Then write Burst Table. Also, take AQ table trigger out of a conditional. It should write no matter if burst values are available or not.

[violetteta8]

As part of this issue, I'm working on the issue Leah described to us with the Burst values being offset 15min (behind) the internally logged sonde data. Need to adjust timing for when Burst is happening so the values are written BEFORE mm:15. As it currently stands the burst isn't complete until mm:30+ which means that when the AQ table gets written, it grabs the PREVIOUS values.

[violetteta8]

@USGSminc we need to discuss our timing problem. I found a much bigger bug and learned that we are NOT doing controlling our timing NEARLY as well as I thought we were due to a misunderstanding on the RealTime function WAAAAAY back when this program started. Let's talk tomorrow and see about tackling this. You wanted to learn about timing? Well... hopefully this new discovery can help us learn more and fix some other problems. In the meantime, I'm adjusting my Burst flag thing to trigger earlier in the current state, that way we can at least push this urgent fix to the station's being affected by the problem.

[violetteta8]

Got the burst to work correctly (timing-wise) tonight. Need to re-assess when/how variables are being written into the AQ table. But I think we can do a "MedBurstTurbFNU = Burst.BurstTurbFNU_Med" or something similar. Right now, the burst table gets written properly. But, because the AQ table doesn't get written until the beginning of the next scan using the "Median (1,BurstTurbFNU,MaxBurstCnt,IEEE4,WqBadData) " in the AQ table, it won't write properly. I believe this is because that by the time the AQ table writes, the Burst values haven't reached the "MaxBurstCnt" threshold because the burst happened the minute before... timing, timing, timing... gah. But I think I'm close on this issue.

[violetteta8]

Burst table seems to be working properly now. All Median values are populated for the AQ table by way of "MedBurstTime = Burst.BurstTime_Med". AQ then samples all Median values rather than calculating it's own medians within the table. After the AQ table is actually called, I reset all the Med... values to NANs. Then, if the Burst table doesn't get called, the AQ table will be populated with NANs. Otherwise, Burst tables gets triggered, and the above occurs. Seems to be working on the test logger for now. We should keep an eye on how many times the burst DOESN'T occur and debug that at some point. But for now, I think we have it. I'm keeping the branch open and separated until we are confident. I'll send my "final" version over and let it run for a couple days.

[violetteta8]

Not quite right yet. Seems the NAN function is working, but the Burst Table is getting written to A LOT with NANs. Overnight, the Burst table had 521 records, AQ had 71. It looks like if it misses a Burst, it writes to the table ~30 new records full of NANs...

[violetteta8]

Really freaking close.

I pulled the data from the board in the middle of a burst sample and it properly exited the loop and (mostly) calculated how many times it successfully went through the loop (need to adjust the math). Burst table still gets populated with medians based on the number of values it had received before disconnect and it records the number of values that were used for the calculation. Need to check with Cathy/Alex on how we want to use this. We can either keep the Burst table populated with numbers and pull those numbers to the AQ table (and maybe add the number of values in the AQ table as well as another time series?) OR we could keep the burst table populated and NAN the AQ table, OR NAN both if we don't reach our minimum threshold for what's considered a "burst". Chat tomorrow.

[violetteta8]

@USGSminc

I was finally able to get an instance that the datalogger shifted values while I was recording in BBTalk. Looks to me like that sonde output all the values without issue but the datalogger didn't read them in appropriately/shifted them in some fashion. I'm curious what Campbell had to say to you the other day.

I don't believe eliminating the zeroes in the median calc for the Burst table will change the shifting values problem. I still think we should do it though.

Attached to this is the example I was able to record. One is the logger's emulator, one is BBTalk, and one is the SecBurst table. (Also, just attaching here to see how GitHub does it).

WQBurstTest_SecBurst Table_20190225_1314pst.txt

BurstTestBBTalk20190225_1313pst.txt

WQBurstTest_CR1K_20190225_1313pst.txt

[violetteta8]

Finally all is fixed. Loop works properly. Total Number of Values is working properly, Burst table omits zeroes (ERRORS, not REAL zero values) from it's median calculations. Comms work properly with and without Max232 on CR6 (C-Ports) and works with Max232 on CR1000 ComPorts. Set the "SetPeriod" to 1150 (1.150sec output rate) for both systems as CR1000 couldn't read data in fast enough for setperiod 1000. This way we are consistent between both systems.