ac2cz / FoxTelem

AMSAT Fox-1 Telemetry Decoder
GNU General Public License v3.0
52 stars 18 forks source link

Long frame que causes Slow Upload/No Decode/Hang on restart after long internet downtime #364

Open burnsfisher opened 7 years ago

burnsfisher commented 7 years ago

We had a storm here and the internet was down for several days, although my Fox-1E EM and the computer with FoxTelem kept running (may have been off for a couple hours). In any case, I got thousands of packets queued up to be sent to the server. When the internet came back. it does appear that data started being uploaded, although pretty slowly.

When I finally got home to look at FoxTelem, I found that it was using something like 80% of the CPU time, and FFT display was jerky (although not lost audio being shown) and nothing was being decoded (although given the time of the last decode, I don't think the internet coming up and having it stop decoding were coincidental.

I stopped and restarted FoxTelem. Now it is hanging (at least 5 to 10 minutes) on Loading Logged Data 80%. I first hypothesized that it was trying to upload all the thousands of packets to the server, but I don't see anything on the server.

burnsfisher commented 7 years ago

BTW, the log says that the last thing it was doing was loading rawPSKframes.log.
That file is about 16.8MB. Renaming it allows FoxTelem to start quickly, and when it starts, it runs and decodes. (This is 1.05t. I did not want to interrupt it to upgrade while I was doing a long-running test on the EM).

ac2cz commented 7 years ago

This is a bug because of the way that the queue is handled. We save the records into a file in case we crash. When a record is deleted we remove it from the queue, then recreate the file. So it is repeatedly writing a file with thousands of records. Given that the queue is usually short. this is not usually an issue. I've thought of changing this before. We could just write the queue on exit, but then we might lose records if we crash. The server stores the files in a directory, with one per STP record. I could consider that instead.

Fixing this would be low priority. This is only an issue in testing when the queue can grow into the thousands.

Chris

On Wed, Nov 1, 2017 at 3:55 PM, Burns Fisher notifications@github.com wrote:

Assigned #364 https://github.com/ac2cz/FoxTelem/issues/364 to @ac2cz https://github.com/ac2cz.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ac2cz/FoxTelem/issues/364#event-1321271194, or mute the thread https://github.com/notifications/unsubscribe-auth/AOI1WdX7u7D3pCKI0YaUnCXs-lBF2Ok6ks5syJSMgaJpZM4QOZuF .

-- Chris E. Thompson chrisethompson@gmail.com g0kla@arrl.net