Export Failures Since v0.3.11

[louacri00]

Describe the bug When exporting television shows from the Tablo using Tablo Tools v0.3.11, with Overwrite set as the duplicate-handling option, and delete when finished enabled, the queue will finish with "File does not exist after export" errors, and the exported files are not found on the destination drive. Sometimes they still exist on the Tablo and at other times I find the file size is only 35-40MB.

Before export, I re-sync the database, then apply two search scripts: one to detect files that are failed/dirty/98% or less complete, the other is for files less than 52% complete.

Tablo is an ethernet-connected Quad, SATA-connected SSD storage with ample space (has been attached to PC, no SMART errors), destination PC is a WiFi 6E-connected desktop, and average transfer rate is ~100 Mbps at peak with this version of Tablo Tools.

To Reproduce Steps to reproduce the behavior:

1. From the Search menu, select the quantity drop-down, then select the +ALL button to add to export queue
2. From the shopping cart drop-down, select export
3. On the export queue screen, set Max to two, click delete when finished radio button, on duplicate overwrite
4. Click export
5. See error

Expected behavior When an error is encountered, the application would halt/stop and show an error message. Error reports are allowed in the settings. Under advanced, I've enabled JSON export and debug level logging after this last occurrence, but I'm hoping you can pull an error report that might have been sent your way.

Screenshots See attached (search conditions I set, as well as errors shown in Export queue)

Environment:

• OS: Windows 11 22H2
• App Version from the Home screen: v0.3.11

Additional context I've been using Tablo Tools for quite some time now, but never experienced this sort of error until v0.3.11; while the throughput rate is massively increased, I've encountered this error 3-5x, and investigated the network connection (no drops on PC or Tablo side), disk capacity on Tablo storage and PC storage are ample.

[louacri00]

Just as an update, I haven't been able to reproduce this issue if the duplicate handling option is set to Skip, instead of Overwrite, so I'll keep using it with that option for the time being.

[jessedp]

Thanks for the follow-up - I'm poking around to see where things may be messing up. At the very least, it certainly sounds like this is not a network/storage issue, so that's helpful.

Since you turned on "Enable Debug logging" in Settings->Advanced, there should be extra logging info on your computer. Go back there and just below the setting you'll see the path on your computer where the logs are located (and you should be able to click the folder to open it).

There you should find a log file named something like this xxxxx-Archie Bunker's Place-YYYY-MM-DD_HH-mm-ss.log for the failed episode in your screenshot. It may look like garbage, but feel free to upload it for me to look through.

xxxxx = a "random" 6 digit number
YYYY = the Year the recording started
MM = the Month the recording started
DD = the Day the recording started
HH = the Hour the recording started
mm = the Minute the recording started
ss = the Minute the recording started

FWIW, the errors reported back to me are generally ones that cause the app to crash or do other bad things that you guys wouldn't notice. I also don't currently see anything that could relate to this there. Also the Export Tablo Data/json exports are unlikely to be helpful for you and won't help here.

[louacri00]

Thanks for the follow up!

Unfortunately, I didn’t have debug logging enabled before, since I didn’t expect this sort of behavior, and now that it is enabled I haven’t had it happen again.

I’m currently recording some more shows, so I’ll export them and try to reproduce the behavior after re-enabling the overwrite setting; haven’t seen a single recurrence since setting it to skip duplicates instead of overwriting them.

Also, it occurred to me that it might only happen for larger queues. I was exporting up to 300 shows, some smaller SD shows, larger HD shows (1080i), and mid-sized 720p shows. In my Tablo the recording quality is set this way:

I also noticed that after 300 shows, the queue took awhile to refresh. Is that a limitation of Electron, or am I possibly hitting a threshold somewhere? I tried 1,000 shows one time, more as a test, as at the time I was only exporting at about 10Mbps tops.

With this newest release, I’ve seen a massive performance boost, and really appreciate the work you did to improve the queue! The new UI enhancements really add to useability. I go to bed after setting up the export queue, and when I wake up it’s all exported without a single failure!

On Fri, Feb 3, 2023 at 1:10 PM jesse @.***> wrote:

Thanks for the follow-up - I'm poking around to see where things may be messing up. At the very least, it certainly sounds like this is not a network/storage issue, so that's helpful.

Since you turned on "Enable Debug logging" in Settings->Advanced, there should be extra logging info on your computer. Go back there and just below the setting you'll see the path on your computer where the logs are located (and you should be able to click the folder to open it).

There you should find a log file named something like this xxxxx-Archie Bunker's Place-YYYY-MM-DD_HH-mm-ss.log for the failed episode in your screenshot. It may look like garbage, but feel free to upload it for me to look through.

xxxxx = a "random" 6 digit number YYYY = the Year the recording started MM = the Month the recording started DD = the Day the recording started HH = the Hour the recording started mm = the Minute the recording started ss = the Minute the recording started

FWIW, the errors reported back to me are generally ones that cause the app to crash or do other bad things that you guys wouldn't notice. I also don't currently see anything that could relate to this there. Also the Export Tablo Data/json exports are unlikely to be helpful for you and won't help here.

— Reply to this email directly, view it on GitHub https://github.com/jessedp/tablo-tools-electron/issues/180#issuecomment-1416225805, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGE5UUCYZZFGYTQCFBTXG3WVVCZ7ANCNFSM6AAAAAAUND5HR4 . You are receiving this because you authored the thread.Message ID: @.***>

[jessedp]

I put out v0.3.12-beta.0 yesterday and promptly forgot to reply here.

I've not yet been able to replicate the problem you had, but did find a couple of places that had a slight chance of appearing to cause what you saw. I'll mess around with it some more, though.

Your recording quality screenshot didn't come through, but that shouldn't directly be an issue (though I can test with what you have). If it's related at all, it's simply that recording sizes are longer meaning exports take longer giving a greater opportunity for a network/computer "hiccup" to cause a problem. The debug logs you have turned on now would help answer that.

As far as 300+ (large numbers) slowing down the export queue display (and sometimes other places) - that's most likely sloppy coding on my part because I really didn't expect folks to do that :) It's certainly something I can work on, but I have no idea how long that could take to unravel.

[louacri00]

Thanks for getting back to me! On average, the recordings are downloading within 1 minute for a ~350MB 480i/p file, 1-2 minutes for a ~600-750 MB 720p file, and 3 minutes for a 1.1 GB 1080i file, even with a larger queue size.

I did change the setting back to Overwrite, but haven't experienced this again either, which is great. Also, as far as environmental stuff goes, I noticed Windows Defender Antimalware service was chewing up a ton of cycles yesterday when running an export job, so I wonder if that played into this issue at all.

I didn't get a Threat Notification whatsoever, so I suspect not, but possibly fewer CPU/RAM resources could have caused an odd race condition?

Also, if the majority of users are happy with the current queue handling, that's okay if you can't get to the larger queue handling performance stuff; for the time being, I've just been limiting it to a smaller queue size and running it repeatedly as needed. Maybe as an interim solution you could try limiting the queue size, or a warning if a larger queue is detected, but it's likely an edge case compared to the rest of your user base so not a huge priority.

I'm installing 0.3.12-beta.0 and will let you know how that plays out, if I can't reproduce this issue again over the next few days, we can likely close out this issue too; it happened several times before but of course now that I filed this it isn't reproducible!

[dougK44883]

As for que limits, I have had over 100 (likely < 125) at times without issues. I've learned to set duplicate at increment or just skip (personal preference).

It may be semantics, you interchange download and export?
I believe it will stream about as fast as a system can process within network limits. Within sensible limits, I suppose.

[louacri00]

Since the beta was installed, I've not been able to reproduce this issue. Thanks for checking into it!

[jessedp]

I'm interested to hear your feedback on v0.3.12-beta.1.

I know one broken thing is the Disk Info display when you leave and return to the page...

[louacri00]

Hi! I really have enjoyed using Tablo Tools for quite some time, but even more so lately! I'm currently running through an export queue that I started earlier this morning, and it's successfully exported over 1K files without a single error!!

[jessedp]

@louacri00 Wonderful to hear, thanks for the feedback!