Version 3.0 alpha Test

[mackworth]

So, TVDB is about to turn off its old API, which we use for artwork and to make up for TiVo not providing Season/Episode info any more. So, I've ported to their new API, but I was also inspired to add artwork to the UI, and also to use TiVo's semi-documented RPC mechanism. So this turned into a bigger project, and here's the first test version.

Please note that this is an alpha release, and should be expected to have problems, but I'd like to get feedback on any issues that come up. Very few changes have occurred in the download process, so that shouldn't give you any grief, and you should be able to go back to 2.5.1 without any problem (just PLEASE don't run both at once). If you do download it, please "Watch" this Issue to ensure you get notified of updates, and please let me know of any glitches whatsoever.

See the Release page at https://github.com/dscottbuch/cTiVo/releases/tag/3.0alpha1

3.0 Features:

New Artwork column in Now Playing List Your choice of artwork from TiVo or theTVDB (Series, Season, Episode, or Movie from theMovieDB); let's just say this was tricky. Manual selection of artwork by drag/drop onto table (even retroactively for shows already downloaded) Migration from theTVDB older API (which expires in October) Much better matching with TVDB; enhanced statistic information Episode and Genre information from TiVo now correct

Minor changes:

Manually edit season/episode information in drawer if Tivo is incorrect, (or prefer TVDB info) Extended Subscription option (no GUI, set SubscriptionExpirationDays in Console) Don't record Subscriptions if already on disk Reload TVDB contextual menu item Newest version of HandBrakeCLI

Bug Fixes:

Fix problem of Tivos sometimes not getting called during startup Allows season zero (specials); formatted as S00Exx Sorting by Names now sorts series without episode information by Original Air Date Change to .ts for unencoded files copied over Transport Stream (for compatibility with PyTiVo)

Internal changes:

Moved artwork and TiVo detail information to Cache folder, where they belong. Remove iTiVo migration Remove Growl support

Usage notes:

Thumbnails in Artwork column reflect images that will be added to the show when downloaded. You select which kind of artwork you want in Preferences. If you don't like the artwork for a particular show, you can drag a better one onto the artwork cell, and your manual one will be used instead. Note that this manual one will also be applied if you have already downloaded the show (and cTiVo can still see the file). If you don't want any artwork for a particular show (or to reverse the manual choice), just drag the image to the trashcan. Note: You can change the size of the image by changing the width of the column, which will increase the height of the row accordingly.

Requires Mac OS 10.9 and up. This version does NOT fix the double window High Sierra problems folks have identified. It may fix the tmp startup problem, but probably not.

Much thanks to Kevin Moye (of KMTTG) and Anthony Lieuallen for blazing the trail on the undocumented TiVo RPC information.

[tannebil]

Ran build 754 for about a week with excellent results. It fixed 80% of my issues with missing season/episode data. Not sure what magic fixed BBCA shows like Endeavour on Masterpiece but they are almost all coming through with season/episode data now. I've got 536 shows on my TiVo at the moment and the only missing season/episode are sports, some episodes of @Midnight, a couple of BBCA shows, and a couple of episodes of Full Frontal. No crashes.

Some unreliability with Handbrake encodes but nothing annoying enough to investigate too closely. I just rerun them with Default format and that does the trick. I'll take a closer look at the next one to see if there's something useful in the log.

[mackworth]

Great to hear. So, season/episode info is now taken from both TiVo's RPC and TVDB (priority on conflicts given to whichever you select in Adv Pref), so if you see any conflicts with their information, let me know. Both were completely rewritten from scratch for 3.0, so glad to hear they're working for you!

Just FYI, TiVo divides the world into Movies, Episodic and non-episodic (such as news, sports, etc). You can see which a show is categorized as in the Episode ID: Movies start with MV, Episodic with EP and non-Episodic with SH.

[mackworth]

So there's definitely a crash on High Sierra with drag/drop. I think I have it fixed, but I'd like someone to test it before I post a new version.

I'm also seeing now a repetitive crash on Sierra (10.12), but only a couple of people showing it, and the crash data alone isn't enough for me to isolate. If you're having this problem, please let me know to figure out what's going on.

[galacticacic]

crash on High Sierra with drag/drop

On Tue, Sep 26, 2017 at 3:56 PM, Hugh Mackworth notifications@github.com wrote:

So there's definitely a crash on High Sierra with drag/drop. I think I have it fixed, but I'd like someone to test it before I post a new version.

I'm also seeing now a repetitive crash on Sierra (10.12), but only a couple of people showing it, and the crash data alone isn't enough for me to isolate. If you're having this problem, please let me know to figure out what's going on.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dscottbuch/cTiVo/issues/265#issuecomment-332316506, or mute the thread https://github.com/notifications/unsubscribe-auth/AJwir9hO0cvwftn9p26c7TIGhIqnP9Imks5smVbXgaJpZM4PNtM7 .

[mackworth]

Great! (I guess)

Can you try this version? https://www.dropbox.com/s/e36lch2to9mukwe/cTiVo3.0a4%28Drag%20test%29.zip?dl=0

[galacticacic]

Crash continues

On Tue, Sep 26, 2017 at 4:28 PM, Hugh Mackworth notifications@github.com wrote:

Great! (I guess)

Can you try this version? https://www.dropbox.com/s/e36lch2to9mukwe/cTiVo3.0a4% 28Drag%20test%29.zip?dl=0

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dscottbuch/cTiVo/issues/265#issuecomment-332325488, or mute the thread https://github.com/notifications/unsubscribe-auth/AJwir_t06ixQJW84uZar_98Hx7774ANwks5smV5sgaJpZM4PNtM7 .

[mackworth]

Thank you. How 'bout this one? (pls confirm it says 712 in About cTiVo...)

https://www.dropbox.com/s/66ju8vizv80yzrd/cTiVo3.0a4-712.zip?dl=0

[galacticacic]

Says 772 in about

Drag n Drop worked without crash on this one.

On Tue, Sep 26, 2017 at 5:07 PM, Hugh Mackworth notifications@github.com wrote:

Thank you. How 'bout this one? (pls confirm it says 712 in About cTiVo...)

https://www.dropbox.com/s/66ju8vizv80yzrd/cTiVo3.0a4-712.zip?dl=0

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dscottbuch/cTiVo/issues/265#issuecomment-332336419, or mute the thread https://github.com/notifications/unsubscribe-auth/AJwir8M88QCsOh4BQBzmSP8NR1aO1cmnks5smWd6gaJpZM4PNtM7 .

[heyneenz]

Encoding fails on High Sierra.

[mackworth]

So, a little more detail would be good. Which Format were you using? Did it fail immediately or part way through? Is it true on every show, or just one? Could you attach the log (Help> Show logs in Finder, take latest log and Compress, then attach to a note here Thanks!

[tannebil]

Running a4 build 772 on High Sierra. 13 programs downloaded using either a custom HB encode or the Default format. 100% success. No issues.

[heyneenz]

I'm not sure what the issue was but it seems to be resolved. I had to uninstall cTivo and all related system files/folders, reboot and reinstall. Encode working fine on the 1080p setting now. But it had been failing at the encode stage - it would download, detect commercials and then begin the encode process and then fail at the end. I'm sorry I didn't save the logs.

As always, thank you for your hard work on this. It's much appreciated!

[mackworth]

Ah, sorry to hear that you had to go through all that. Any chance the logs are still there, or in the trash?

Your original question asked about disabling the art. If you mean in the UI, just Right-click on the column headings and turn off the column (that will also not download the art). If you don't want it in the file, just select None in the Preferences (which will also turn off the column contents as that reflects what will be in the file).

[mackworth]

@tannebil Thanks for the report. It’s good to hear when things are working as well!

[jgulla]

I just downloaded Alpha 4 (772), and after initial install (and entering my MAK) and it crashes on startup every time. latest 27" iMac running High Sierra/APFS. Happy to provide more information if it helps.

[mackworth]

That's great! (Your offer to help, not the crash.)

I'm seeing the crashes (anonymous reporting service), but there's almost no information about what's happening. Very few are getting these crashes, but those that are are getting many of them.

First, can you just post the log (Cmd-Shift_G in Finder, then ~/Library/Logs/cTiVo )?

Second a few details: Were/are you running cTiVo before? Sounds like not, if it asked for the mediaKey... Could you run 2.5.1 to see if it works for you? And to confirm, when it happened, you only launched the program and typed in the mediaKey? It's never run successfully? Do you see it finding/beginning to load the TiVo? Do you have anything in the Download queue waiting to process? (presumably not if you've not run 2.5.1 before) But if you do, and you immediately type Cmd-U to pause the queue when cTiVo starts, does it still crash?

Third, can you run the program and hold down Control-Option when it starts? This changes the logging to "Verbose" which will give me more info. (Later, when this is working, be sure to set it back to Report or Major or it will slow things down for you.

Again, thanks for checking in.

[jgulla]

Thanks for getting back to me so quickly! (and for the software of course!).

So I had not run cTivo on this iMac before (I have on another system). Obviously, nothing in the download queue. And yes, I launched the v3 the first time, dialog box popped for for my MAK which I entered, then the main screen popped up. Nothing populated, then a few seconds later the crash. After that, every launch no MAK dialog, nothing gets populated, a few seconds then crash.

I am on a latest gen 27" iMac (i7) with SSD and APFS. I have run cTiVO from an external (HFS+) drive, and I tried coping it to my SSD (APFS) - same crash.

Here's the regular log: https://www.dropbox.com/s/ilf4hty67hqlzee/com.cTiVo.cTiVo%202017-09-30%2014-31.log?dl=0

and the verbose log: https://www.dropbox.com/s/xdco6mbvy0e4cdu/com.cTiVo.cTiVo%202017-09-30%2023-46-VERBOSE.log?dl=0

Version 2.5.1 runs and connects to my TiVo (I assume it's reading the config file that v3 set up, as it didn't ask for my MAK again.)

I made a screen recording of the launch / crash so you can see the timing: https://www.dropbox.com/s/d2lg0ag98ro7px0/cTiVo%20Crash.mov?dl=0

Let me know if you need any more info or want me to try something else. Happy to try any test builds if you think it will be helpful.

Jerry

[jgulla]

OK, so here's an interesting update. I cloned the repo, built and ran from Xcode. That fired up just fine. I then quit Xcode, ran the v3 Alpha version I downloaded and that ran fine, too.

Not sure if that data is helpful, but thought I should pass it along. If there are any files you want me to clean and try and re-create the problem let me know and I'll give it a go.

Jerry

[mackworth]

---For the non-technical copied on this thread, feel free to delete this email immediately!--- Well, that rebuild question is certainly fascinating, and I've have a hard time explaining that (unless maybe the TiVo queue changed in the mean time; see below).

The bad news is for some reason you can't reproduce it now, so it'd be certainly be great if you can restore it back to the same state... Only files/directories that should be created are: ~/Library/Preferences/com.ctivo.ctivo.plist /tmp/ctivo ~/Library/Caches/com.ctivo.ctivo Also, if you clicked on Keychain, then it would store the Mediakey in your Keychain as well.

Just for my own thinking at least, let me walk through what I've got (with two reporting cases with logs, plus a few others that are seeing the problem, but haven't reported in). This is High-Sierra specific, but most people on HS don't run into it. It occurs during initial launch, but not necessarily first time on a machine. When it does occur, it happens over and over.

When we first see a TiVo, we ask it for the first 15 shows, and it responds with an XML file that gives us the basic information. We parse the first x shows fine (in one case the first 10, in the other the first 8). In your (@jgulla's) case, it gets through these shows:

The Late Show With Stephen Colbert - Jerry Seinfeld Carol Burnett and Friends Fresh Quilting - Design Options The Vietnam War - The Weight of Memory (March 1973-Onward) The Vietnam War - A Disrespectful Loyalty (May 1970-March 1973 The Vietnam War - The History of the World (April 1969-May 1970) The Vietnam War - Resolve (January 1966-June 1967) And it fails before The Vietnam War - The River Styx (January 1964-December 1965

The XML it's parsing at the time (in both cases) looks like:

<AcceptsParams>No</AcceptsParams>
</TiVoVideoDetails>
</Links>
</Item>
<Item>

and the parsing report looks fine, but then stops...

> Start: AcceptsParams
> AcceptsParams --> No
> TiVoVideoDetails finished which doesn't do anything
> Links finished, which doesn't do anything.

I don't see `Item` finished, but that may be lost in the crash. The other log file finishes at almost exactly the same point, except it doesn't show Links finished, so again, that may be a caching issue on the log file.

When item finished, then a buncha stuff happens:
   it checks for duplicates of the shows so far (and prints out `skipping dup` if it finds one).
   Then it checks if it's an` in-progress` show (to handle a strange interaction with tivo start times), which this isn't
   then if it's an full tiVo XML load, then it looks to see if it had it in a prior pass (not in this startup case)
   Then it checks to see if there's a Download matching this one to connect the two (not in this case)
   Then it saves this new show in a mutable array, and puts it in a queue for a background task to get the "detailed information". There's only four of these running at a time, so it's probably not launching one right now.
Then what should happen, is that it sees (or should see) the next "item", then it creates a new Show, and starts loading that data (none of which we see here).

So why is number 8-10 different that all the ones before?

The thing that's totally frustrating is that the crash log (see below) is clearly showing as happening in tableView layout. And consistently; every time the exact same main-thread stack across each machine (but same High Sierra OS)) The problem is that all that XML parsing also occurs on the main thread (no judging! i've checked and it's very fast), and the logging system shows it happening 55-65% of the way through the parse, so it can't be doing table layout until the parse finishes. (Unless the logging system is losing hundreds of lines and then ending up in both cases within 1 line of the end of 70+ lines/item).

So if you can reproduce it, that would be great.  I'd love to know if it happens on the same exact show the next time. Also, one possible explanation why it stopped happening would be that the shows moved down in the queue to the next load group.  (Don't know why that might matter, but it'd sure be interesting). I'd also like to get the actual stack trace from one of these runs to make sure that Crashlytics is reporting what actually happened.

0 Crashed: com.apple.main-thread

EXC_BAD_ACCESS KERN_INVALID_ADDRESS 0x0000000000000000

Crashed: com.apple.main-thread 0 AppKit 0x7fff385b5974 57-[NSTableRowData _addRowViewForVisibleRow:withPriorView:]_block_invoke + 47 1 AppKit 0x7fff385b5191 -[NSTableRowData _preparedRowViewForRow:storageHandler:] + 131 2 AppKit 0x7fff37cc8b97 -[NSTableRowData _addRowViewForVisibleRow:withPriorView:] + 392 3 AppKit 0x7fff37cc8958 -[NSTableRowData _addRowViewForVisibleRow:withPriorRowIndex:inDictionary:withRowAnimation:] + 416 4 AppKit 0x7fff385b73df -[NSTableRowData _updateVisibleRowEntries] + 1906 5 AppKit 0x7fff37cc7656 -[NSTableRowData updateVisibleRowViews] + 294 6 AppKit 0x7fff37cc6e26 -[NSTableView layout] + 206 7 AppKit 0x7fff3846f080 _NSViewLayout + 587 8 AppKit 0x7fff37c6ab1b -[NSView _layoutSubtreeWithOldSize:] + 437 9 AppKit 0x7fff37c6ad94 -[NSView _layoutSubtreeWithOldSize:] + 1070 10 AppKit 0x7fff37c6ad94 -[NSView _layoutSubtreeWithOldSize:] + 1070 11 AppKit 0x7fff37c6ad94 -[NSView _layoutSubtreeWithOldSize:] + 1070 12 AppKit 0x7fff37c6ad94 -[NSView _layoutSubtreeWithOldSize:] + 1070 13 AppKit 0x7fff37c6ad94 -[NSView _layoutSubtreeWithOldSize:] + 1070 14 AppKit 0x7fff37c6ad94 -[NSView _layoutSubtreeWithOldSize:] + 1070 15 AppKit 0x7fff38475ddc -[NSView _layoutSubtreeIfNeededAndAllowTemporaryEngine:] + 1380 16 AppKit 0x7fff37c88303 -[NSWindow(NSConstraintBasedLayout) layoutViewTree] + 163 17 AppKit 0x7fff37cf3b6b -[NSWindow(NSConstraintBasedLayout) layoutIfNeeded] + 268 18 AppKit 0x7fff384a37a1 NSWindowGetDisplayCycleObserver_block_invoke.5891 + 67 19 AppKit 0x7fff37d041e2 37+[NSDisplayCycle currentDisplayCycle]_block_invoke + 695 20 QuartzCore 0x7fff4567c55f CA::Transaction::run_commit_handlers(CATransactionPhase) + 49 21 QuartzCore 0x7fff4567b121 CA::Transaction::commit() + 171 22 AppKit 0x7fff384badbc 65+[CATransaction(NSCATransaction) NS_setFlushesWithDisplayRefresh]_block_invoke + 283 23 CoreFoundation 0x7fff3a5c60c7 CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION + 23 24 CoreFoundation 0x7fff3a5c5fef CFRunLoopDoObservers + 527 25 CoreFoundation 0x7fff3a5a85a8 CFRunLoopRun + 1240 26 CoreFoundation 0x7fff3a5a7e43 CFRunLoopRunSpecific + 483 27 HIToolbox 0x7fff398c7866 RunCurrentEventLoopInMode + 286 28 HIToolbox 0x7fff398c75d6 ReceiveNextEventCommon + 613 29 HIToolbox 0x7fff398c7354 _BlockUntilNextEventMatchingListInModeWithFilter + 64 30 AppKit 0x7fff37bc544f _DPSNextEvent + 2085 31 AppKit 0x7fff3835a508 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044 32 AppKit 0x7fff37bba25d -[NSApplication run] + 764 33 AppKit 0x7fff37b893fe NSApplicationMain + 804 34 libdyld.dylib 0x7fff61b0b145 start + 1

[jgulla]

The good news is that after deleting the app and associated files, I am able to reproduce the crash. I was worried that my wife may have deleted some shows that would mask the problem (it's an old TiVo and we save far too much - hence the desire to offload some of it with cTivo ;-) but it does crash again. I saved another verbose log. You can find it here:

https://www.dropbox.com/s/p1u3oo16gjvpw54/com.cTiVo.cTiVo%202017-10-01%2011-35.log?dl=0

Let me know if you'd like me to try anything else.

Jerry

[tannebil]

Had my first glitches with alpha 3.

The Find command doesn't bring up the Find dialog. Neither the menu nor the keyboard shortcut worked.

I like a high density display so I don't display art. I quit from the above issue and when I relaunched cTiVo the program listing came up double-spaced. I added Art, removed Art, sorted on Date and then the table changed to single spaced.

A couple of failures using a Handbrake custom format but that's normal. I rerun with Default and that seems to always work.

[mackworth]

Hi, Bill!

So on the first one, I’m not seeing that. Which version of the OS are you on?

On the second, I see a path that could happen, so I think I’ve fixed it.

Just about finished with a massive High Sierra bug hunt, so expect a new version shortly and we can track down the Find problem if it’s still there.

[mackworth]

Closed this issue due to initial beta release