ctivo 3.6 keeps crashing

[ataridude]

Hi. I upgraded to cTiVo 3.6 in the last few days, and it has been crashing repeatedly since I upgraded. Is this a known issue? I can post a log next time it happens.

[splark]

I am having this same issue on a late 2014 iMac stuck on Big Sur (11.7.8). Downloading using the Default format. The failures seem to occur neat the end of encoding or perhaps while adding to Apple TV. I have only attempted to download this one file.

Steve

[mackworth]

Can you post the log? (Help>show logs in finder, then option-click on the latest, Compress file and attach it to a comment here)

[ataridude]

ctivo.crash.txt.gz

[ataridude]

Sorry, I realize that was the crash log, not the cTiVo log, which is what you requested. That is attached here. com.cTiVo.cTiVo 2023-07-28--12-02-13-762.log.gz

[mackworth]

That's very strange. It's crashing in the mpv4 code, which hasn't been touched in years.

I've just uploaded 3.5.3 again. Can you try downloading it and see if you get the same crash?

[mackworth]

I note also you're getting a lot of errors of this form: No base name; with file pattern [[MainTitle / MainTitle " - s" Season "e" Episode " - " EpisodeTitle | Guests | OriginalAirDate]|["Movies" / MainTitle " (" MovieYear ")"]] for show Leave It to Beaver: Found Money

I haven't figured out why that might affect it, but maybe try downloading that show with a default file pattern instead.

And can you try same show with HB Default instead of Default as a Format?

@splark can you upload your log to see if there's anything similar?

(According to crashlytics, there's only two folks reporting this problem, and so I'm guessing that it's the two of you :) )

[ataridude]

Version 3.5.3 does not crash for me. Thank you.

[splark]

Here is the log. It includes both a Default and HB Default attempt to process the file. Both failed.

com.cTiVo.cTiVo 2023-07-29--15-46-08-541.log.zip

[splark]

A couple other things...

I noticed the destination folder had and .mp4 and a .srt file for the movie.

[mackworth]

So, what's happening is that the low-level calls to set the subtitles or the chapter markers is crashing. (This happens at the end of the download, so yes, the folder will already have the MP4 and the subtitle srt file, if any) It's trying to create a new track in the MP4 file and apparently failing. I've added a check for this to report and avoid the crash, but I'm first interested in why it's failing, as the patch will leave the subtitles/chapter markers unset. Nothing in this whole area was touched in 3.6, so it doesn't make much sense.

Can you do one more run of the file that's failing in 3.6, but with Debug set to Detail and then post that log?

(FYI, I cancelled the auto-upgrade to 3.6 until I resolve this.)

[splark]

Here you go....

com.cTiVo.cTiVo 2023-07-29--20-43-50-963.log.zip

[mackworth]

Well, that didn't add much, and I can't reproduce the crash on my systems. (So far the only crashes on are Intel iMacs on Big Sur and Catalina)

I've just released an initial 3.6.1 version, which should trap the errors and post some information about them. Let me know if it works for you on those files. If it works, then it won't crash, and the log will show "Error encoding chapters in show" and/or "Error encoding subtitles in show" as well as some information about what's being encoded.

I'd love to know if you're getting the same errors on all shows, or just the ones that crashed before.

[splark]

I backed 3.6 off after the first file I tried failed 3 times. For the testing I did today, I created a short recording on the same channel as the original video. It failed using the Default, HB Default, and 1080p formats. That seems consistent with your description of the issue.

I’ll create a few short recordings tonight and try to process them with both 3.6 versions tomorrow, if I can.

Is deselecting subtitles a work-around for this?

[mackworth]

Yup, all this occurs after the encoding, so the Format is pretty much irrelevant at that point.

Correct on the work-around, although you would need to deselect both Marking commercials and subtitles. 3.6.1beta should recover automatically, albeit missing the chapter markers and/or subtitles for the afflicted files. So the next question is whether it’s all your files or just that channel, or all Program Streams.

Assuming that works, then I’ll need to figure out why it’s failing. mp4v2 is very old code used in a bazillion applications. The failure mode is an odd one. As I can’t recreate it, I’m hoping you won’t mind running a few test runs.

[splark]

3.6.1 is blowing off the same way 3.6 did. This is log shows an attempt with the same show I had been testing and another on a different channel.

com.cTiVo.cTiVo 2023-07-30--17-16-33-793.log.zip

[splark]

I did a little testing on 3.6 on different channels. No surprises. Testing is kind of tedious, even with short files, since I have to wait for the crash, restart cTivo and start the next test. After a few crashes, my machine starts to behave erratically and has to be restarted.

com.cTiVo.cTiVo 2023-07-30--19-17-02-048.log.zip

[mackworth]

Thanks!

Okay, I now know what the problem actually is. Having read through the mpv4 code in more detail, the crashing code uses a c++ exception to pass info back up to its caller (something frowned upon by Apple, but common in c++). But, for some bizarre reason, in your case, that exception isn't being caught by that level, and my trying to catch it at the highest level in 3.6.1 also failed. So it's nothing to do with the particular file, or channel, or anything else. Apparently any attempt to create a new track (e.g. subtitle or chapters) on any MP4 will crash.

Now I need to figure out how to fix it; it's the same binary code being executed as 3.5.3, but somehow the execution environment has changed. It continues to look like an Intel-CPU problem. Without being able to recreate the issue, I have two theories: I used a newer compiler for 3.6.1 (although not for that chunk), but the linking process may have changed. The other is that I need to recompile that code with the latest compiler for some compatibility reason. So if you don't mind, I'll create two more versions with those two changes, and have you run them. Doesn't require multiple passes, just a binary crash/no crash on each version.

[splark]

Sounds good. Let me know...

[mackworth]

I've rebuilt the mp4v2 files, so here's the first one available at this link. BTW, About cTivo will show Version 3.6 (1207

[splark]

Looks like it blew off the same way.

com.cTiVo.cTiVo 2023-08-01--15-45-38-113.log.zip

[mackworth]

Sigh. That’s very disappointing. I rebuilt my system to be able to do the next test, but I don’t have much hope for it. Let me know if it fails again…

https://www.icloud.com/iclouddrive/046SXxliULZYdG64scoFs6mQQ#cTiVo.3.6.0.test2_copy

[splark]

Your pessimism was misplaced. Subtitles and all:

com.cTiVo.cTiVo 2023-08-03--16-46-57-495.log.zip

[mackworth]

That's great to hear. The only difference between those two versions was building it on Catalina/Xcode14 v Sonoma/Xcode15, so sounds like the beta compiler might have a bug, which is odd.

So far from field reports, it seems to be specific to Intel MacOS < 12.0, probably why it didn't come up in beta testing. If you happen to have an Intel Mac on MacOS 12/13/14, that would be great to test if you do. My theory is that there's an incompatibility with the libc++abi on those older OSs and Xcode15

I have a couple more tests if you don't mind; they're in this folder.

The first is to confirm that Sonoma/Xcode14 works (not technically supported by Apple, but it seems to work in general). That would be cTiVoTest.14S.test3.zip It seems very unlikely that the problem would be triggered by building on Sonoma (versus the Xcode version), but I want to be sure.

Second, to be able to report the problem, I've created a much simpler version of the file. It's a terminal app that creates a fake set of chapters in an MP4 file. I have created two versions TestException14 and TestException15 (created by Xcode 14 and 15). I'm hoping the 14 one passes and the 15 one crashes. To run them after downloading, find a small MP4 file that you don't care about (or duplicate one) drag the program into a Terminal session and (before hitting return), drag the MP4 file onto the command line. you should then see something like:

This program will add fake chapters to an MP4 file
argv[0] = '/Users/hugh/Developer/TestException/TestException 2023-08-03 17-55-23/Products/usr/local/bin/TestException'
argv[1] = '/Users/hugh/Downloads/test2.mp4'
Chapter 1: duration 1000, title: chapter 1
Chapter 2: duration 2000, title: chapter 2

unless it crashes. You can reuse the same MP4 file. You can see the result in QuickTime Player, selecting the >> icon and then hovering over the Chapters command. Note that you may get a Gatekeeper warning on the execution, as they're not notarized.

I hope to only have one more test after this: If these both work, I'll create a GUI version of the same thing to see if that's necessary to trigger the problem. If the Xcode15 version fails, then I'll create a much simpler library to see if I can still trigger the problem.

[splark]

My next newest machine is an M2, so I can’t help with the less old Intel stuff.

I’ll get into the rest over the next couple days.

[splark]

The Sonoma build worked.

com.cTiVo.cTiVo 2023-08-04--14-41-35-843.log.zip

I am having permission issues trying to run the test scripts. I don't seem to have access to the mp4 file and I am not sure what needs to be changed. I used to install and support Unix apps running on z/OS (mainframe), but as a retired guy, my skills are rusty.

root@Jupiter-4 ~ # /Users/spl111/Movies/TiVoShows/Movies/Movies/1917.mp4 /Users/spl111/Downloads/TestException14 zsh: permission denied: /Users/spl111/Movies/TiVoShows/Movies/Movies/1917.mp4 root@Jupiter-4 ~ #

Any thoughts?

[mackworth]

Yeah, sorry; that's the Gatekeeper stuff. Rather than find a way around it, I went ahead and built a Mac App instead. The 14 and 15 version are both in the same folder as before. When you run them, they will ask for an Mp4 file; you can then drag/drop it in. Any error messages will only be in the log (console), but if it doesn't crash, it probably worked; again you can check in QT Player.

[splark]

I see.

14 ran, 15 failed.

[mackworth]

Great! Much easier to submit. Here's testException4: the simplest possible test case; moving the c++ exception out of the library entirely, into the main app. I didn't bother with the 14 version, just the 15 one as I'm sure 14 will be fine. If it doesn't crash, the screen will just say "Third" (meaning it got to the third point in the program). If this one crashes, I'll submit it to Apple, but if it works, I'll try one more: a trivial library.

[splark]

I'm not sure what is out there. Here is the log for cTiVoTest.14S.test3, which I think I already had.

com.cTiVo.cTiVo 2023-08-05--16-11-17-302.log.zip

TestException4 will not execute on my hardware.

[mackworth]

Sorry, I created a clean project, and forgot to mark it to run on earlier versions. Here's a revised one (testException4.2)

[splark]

Ok, it worked, but I don't think that's what you were going for.

[mackworth]

Correct., although I would have been shocked if c++ exceptions weren't handled. Here's what I think is the final version of these tests. I'm replacing mp4v2 with a trivial library to see if the problem still shows up.

You'll see three apps TestException5.X.X Please run all three.

The first X is which compiler compiled the library and the second is which compiled the app.

I expect TE5.14.14 to pass (meaning it runs without a crash and the word Fourth appears). I'm hoping TE5.14.15 crashes. (If not, then I'm sure TE515.15 will also pass, and I'll have to submit the bug with mp4v2.) If TE5.14.15 crashes andTE5.15.15 then passes, then that gives us a possible fix for cTiVo (as well as a minor bug for Xcode). If it also crashes, we've got a stripped-down major bug for Xcode to report.

Thanks again for your persistence with me. I really wish I had the hardware to test this.

[splark]

All three passed (displayed "Fourth")

[mackworth]

Sigh. I'm about to submit the bug report, but I'd like to confirm one thing. I've been using a precompiled version of mp4v2. This test app below uses a recompiled version. (Use is same as before: use it to mark a junk mp4 with some junk chapters. After you select a file, either it'll crash or just return back to a blank window.) Thanks again.

TestExceptions6.zip

[splark]

I used to work for a guy who couldn't understand why I got excited when something blew off the way I expected it to. 20 years and a few role changes later he still introduced me as the only person he ever met who got excited when something didn't work.

You should be happy, it failed.

[mackworth]

Thanks. Can I get the crash report from this one to submit? Should be available under Console>Crash Reports>TextException6>(Cmd-Click)Show in Finder), but might be different under Big Sur.

[splark]

TestExceptions6_2023-08-08-162817_Jupiter-4.crash.zip

[mackworth]

Can you confirm this one still crashes?

TestExceptions6b.zip

[splark]

It crashed, but I didn't get "Quit unexpectedly" pop-up, which I thought was odd.

TestExceptions6_2023-08-10-134329_Jupiter-4.crash

[mackworth]

Apple has moved to a new linker. This is the same as the last one, but with the old linker. I'm hoping this one won't crash...

TestExceptions6c.zip

[ataridude]

Sorry I disappeared for a couple weeks. I'm back again and am happy to help test new versions if needed.

[mackworth]

That'd be great.
1) download and decompress the zip file into TestException6c.app 2) find a small MP4 file that you don't care about (or duplicate a small one) 3) Run the app 4) Drag/drop the Mp4 file onto the window. Either it'll crash, or nothing visible will happen. That's all I really need to know. (If the latter happens, you can confirm that it worked by looking in Console's log, or by opening the MP4 file and seeing if there are two chapters ,each about a second long. )

Either way, I'll have what I think is a 3.6.1 version that should run fine.

[splark]

It worked. Chapters were created.

[mackworth]

Great. Here's the beta of cTiVo 3.6.1 which should run fine on your machines. Let me know!

[splark]

Looks good. The files converted and had subtitles.

Anything else I should to look at?

[mackworth]

Nope. @ataridude, is yours good as well?

[ataridude]

Running your test app, I got 1 chapter at 00:01. I will try your beta of 3.6.1

[ataridude]

3.6.1 seems to be working on my system; I'll let it run overnight (I leave it running 24x7) and see how it's doing. It's much better now than before, so you probably got it. Thanks!

[ataridude]

3.6.1 beta still up and running overnight, and it downloaded a few shows. Looks good!

[mackworth]

I've released 3.6.1 and filed the bug w/ Apple regarding its linker. If they eventually say they've fixed it, I may ask for your help again to test a build with that system, but for now I'll close this issue.

Thanks for your help!