iTunes files disappearing from iTunes d/b, files themselves unaffected

[willson556]

From dynamite...@gmail.com on September 20, 2012 16:46:03

What steps will reproduce the problem? 1.renaming get-iplayer-automator to "iPlayer permanent downloads"(to make name more pertinent for the oldies) 2.downloading iplayer file 3.playing file in iTunes What is the expected output? What do you see instead? to keep the database intact within iTunes once a file (any TV show file) has been played, the file in iTunes d/b disappears, but file still in original folder What version of the product are you using? On what operating system? get iplayer automator v1.4.2 OS Mountain Lion, mac Please provide any additional information below. I renamed the get iplayer automator, as I have installed it on my parents' MacBook Air running 10.8;1 as the name is not intuitive for them.

Files are downloading fine, and are being added to iTunes as they should.

However, since the name change, every time I play ANY TV show (not just a TV show downloaded through the automator, but ANY TV show), the file mysteriously disappears from the iTunes d/b.

The original file remains in it's folder.

This is a problem with the automator, as all other files are unaffected - songs and movies do not self delete.

In other words, my poor parents cannot understand why the downloads - seemingly permanent - erase themselves, as they do after the 30 day limit in the iPlayer app itself.

It seems as if a simple name change has corrupted some iTunes plist (or equivalent) file.

Re-installing iTunes does not solve this problem, and my parents have no idea how to rebuild these corrupt iTunes d/b files.

Please help!

Original issue: http://code.google.com/p/get-iplayer-automator/issues/detail?id=104

[willson556]

From dinkypumpkin on September 20, 2012 11:42:11

Renaming GIA won't make any difference to iTunes. I can't duplicate the problem you described, but that doesn't mean much for such a bizarre situation. This could easily be a case of coincidence rather than causation. You've also contradicted yourself: If the source of the video files doesn't matter, why blame GIA? If iTunes is 86-ing files, the problem is almost certainly with iTunes, not GIA. If you really think a renamed GIA is the culprit, the obvious thing to do is to install a fresh copy of GIA without renaming it and see if the problem disappears. My guess is that it won't, but you can't rule anything out where iTunes is concerned. If you think you have a damaged iTunes database, there are many places on the internet to find instructions for rebuilding it. But I wouldn't start there. If the database were corrupt it seems like you would see other problems, e.g., lost links between iTunes entries and physical files.

If I understand you correctly, you're saying that when you use iTunes to play any video file classified as a "TV Show", the file completely disappears from the list of catalogued tv shows, i.e., the list you see when you click the TV Shows item under LIBRARY in the iTunes sidebar, not a playlist or any other view. If you have the "Copy files to iTunes media folder when adding to library" setting (Advanced in preferences) checked, does the copy of the the file under "~/iTunes/iTunes Media/TV Shows" disappear as well? Does it matter how long you play the video? If you drag a file into the iTunes library from Finder (instead of using GIA to add it), is the behaviour any different? When a video disappears from the iTunes catalogue, can you immediately drag the file back into the iTunes library and play it again? If so, does it then disappear again?

[willson556]

From thomas.w...@me.com on October 08, 2012 15:15:20

To clarify, GiA's involvement with iTunes simply consists of it sending an Apple Event consisting of the Add command and the location of the file. After that, it's up to iTunes to manage the file.

Status: Invalid

[willson556]

From mark.w...@gmail.com on October 22, 2012 12:39:01

I also seem to be suffering from the same file - I don't believe, however, the issue is with GiA but with iTunes - although it also could be the expiry date from the iPlayer in the metadata being followed? Does GiA get the expiry metadata from the iPlayer site?

[willson556]

From dinkypumpkin on October 22, 2012 14:59:01

Neither GiA nor get_iplayer tags files with any expiry date, and GiA never purges downloads. iTunes has no concept of expiry unless you count rentals from the iTunes Store, but that has nothing to do with GiA. The OP confused the issue by mentioning the BBC's iPlayer app - a completely different beast. He/she never followed up, so I presume whatever they experiences was some aberration unique to their system that had zero to with GiA, BBC programmes, or anything else relevant. If you can make a better description of the problem and come up with a repeatable test, feel free to post it here.

[willson556]

From mark.w...@gmail.com on October 23, 2012 08:42:24

The symptoms I see (and it only affects files inserted into iTunes by GiA and only started happening recently - around the same time the HD-SD tags started appearing on GiA items in iTunes) is as follows; (Note this is Mountain Lion latest patch release, iTunes current version on a 2011 Mac Mini Server with 16Gb RAM and 1Tb storage. Mac Mini is running 24x7 logged in with iTunes loaded 24x7 as a server. I set HD as #1 preference with Very High as second. I only download BBC. I have ITV and CH4 turned off in options. I have a cache folder in \username\movies\tv shows\ and iTunes library is on an external Thunderbolt drive)

Recent programme I noticed this on was 'Best Possible Taste - the Kenny Everett docu-drama'. So I use GiA to download a programme. Usually works OK. It seems to do the AtomicParsley bit and then loads it into iTunes. iTunes manages library and copies file into TV Shows folder under iTunes library folder as per normal.

Appears in TV shows in iTunes. All as normal.

Watch programme on Apple TV. Get, say, 60% through and stop watching. Don't delete or otherwise.

Come back to try to watch file next day, item suddently not in Apple TV. Deeper look shows it's not in iTunes either. However, checking the 'TV Shows' folder within the iTunes library folder using the finder shows the original MP4 file is still there. However, cannot re-add file into iTunes from that folder.

Workaround is to copy files out of folder to, say, desktop, and delete source iTunes TV Shows 'programme-name' subfolder using finder in iTunes library directory (not in iTunes itself) Then re-import MP4 into iTunes.

I've also had issues with some programmes (again only recently) that disappear both from iTunes library but also the folders are gone from the iTunes library folder within Finder. I've checked to see if they have been hidden or permissions are duff but nothing obvious. It really does seem as though things are just deleting themselves from iTunes for what seems like no reason.

This may not be a GiA issue and more likely something iTunes is doing with these files- however, it only ever happens with GiA files (I've 300Gb of movies I've backed up from my DVD's and they work fine) and it only started recently with the CH4 update to GiA. It's as though the files are being pruned or deleted after a period of time by iTunes. But usually iTunes will pop up and tell you when stuff has been deleted.

What does GiA do when it prompts iTunes to load in the MP4 files from the cache folder? (It never deletes the source cache btw when GiA downloads/imports and have to clean the cached directory out regularly)

[willson556]

From dinkypumpkin on October 23, 2012 16:14:28

Thanks for the detailed explanation, though it raises more questions:

1. It isn't clear when the problem first appeared. You say it started when you began seeing HD-SD icons in iTunes, but then you say it only started recently with the CH4 update to GiA. Those didn't occur at the same time. I assume you're using the latest release version 1.4.3?
2. Do you get the same behaviour with ITV programmes? If not, then that points the finger at BBC downloads rather than GiA itself.
3. Do you get the same behaviour that you download with GiA but don't first add to iTunes with GiA, i.e., you drag it to iTunes yourself? If not, you may be able to use a back door method of adding BBC downloads into iTunes
4. Do you get the same behaviour if you configure iTunes not to copy files to its media folder when they're added to the library?
5. Do files disappear if you play them in iTunes itself rather than on the Apple TV?
6. Did you download (and add to iTunes) any files with GiA after you started playing a file that later disappeared?
7. Are you downloading HD and SD versions of the same programme and adding both to iTunes?
8. Are the files that disappear from HD-SD pairs?

After testing the above, I suggest your next move should be to get my latest build of GiA and test with that. The reason is this: It may be the case, in some way I don't yet understand, that the HD-SD combo feature is somehow at fault. That feature is broken in GiA 1.4.3 because it was broken upstream in get_iplayer, so I fixed it and disabled it by default in my build. See Issue #75 for details and instructions on how to get the build. It's something of a long shot since the problem normally manifests immediately when you add files to iTunes (the files are catalogued incorrectly and appear to have disappeared). Still, it might be worth excluding that feature from the mix.

I can't duplicate the behaviour you describe, so perhaps it's limited to certain programmes. I've seen iTunes silently delete files in a couple of circumstances:

1. I think it was when iTunes 10.6 came out that some MP4 validity checks changed. Files already in the database that failed would be removed when you tried to play them or refresh metadata.
2. Occasionally when BBC downloads were dropped in the "Automatically Add to iTunes" folder and failed iTunes MP4 validity checks, they were deleted rather than being moved to rejects folder. I never got to the bottom of this, but I've also never seen the problem with iTunes 10.7 and OSX 10.8

I can't see how either of those relates to your case, though.

To answer your question, when GiA adds a file to iTunes, it just makes a few calls to the iTunes scripting interface to add the file to the library and write a basic metadata tags. iTunes does all the work (e.g., copying the file to its library), which is why I don't think GiA itself is at fault here, barring the possible exception mentioned above.

[willson556]

From djgilm...@gmail.com on October 28, 2012 10:20:52

I have been having similar problems and from what I have read here this is an iTunes issue – it appears to be a GIA issue because GIA is making the most additions to my iTunes collections.

Starting with an empty iTunes library and adding GIA-downloaded programs one by one ... suddenly the one I add merges with another one. The textual meta-data remains correct but the video that plays is a different one. What I believe is happening is that iTunes is deciding that one is the HD version of the other and is merging them into one library reference but two files.

What I cannot work out is how iTunes is deciding that they are the same. I have tested it enough to know that the files that get confused are not always the same. Thus adding program C to my library will merge with Program A the first time, but with program B on a second attempt.

I hope this helps us all work out what is going on here. I'll add more if I work out more.

[willson556]

From djgilm...@gmail.com on October 28, 2012 10:34:17

Apologies for a quick second post – but if you right-click one of these troublesome files in iTunes (they have the legend HD-SD beside them) then you can choose which version plays and it is possible to see that what iTunes thinks are two versions of the same thing are in fact completely different programs.

Now we just need to work out what makes iTunes think they're the same and work out how to separate them.

[willson556]

From dinkypumpkin on October 28, 2012 11:27:40

That is almost certainly the problem discussed in Issue #75 (see comment 6 above). The OP never established if that was the problem he encountered, but your description sounds spot on. See Issue #75 for instructions on how to get my GiA build, which has a fix/workaround for the problem.