Allow NFO file to override information from scanner

[Omertron]

Original issue 101 created by Omertron on 2008-07-13T06:59:44.000Z:

What steps will reproduce the problem?

1. Running the tool after adding media.

What is the expected output? What do you see instead?

2. Expected: Appropriate navigation for details pages. Actual result: Instead you get appropriate navigation for when the data was originally scanned, it doesn't change to reflect the addition of media.

What version of the product are you using? On what operating system? All versions including 1.0.10 beta

Please provide any additional information below.

By default the XML files are created once, meaning by default they don't get over ridden on repeated execution of the tool. Using a collection containing MovieA, MovieB, MovieD, MovieE as an example the XML for MovieB will have the following XML entries used for navigation in the generated HTML file.

<first>MovieA</first> <previous>MovieA</previous> <next>MovieD</next> <last>MovieE</last>

With the default setting of 'mjb.forceXMLOverwrite=false' this XML file will never change. The additions of MovieC and MovieF should change the above to reflect MovieC for next and MovieF for last, but they won't without rewriting the XML and HTML files.

The problem with setting 'mjb.forceXMLOverwrite=true' is that currently not overwriting those files is the only way to preserve changes made to them like adding plots for movies that couldn't retrieve one from IMDB, or correcting titleSort information on occasion to help it sort correctly .

I'm not sure what the best way to fix this is. Rewriting all that data on every execution is going to increase the time it takes to run the process each time. Is it worth the time to ensure navigation integrity on a page some people may not have realized you could even navigate from? Personally I use it and would sacrifice a longer execution to get correct navigation.

Is there a way to take the preexisting xml, preserve data previously stripped from IMDB/mediainfo (not re retrieving it, saving time), yet updating those xml entries that would change with the addition of media, like navigation?

Alternatively, and I've seen this brought up in the past on the forum thread, add to what can be stripped from the nfo file. Not sure why this wouldn't work. We know the tool can search a lengthly nfo containing all kinds of info for the imdb information. Why not add to that.

Examples: plot: 'Rat takes over the world, cooks spaghetti. ' titleSort: 'Ratatat Tat' season: '1' country: 'Vietnam' genre: 'Documentary'

Anything thats in the XML could be over rid via nfo. Ultimately I'd like to maintain the integrity of the navigation, while being able to manipulate data so as to add or change data stripped form imdb/mediainfo. At this time we have to choose one or the other.

An advantage to using an nfo to over ride anything that might be picked up by imdb/mediainfo is that when changing versions, from v1.0.9 to v1.0.10 to v1.0.11, etc ... the nfo doesn't change as it resides in the movie folder not the Jukebox data folder.

[Omertron]

Comment #1 originally posted by Omertron on 2008-07-14T20:13:50.000Z:

I'm investigating this as a possible enhancement.

[Omertron]

Comment #2 originally posted by Omertron on 2008-07-17T16:10:55.000Z:

I think we need 2 XML files for each movie: The first contains the movie info obtained from the internet source (imdb, etc). The second is generated from the first and is used only for generating the detail HTML file, and would contain additional information such as navigation (including next/prev) URLs. This second XML file (and associated HTML file) would have to be regenerated on every run in order to make sure that all URLs are accurate as the library changes.

The XML files should be kept only on local disk, seperate from the HTML files, which could be stored on the network.

[Omertron]

Comment #3 originally posted by Omertron on 2008-07-17T18:51:02.000Z:

The requirement so far is that there is a need to override the details that are brought back from scanners (e.g. IMDB) or derived from the filename.

Within the "nfo" file there should be the option to override each of these details:

• ScannerId = The IMDB title (used instead of the full url)
• Title = The display title of the film (as shown on the index page
• TitleSort = Version of the title for sorting and indexing purposes
• Year = Filming year of the movie
• ReleaseDate = Release year of the movie
• Rating = Rating of the movie, e.g. 5.9/10
• Poster_URL = Alternative poster url, instead of the scanner derived one
• Plot = Summary of the plot
• Country = Country of origin
• Language = Language of the film (used to determine the flag)
• Company = Production company of the film
• Runtime = Length of the film
• Certification = MPAA or BBFC rating for the movie (18, 12, PG, U, G, etc)
• Season = -1 indicates no season, 0 indicates "Specials", otherwise season number
• Genre = Genre of the film, multiple allowed
• Actors = Actors in the film, multiple allowed
• Director = Directors of the film, multiple allowed
• File = Files that make up the film, multiple allowed

Is there anything else that you want included?

[Omertron]

Comment #4 originally posted by Omertron on 2008-07-17T21:14:20.000Z:

Stuart.Boston - This is a very nice idea....

[Omertron]

Comment #5 originally posted by Omertron on 2008-07-18T07:57:40.000Z:

This is exactly what I was looking for a while back - I wanted the info to be with the movie file, not in a horrible .xml file in a Jukebox folder somewhere.

[Omertron]

Comment #6 originally posted by Omertron on 2008-07-18T15:57:15.000Z:

It can be usefull to use multiple "ScannerId" related to different MovieDatabase. For example in the AllocinePlugin I developed I use primarily a allocineID but if I can't find any movie infos using it I fallBack to ImdbID. I updated the Movie object in order to be able to store multiple id couples [("Imdb","tt12345"),("Allocine","34258")].

So the "nfo" should be able to contain multiple ScannerId for example like this : ScannerId = Imdb:tt12345 ScannerId = Allocine:34258

or ScannerId = Imdb:tt12345,Allocine:34258

[Omertron]

Comment #7 originally posted by Omertron on 2008-07-19T02:45:59.000Z:

I'd like to be able to sort movie series in chronological order, but movie titles don't always allow for that. What I'd like to see is a way to force a sort for a series of movies. If we added 'SeriesName', 'SeriesOrder', 'SeriesTitle' this could allow for forced sorting of titles.

I had an issue with the sorting of Indiana Jones. The first movie's official title is 'Raiders of the Lost Ark'. Which sorted it under 'R'. The other two were 'Indiana Jones and the Temple of Doom' and 'Indiana Jones and the Last Crusade', which sorted backwards.

What I'd like to see is something like this ...

• SeriesName = Series movie belongs too, and primary sort item.
• SeriesOrder = 1,2,3, etc. Secondary sort item used for sorting but ignored beyond sorting purposes.
• SeriesTitle = Movie title, ignored for sort.

Series determines where it sorts ("I" for the Indiana Jones series), SeriesOrder determines the sort order within the series (1,2,3), and title is title.

Indiana Jones 1 Indiana Jones 2 Indiana Jones 3

Would allow sort to override alphabetical sort. It could use 'Series' and 'SeriesOrder' for navigation, and 'Series' + 'Title' for HTML text (replacing what IMDB provides).

Currently I've been altering the titleSort in xml to achieve this ... Indiana Jones [1] Raiders of the Lost Ark Indiana Jones [2] and the Temple of Doom Indiana Jones [3] and the Last Crusade ... but having the [#] in the HTML is UUUUUUUGLY.

I believe this type of sort function override would be perfect for sorting movie series like Indiana Jones, James Bond, Batman, and Die Hard where the movies otherwise wouldn't necessarily sort chronologically.

[Omertron]

Comment #8 originally posted by Omertron on 2008-07-19T02:53:20.000Z:

James Bond 03 James Bond 04 James Bond 13

Navigation and sorting would be based on James Bond 03 James Bond 04 James Bond 13

Titles in HTML would appear as James Bond: Goldfinger James Bond: Thunderball James Bond: Octopussy

[Omertron]

Comment #9 originally posted by Omertron on 2008-07-19T02:59:12.000Z:

I'd like to be able to sort movie series in chronological order, but movie titles don't always allow for that. What I'd like to see is a way to force a sort for a series of movies. If we added 'SeriesName', 'SeriesOrder', 'SeriesTitle' this could allow for forced sorting of titles.

I had an issue with the sorting of Indiana Jones. The first movie's official title is 'Raiders of the Lost Ark'. Which sorted it under 'R'. The other two were 'Indiana Jones and the Temple of Doom' and 'Indiana Jones and the Last Crusade', which sorted backwards.

What I'd like to see is something like this ...

• SeriesName = Series movie belongs too, and primary sort item.
• SeriesOrder = 1,2,3, etc. Secondary sort item used for sorting but ignored beyond sorting purposes.
• SeriesTitle = Movie title, ignored for sort.

SeriesName determines where it sorts ("I" for the Indiana Jones series), SeriesOrder determines the sort order within the series (1,2,3), and SeriesTitle is title.

Indiana Jones 1 : Raiders of the Lost Ark Indiana Jones 2 and the Temple of Doom Indiana Jones 3 and the Last Crusade

Would allow sort to override alphabetical sort. It could use 'SeriesName' and 'SeriesOrder' for navigation, and 'SeriesName' + 'SeriesTitle' for HTML text (replacing what IMDB provides).

Currently I've been altering the titleSort in xml to achieve this ... Indiana Jones [1] Raiders of the Lost Ark Indiana Jones [2] and the Temple of Doom Indiana Jones [3] and the Last Crusade ... but having the [#] in the HTML is UUUUUUUGLY.

I believe this type of sort function override would be perfect for sorting movie series like Indiana Jones, James Bond, Batman, and Die Hard where the movies otherwise wouldn't necessarily sort chronologically.

+++++++++++++++++++++++++++++++++++++++++++

James Bond 03 : Goldfinger James Bond 04 : Thunderball James Bond 13 : Octopussy

Navigation and sorting would be based on James Bond 03 James Bond 04 James Bond 13

Titles in HTML would appear as James Bond: Goldfinger James Bond: Thunderball James Bond: Octopussy

[Omertron]

Comment #10 originally posted by Omertron on 2008-07-22T04:48:49.000Z:

Thank You... someone else is thinking the same way. First off, another XML file is not the way to go. Use the NFO file for all the extra info to override the XML settings. I should be able to delete the whole JukeBox directory and not lose anything but time in regenerating everything.

I've been spending the last few days with the program to parse the whole internal 750GB HDD, mostly kids TV shows, DVD movies, 720p Movies, and Daddy's 720p TV shows.

The problems I encountered were some wrong IMDB lookups, but many bad pictures with mostly kids TV shows. I've taken the time time to look for new poster images, or better ones than the default found. I may also edit some images myself and so I want those used instead. I don't want to manually edit XML files, only look in there to see what the fields are to override... and put them in the NFO file.

I think something simple like... XML-tag: value (name with a ":" and then the value) Ex: http://name....img.jpg would be PosterURL: http://name....img.jpg etc And the image link should work for a local file: file://name...img.jpg

Side note: One thing to bring up... as well as NFO associated with the movie, perhaps MovieJukebox should look for movie.jpg in the same place and use that... no lookups. I will be adding this as an issue to better explain.

This will also allow me to use my Movie Collector images that I already and have tweaked.

Having the ability to override would be very helpful. I also ran into the same sorting problem with the Indiana Jones trilogy. I would also want to override which genre category they belong to, as well as the rating.

I found a BIG problem with all my stuff being mixed with the kids stuff. I want to be able to have TV-Kids and TV-Other, also Movie-Kids, DVD-Kids, etc. So I think the only way to do that is tweaking their category or creating one of my own.

Looking forward to seeing this one done. :D

[Omertron]

Comment #11 originally posted by Omertron on 2008-07-22T17:54:50.000Z:

Would it be possible to include resolution as one of the details that are overridden? For movies that are rar:ed resolution is marked as UNKNOWN and they are not properly tagged as HD. Providing the resolution in the NFO file would fix that.

[Omertron]

Comment #12 originally posted by Omertron on 2008-07-22T18:17:30.000Z:

This issue is supposed to be about the problem where the prev/next and first/last navigation links in the detail pages get stale because they aren't updated every time MJB is run. All this discussion about overriding data in the XML files is totally unrelated and simply doesn't belong here. Use another issue for that.

[Omertron]

Comment #13 originally posted by Omertron on 2008-07-22T18:35:16.000Z:

Actually, that's my fault partially. The original request was for YAMJ to be able to propagate the next/prev properly after the xml had been edited/changed after new movies had been added. About 1/2 way down the OP's solution is to allow the nfo file to have more information sources.

[Omertron]

Comment #14 originally posted by Omertron on 2008-07-22T22:06:27.000Z:

Sorry about that, I was fooled by the name of the issue that actually is "Allow NFO file to override information from scanner". Didn't realize that this was actually a solution to a completely other problem. Seems that the two are closely related though. I'll hang back and see what happens and move my comment to another issue if necessary.

[Omertron]

Comment #15 originally posted by Omertron on 2008-07-31T14:27:36.000Z:

I've added the specification document for this issue. Hopefully one of the developers will pick this up now.

[Omertron]

Comment #16 originally posted by Omertron on 2008-08-01T04:48:23.000Z:

Stuart, nice work.

I agree with the majority except for how to define the multiple fields.

Brad Pitt Sharon Stone

This is XML code and not user friendly. The whole point should be that average users open up Wordpad and make simple, quick changes. This is programmer-thinking and not user-thinking... and yes I have years of programming. The computer is supposed to do the work, not the user. So IMHO, we should only use a simple style like this: Actors: Brad Pitt, Sharon Stone where extra white space is ignored (or stipped) before and after names but comma is the delimeter.

All other tags should also use a simple Name: style followed by the info. For consistency, the same names could be used as are normally found in the XML file.

One thing I don't see addressed is the possibility to have a season NFO file as a kind of master for all episodes of that TV series for that season. Example: You only need the IMDB link, Plot and Poster once per season. I hate the idea of having an NFO file for each episode. Right now only the first episode NFO file is used but what happens after you've watched that and deleted it? The next ep has no NFO file! I believe that once an episode is found, say Dexter 1x01, and knows it's a TV series, it should first look for Dexter s01.nfo and then Dexter 1x01.nfo, etc.

[Omertron]

Comment #17 originally posted by Omertron on 2008-08-01T06:35:25.000Z:

Thanks for the feedback, I've edited the document and uploaded a version with the comma delimited fields as a suggestion.

As to the seasons for TV shows, that's another issue and plan to address it after my holiday.

[Omertron]

Comment #18 originally posted by Omertron on 2008-08-19T07:45:48.000Z:

Allow NFO file to override information from scanner

• Love the work to date. Nothing I can add accept maybe a tool to auto-generate the override file. If there was a GUI to generate these files then the output wouldn't matter. NFO or XML is pretty much the same once you learn how to read an XML file.

If someone was going to write a front end to generate the override files, the ability to select a few different videos and bulk generate the fields with common information would be nice. i.e. a sporting event with 6 different files. You could create 6 files with core info then add the individual info, like pregame, post game etc, seperately.

[Omertron]

Comment #19 originally posted by Omertron on 2008-09-10T08:44:16.000Z:

Really needed!

[Omertron]

Comment #20 originally posted by Omertron on 2008-09-25T03:00:12.000Z:

Please take a look at this page http://xbmc.org/wiki/?title=Import_-_Export_Library for the XBMC project's use of NFO files.

It would be great to somehow incorporate many of the same fields for the YAMJ project. Would also be nice for anyone that also uses an XBOX to be able to use the same NFO files for both (or an easy conversion program XMBC -> YAMJ, and YAMJ to XBMC.

[Omertron]

Comment #21 originally posted by Omertron on 2008-10-09T16:43:13.000Z:

I'll look at updating the requirements doc over the next couple of days

[Omertron]

Comment #22 originally posted by Omertron on 2008-10-13T08:39:59.000Z:

Now that i looked at xbmc documentation i think that nfo xml check and use if valid format or otherwise use normal imdb url parsing method would be good.

[Omertron]

Comment #23 originally posted by Omertron on 2008-10-25T11:27:35.000Z:

+1

[Omertron]

Comment #24 originally posted by Omertron on 2008-10-28T03:14:15.000Z:

Ok, we'll be using the XBMC nfo format as a base. There is additional requirements we'll be adding to that, such as various image locations. It was mentioned by someone previously that an xml file is not the way to go. However, consensus says that it is the way to go, and the XBMC nfo file is in fact an xml file. We will be following this standard, and building upon it.

[Omertron]

Comment #25 originally posted by Omertron on 2008-10-28T03:42:20.000Z:

Look forward to seeing how this progresses along.

[Omertron]

Comment #26 originally posted by Omertron on 2008-10-28T18:31:30.000Z:

I'd just like to toss out the suggestion of being able to use local paths in this proposed nfo file.

For instance, say someone has already downloaded fanart (or a cover image) for The invincibles...instead of providing the url to the image on the internet, it'd be nice if they could just put in the local path and file name into the or

So if possible i'd propose that YAMJ have the ability to take both a URL there and a local path/filename.

[Omertron]

Comment #27 originally posted by Omertron on 2008-10-28T18:42:21.000Z:

Makes perfect sense...will do.

[Omertron]

Comment #28 originally posted by Omertron on 2008-10-29T08:23:01.000Z:

If we will use XML files as NFO files, maybe XML files genrated by YAMJ should be the same as NFO files? Then users could copy them from output to input directory.

[Omertron]

Comment #29 originally posted by Omertron on 2008-10-29T14:45:10.000Z:

That's part of the discussion that's going on. Whether to adopt the XBMC nfo layout for cross-compatibility and tools usage, or to go with our home-brew solution (like our current movie xml files). Pros and cons for each.

[Omertron]

Comment #30 originally posted by Omertron on 2008-11-02T20:02:52.000Z:

This is implemented with the SVN r371 version and greater.

Utilizes the unmodified XBMC movie layout as specified here http://xbmc.org/wiki/?title=Import_-_Export_Library#Video_nfo_Files The tvshow, episodedetails, & musicvideo layouts are not currently implemented, and will require additional internet database scanning features to be implemented first.

Some fields map directly to YAMJ, and others do not currently have a YAMJ counterpart. These fields are commented on below.

Any fields may be populated or left blank as the user sees fit. If a field is populated, then it will take precedence over anything retrieved from IMDB. However, if a field is left blank in the NFO, then that field will still be loaded from IMDB as before.

On fields where multiple may exist, like or , if even one is present in the NFO, then YAMJ will not attempt to look up additional from IMDB.

// currently unused in YAMJ // 0 - 10 rating, can be decimal // currently unused in YAMJ // currently unused in YAMJ // currently unused in YAMJ // currently unused in YAMJ // url of poster image. use URL formatting, such as http:// for internet resources or file:// for local resources // currently unused in YAMJ // currently unused in YAMJ // the IMDB id of the movie. includes the leading "tt" // currently unused since YAMJ determines path from searching the configured libraries // multiple trailer records may exist // multiple genre records may exist, including any custom ones // currently unused in YAMJ // multiple actor records may exist Paul Begala Himself // currently unused in YAMJ

[Omertron]

Comment #31 originally posted by Omertron on 2008-11-02T20:10:52.000Z:

wuhuu! Thank you SO much!

[Omertron]

Comment #32 originally posted by Omertron on 2008-11-03T03:27:56.000Z:

Thanks :D

[Omertron]

Comment #33 originally posted by Omertron on 2008-11-03T03:52:05.000Z:

Which field maps to titleSort? Maybe nfo:originaltitle should map to yamj:title and nfo:title should map to yamj:titleSort

[Omertron]

Comment #34 originally posted by Omertron on 2008-11-03T05:51:09.000Z:

The XBMC layout does not have a direct mapping for YAMJ's titlesort concept.

What you suggest can certainly be opened up for discussion. I was under the impression that the originaltitle field had more to do with the movie's original language title vs the language specific titles. But since it's not currently mapped to anything, it may make sense.

[Omertron]

Comment #35 originally posted by Omertron on 2008-11-03T10:38:31.000Z:

I think originaltitle is useful for non-english users. Maybe a new sorttitle should be used for that, and concatenate title and (originaltitle) for longtitle in yamj (just my idea)

[Omertron]

Comment #36 originally posted by Omertron on 2008-11-03T10:49:16.000Z:

I agree; originaltitle uses to be the title in original language when applies and it would be useful for people watching foreign films; Maybe the way to apply originaltitle should be more like concatenating both of them into YAMJ title, and add a field for short title.

[Omertron]

Comment #37 originally posted by Omertron on 2008-11-04T07:32:16.000Z:

I think it should be used for the original non-english title.

It would also go nicely with comment # 5 I made on Issue 246 about this exact scenario.

[Omertron]

Comment #38 originally posted by Omertron on 2008-11-06T17:03:44.000Z: