search

XMLTV / xmltv

Utilities to obtain, generate, and post-process TV listings data in XMLTV format
GNU General Public License v2.0
275 stars 93 forks source link

tv_grab_zz_sdjson is not XMLTV DTD compliant for new and previously-shown #91

Closed garybuhrmaster closed 4 years ago

garybuhrmaster commented 4 years ago

tv_grab_zz_sdjson is not XMLTV DTD compliant for new and previously-shown

Per discussions on the xmltv-devel list (starting on 2020-04-13, subject "Request for clarification"), it was agreed by the project elders that program episodes (other than the first of the first) that are new cannot have a previously shown, and that new is only for true first showings of the first episode (there are always edge cases, but the generic new episode is not new). The source guide data for the tv_grab_zz_sdjson grabber is from Schedules Direct, and while that source has an element named new, that indicates that this is first showing of the episode (i.e. not a rerun), but does not represent the XMLTV definition of new, however, does indicate that the program cannot have had a previously shown.

A pull request will be submitted with proposed fixes to be more XMLTV DTD compliance.

garybuhrmaster commented 4 years ago

FWIW, I have reviewed the output of the tv_grab_zz_sdjson grabber with the proposed patch and it appears to more accurately follow the XMLTV DTD.

lordtaytertot commented 4 years ago

Gary,

Even with your patch, I'm still getting the following error when I run mythfilldatabase:

Malformed XML file, program out of order at line 546109

(which is the last line in the file it pulls down)

Can you confirm that it's working for you?

garybuhrmaster commented 4 years ago

This patch has nothing to do with that error, which is a completely different bug in MythTV itself. MythTV commit a9aa006139da24cbad861a2c25d57fa3f71aba2a (cherry picked as f5d75a6de7c4ac668e1a64cdf31c7100bc81b65b into the 31/fixes branch) added in a validation requirement (that programs must appear in start time order) which does not appear to be in the formal XMLTV DTD (although I would not be surprised that many grabbers ordered programs as such), and was never previously validated by mythfilldatabase before that commit.

Note that whether one gets that error message will likely depend on the specific set of stations and days are in your lineup (my guess is that some of the time the output is ordered appropriately, and just works).

You will need to open a ticket in the MythTV project and explain the issue. Until that is resolved, you can choose to build MythTV without that commit, or you can choose to use the alternative SD-JSON grabber which does happen to order the output by starttime for its own purposes.

rmeden commented 4 years ago

you can probably run the output through TV_SORT too.

lordtaytertot commented 4 years ago

I think the easiest workaround is to set the maximum days to 20. I have a suspicion (without combing through massive XML files) that imperfect data sets on day 21 was contributing to this problem. So far it's working for me. Thank you guys for the support though, it's greatly appreciated!

garybuhrmaster commented 4 years ago

I think the easiest workaround is to set the maximum days to 20. I have a suspicion (without combing through massive XML files) that imperfect data sets on day 21 was contributing to this problem. So far it's working for me. Thank you guys for the support though, it's greatly appreciated!

FWIW, I opened a ticket with MythTV for their lack of compliance and my fix has been applied to their code base.