Support EDL files for Recorded TV

[djp952]

Support loading of EDL (Edit Decision List) files for Recorded TV. Files generated by ComSkip, for example.

Challenges:

• Need to parse the file; there are various formats from various vendors
• Need a way to associate the recording with the file. Comskip, for example, names the output file to match the input .MPG file name. That file name is not accessible from the PVR client, only the metadata provided by the HDHomeRun RECORD engine. The file name will have to match something available in the metadata.
• Tools might provide frame numbers, these need to be mapped into milliseconds for Kodi as well as what should be done (CUT, MUTE, SCENE, COMBREAK)
• Comskip itself provides an easy to parse .edl file that has the necessary millisecond data rather than frame rates, better choice than the .txt file.

Design:

• Add an option "Enable Edit Decision List (EDL) processing" (default: false)
• Add an option "EDL input file location", selectable from existing libraries. User will have to add a network share via Videos or Music first due to UI limitations.
• Add an option "EDL input file format", toggles for Comskip et al, supporting multiple formats automatically is probably not worth the effort
• Implement PVR "GetRecordingEDL" function
• GetRecordingEDL: -- uses available PVR_RECORDING to access metadata to generate file name to look for -- if file is found, load and parse based on selected file format -- populate PVR_EDL_ENTRY[]/count and return to Kodi

References:

• Kodi already supports this for Video library files, there should be reference code for all of the file formats supported by Kodi, likely to the same millisecond/action format expected by the PVR subsystem.

[djp952]

FYI, I've created a temporary branch named "Krypton-EDL" and checked in stubs to begin implementing the GetRecordingEdl() function. I put a couple placeholder settings under the "Advanced" section as well. Might need a third setting to define the file name format, but need to play around to see what can be done first.

[djp952]

It looks like the way Kodi does this can be duplicated rather easily so I'll add that into the Krypton-EDL branch. They basically go through the various file formats they support until one works, that seems reasonable and easy to expand later. I'll probably start with Comskip EDL and Comskip TXT. Still haven't looked into how the PVR will find the file, I'd hate to make people rename them, but there may be enough metadata in the local database to reconstruct the actual file name. Honestly haven't looked yet.

I think this will be a relatively painless addition overall.

[hunsra]

I was looking at this recently. It appears the RECORD service names recorded files using the following properties (as named in the JSON returned by the RECORD service):

{Title} {EpisodeNumber} [{StartTime}].mpg

Where {StartTime} is formatted YYYYMMDD-hhmm

Comskip uses the same base name and changes the extension to .edl or .txt depending on the format chosen. It seems fairly trival to construct the Comskip filename from these property values.

Also, I've been working on a service to manage running Comskip on a set of watched folders. My plan is to have it run Comskip on new recordings is discovers. The watched folder(s) and output folder(s) will be configurable. If possible, I'll also be adding a REST API surface to it so a caller can discover the service, as well as get EDL files for a particular show, by base filename, or by the properties of the show, as described above. As soon as I have something, I'll share it with you so you can test it out. I'm trying to make it cross-platform, supporting Windows, Linux, FreeBSD, and MacOS. I may need some pointers on how to do that well. If I can't pull off the REST API, I mght just share the output folder so files can be directly accessed by callers, such as the DVR add-on.

[djp952]

Thanks! I figured out a couple other details as well. Looks like the folder name is the DisplayGroupTitle (like 'Movies', but typically the series name), the folder and file name portion has to have invalid file name characters stripped from it, and the EpisodeNumber/OriginalAirDate field needs to be optional. I wrote a SQLite scalar function to strip the invalid chars (not checked in yet), but here is the query I'm working from (below) if you'd like to look it over. I will take the output from this and compare it to all of my recorded files to make sure I got all the pieces right for the files I currently have.

{DisplayGroupTitle}/{Title} {EpisodeNumber} {OriginalAirDate} [{StartTime}]

select 
clean_filename(json_extract(value, '$.DisplayGroupTitle')) || '/' || 
clean_filename(json_extract(value, '$.Title')) || ' ' || 
coalesce(json_extract(value, '$.EpisodeNumber') || ' ', '') ||
coalesce(strftime('%Y%m%d', datetime(json_extract(value, '$.OriginalAirdate'), 'unixepoch')) || ' ', '') || 
'[' || strftime('%Y%m%d-%H%M', datetime(json_extract(value, '$.StartTime'), 'unixepoch')) || ']' as filename 
from recording, json_each(recording.data)
order by filename asc

Here are some samples from the output that dealt with the optionals and invalid chars properly so far. Excuse My Little Pony -- it has a colon in the title. My daughter loves this show :)

Movies/A Nutcracker Christmas 20160101 [20170709-0300] Movies/Lee Daniels' The Butler 20131115 [20171106-0400] Movies/Tangled Before Ever After 20170101 [20170311-1800] My Little Pony Friendship Is Magic/My Little Pony Friendship Is Magic 20171007 [20171007-1530] My Little Pony Friendship Is Magic/My Little Pony Friendship Is Magic 20171014 [20171014-1530] My Little Pony Friendship Is Magic/My Little Pony Friendship Is Magic S01E01 20101010 [20170515-1530] My Little Pony Friendship Is Magic/My Little Pony Friendship Is Magic S01E02 20101022 [20170516-1430]

So far so good, once it's checked in and working I'll probably ask if you could run a SQLite query against your database to see if all your file names are correct as well, if you don't mind. Pizza and movie night here, but I should have some time later.

[hunsra]

Excellent! Yeah, I noticed after my earlier post that OriginalAirDate is also sometimes present, and that the folder name seems to match DisplayGroupTitle.

Looking forward to the changes and testing them out.

My daughter liked that show too. She probably still does, but she wouldn't admit it. Enjoy the pizza!

[djp952]

Support for Comskip .EDL files has been put into the main Krypton branch for testing. It looks like I won't be able to do many of the file formats Kodi supports as I don't know what the frame rate of the recording is. The other format that could be added relatively easily is the BeyondTV XML format.

Interestingly, Kodi knows the frame rate for this calculation before it asks the PVR, so maybe I'll add a pull request for them for Leia to send that along to the PVR. The recorded file has already been opened when Kodi asks for the EDL, so while it seems like overkill in theory it would be possible to check the lead-in metadata packets HDHomeRun RECORD sends on the stream. If Kodi has already done the work, it seems silly not to just pass it on to the PVR, though.

In playing with it, I added 2 options as well - start and end time padding. I wasn't super impressed with what Comskip was coming up with for me, it was too aggressive. These settings default to zero but can be adjusted from 0ms to 10000ms. I account for the overlap potential in the code.

I was going to add a third option to convert CUTs (0) into COMBREAKs (3), but Kodi didn't seem to do anything at all for COMBREAKs like it was supposed to. I read that you should be able to skip backwards into a COMBREAK as opposed to a CUT, but since it didn't work I left that option out. Another thing to probably look for in Kodi, COMBREAKs were invented by them :)

Anyway, if you pull and build Krypton, I'd like to get your opinion on the functionality. Only supporting .EDL files with time stamps is more limiting than I think we wanted, but not knowing the frame rate at the time the EDL is loaded is quite limiting.

I'll be deleting the Krypton-EDL branch, everything in there has been merged into Krypton at this point.

[hunsra]

Great work! I'll check it out and get some feedback to you ASAP. Thanks!

[hunsra]

Sorry for the delay in responding. I've tried out the new EDL functionality, and it appears to work, but there might be an issue with fast forward/rewind around skip locations. I haven't had much time to test it, nor have I had time to finish up my comskip manager service. I'll get you some more feedback and detail on the potential issues as soon as I can, probably a week or two out. Thanks.

[djp952]

No worries; that seems to be the way it works in Kodi. Anytime you try to seek into a skipped region, either forward or backward, Kodi skips forward to the end of that region. If the documentation is accurate, there is supposed to be a "Commercial Break" type of skip (3) that would allow you to seek backwards into the region successfully, but it doesn't appear to make any difference.

Is that what you were seeing, or something different? We seem to be limited to just telling Kodi the start/end times of the region to skip over and it handles all the seek operations on its own.

[djp952]

I think this issue is pretty much dead at this point. Insofar as I know the EDL additions work as well as they can, and honestly I never ran into much luck with comskip. I had a much better experience taking the time to manually edit the MPEG files in a video editor :)