Recorded Show Progress

[hunsra]

Is it possible to synchronize recorded show viewing progress across multiple playback devices? I use MySQL to centralize Kodi's libraries and viewing progress of library content. It would be great to have the same for recorded content as well.

[djp952]

I've thought about this one and if you'll excuse my typical verboseness ... I think we can do something here :)

I was under the impression Kodi took care of this if the PVR client didn't, but there appears to be some restrictions and gotchas that prevent it from working all the time. The main thing seems to be that all clients have to be running the exact same Kodi version, if you upgrade from say 17.5 to 17.6, it goes away (this explains why that info disappears on just one PC all the time). I didn't get too deep into it, but deep enough to see that it would indeed be better for the PVR client to do it. My database schema is kept the same for each x.x version, so it would stay compatible across Kodi releases and even across Kodi baselines (Jarvis/Krypton/Leia/etc).

I'm on board, here's what I propose. I would need a schema change to the database, so we'd bump up to version 1.3.x, no problem there. I can start by adding the necessary support for the local single PC database and get that working acceptably, and then add a way to share the database across all your Kodis.

I have two things to work out:

1) Where to put it. Using the HDHomeRun share that the DVR engine creates seems like the best place, but the location really shouldn't matter as long as it's read/write. Kodi doesn't give access to the cool folder browser they have for things like the Video library so it may start out as something you'd have to type in or select from an existing known network share. I have a second PVR that does this, it's a bit annoying to have to go into Videos and add a network share then back out to select it, but it works.

2) Synchronizing the updates from multiple clients. This may not be a problem, the SQLite database is pretty robust, I'm thinking about the wastefulness of downloading and updating the same data from multiple clients every XX minutes. Some of it takes a while, specifically the episode information, but even the fast stuff like tuner and recordings discovery becomes redundant pretty quick.

I don't see either of these as major concerns and they can be hobbled together incrementally as well to improve them over time. We can get the basic functionality working the way we think is best and then smooth out the nit-picks.

So yeah, let's do this thing. I like it! I'll keep you apprised of progress and when I have pieces that appear to work will make builds available :)

[djp952]

So I've run into a bit of a snag here. SQLite (the database engine) is pretty vehement that you should never ever try to share a database file over the network. There are a lot of potential problems in heterogeneous client/server scenarios. This might have to ultimately piggy-back on putting information into MySQL like Kodi does, or it may be a show-stopper (no pun intended).

I was wondering, though, have you tried adding the PVR database to the list of things Kodi shares? Check out this link:

http://kodi.wiki/view/Advancedsettings.xml#musicdatabase.2Fvideodatabase

They claim you CAN share the PVR database if you want, but they don't recommend it. Even if it works, we are still stuck with the "same exact version of Kodi" problem to overcome. I'd be interested to know if that works to resolve the concern on it's own, however, as it means storing this metadata locally has no benefit, and actually becomes a detriment since Kodi will favor what the PVR client says.

Since the base support in this code's database is ready and working I may add a "store recording metadata in discovery database" option to control it and not make it mandatory. This would become v1.3.0 after I look into a couple tweaks on the Leia baseline.

I'll have to look more into the sharing options here, moving the info to MySQL may not be so bad but probably a lot of work, which takes time. The extreme case would be a new service that runs alongside the DVR engine, but if that's necessary we're better off asking SiliconDust to add a way to track this on the DVR engine itself ...

more info as it becomes available ...

[djp952]

WAIT - hold the phones on all of this. The DVR engine does support this function. I just have to learn how to set it.

I watched some recorded shows through the HDHomeRun app, and it stored a "Resume" value that can be retrieved that is compatible with Kodi. Problem will equal solved. Give me some time to figure that one out.

Ugh.

[hunsra]

I added the <tvdatabase> and <epgdatabase> elements to the advancedsettings.xml file as described in the wiki page on two of my kodi devices. The playback progress did not seem to get synchronized between them. I can start the same video on each device and neither has the other device's last played position - only their own. I looked at the MySQL database after setting this up and it did indeed add a new database named "tv29". I tcreated four tables, named as follows: "channelgroups", "channels", "map_channelgroups_channels", and "version". There doesn't seem to be a table for playback progress.

So, adding those elements to the advancedsettings.xml file doesn't seem to add the functionality I was looking for. Too bad :-( I was hoping it would.

[djp952]

Never fear my friend, I figured out how the HDHomeRun app was doing this, and I've got it implemented. All that remains is to test it out a few times and let it rip for you guys to play with :)

Since the HDHomeRun RECORD engine implements this, the impact to the PVR client was very small, so it will be version 1.2.5, target release date is today (11/28/17). Bonus -- the HDHomeRun app is now synchronized with Kodi, so the resume time works between them now.

This was a great enhancement, thank you so much for bringing it up. Once I get 1.2.5 out the door let me know what you think!

[hunsra]

Awesome! Looking forward to checking it out!

[hunsra]

Just updated the add-on to version 1.2.5 on kodi 17.6 on two of my devices. Unfortunately, I'm not seeing any synchronization between them. Do I need to do something to enable the new feature?

[djp952]

You shouldn't have to do anything special, just play a recording on one Kodi, stop it, then from the Recorded TV area of another Kodi start it back up again. Should get the prompt to restart at xx:xx.

Hmmm... maybe I didn't mention that it can't "sync" old information in and of itself, to get the Resume flag set on the RECORD engine you'd have to play it first. I don't know of any way to extract all the current resume points from Kodi and apply them to the record engine. Was that what you were expecting?

I've tried it on 6 Kodis here, all seemed OK to me. But that's all for newly played recordings after 1.2.5 (or recordings played from the HDHomeRun Windows Store App)

[hunsra]

I wasn't really expecting it to extract current resume points, just remember them going forward. I did a bit more testing, including using the HDHomeRun Windows application and saw some strange behavior. The Windows application seems to tell the RECORD service to set a resume point, but the add-on doesn't seem to be doing that. I've attached the RECORD service log from my test. 192.168.0.100 is the Windows computer with the HDHomeRun application, and 192.168.0.15 is a FireTV with Kodi 17.6 and the 1.2.5 add-on. As you can see, the Windows computer sends "cmd set-resume" at 19:50:01 and 19:50:04, then at 19:50:51, the FireTV starts playing at "0MB", then at "2245MB", then back at "0MB". In the add-on UI, a resume point wasn't shown, it just started playing at the beginning when I selected the recording. Does this shed any light? Can I supply any other logs? One thing to note is that I'm running the RECORD service on FreeBSD, not Windows. Could that be part of the issue?

20171128.log

[djp952]

I'm using the DVR on a WD NAS (EX4100), this may have been a bad assumption on my part that all the DVR engines behaved the same way. Let me see what I can find out.

What version is your DVR engine on FreeBSD? You can browse to http://ipv4-api.hdhomerun.com/discover, then find the "DiscoverURL" for the entry with the StorageID. My RECORD engine is 20171118beta1:

{"FriendlyName":"HDHomeRun RECORD","Version":"20171118beta1","StorageID":"ef0291b8-9e70-40c8-ba4e-9aee4bbe077c","BaseURL":"http://192.168.0.220:48843","StorageURL":"http://192.168.0.220:48843/recorded_files.json","FreeSpace":5079176650752}

If you can link me to the exact binary you downloaded from SD, I might be able to get at the string table inside to see if it has the same URL support in there.

Well poop.

[hunsra]

Looks like we're using the same version:

{"FriendlyName":"HDHomeRun RECORD","Version":"20171118beta1","StorageID":"7ef9a769-4459-9426-4f74-aeac2eed94e0","BaseURL":"http://192.168.0.6:60137","StorageURL":"http://192.168.0.6:60137/recorded_files.json","FreeSpace":3004485643776}

I'm doing some additional troubleshooting. I've added some log_notice() calls to the last position get and set functions in the code and rebuilt the plugin. I'll test and send you what I capture. Thanks again for your help with this. This is a great feature that I think anyone with multiple Kodi instances will like.

By the way, I've also found what appears to be a way to add support for Comskip. The PVR add-on interface supports getting EDL data from the add-on. Comskip will output EDL data. I just need to figure out how store the EDL information along with the recordings and to synchronize it with the RECORD service so the add-on can retrieve it using the IDs from the RECORD service. Just playing around with it at the moment, but if I figure out a way to do it elegantly, I'll let you know and maybe I can make a pull request or something.

Thanks, Randy

[djp952]

Sounds great! I never really looked into what the EDL was for :)

[djp952]

Looking at the log file, it looks like the DVR engine definitely accepted the command to store the Resume position. Would you mind looking at the output from http://192.168.0.6:60137/recorded_files.json to see if there are any "Resume":xxx entries in there?

I get the resume position from that JSON directly when Kodi asks, it's certainly possible that there is an error in my SQL statement to get it that isn't happening to me here.

One more thing, you disabled the shared PVR database in Kodi after trying that, right? I never set up MySQL here (didn't think I needed to anymore - ha!) so I never went into the advancedsettings.xml on any of my systems.

If you're building from source, you should also be getting a sqlite.exe (or just sqlite for *nix) that has JSON support enabled too. You seem quite proficient, we could play directly in sqlite against the database too to see if the query isn't working. Let me know, I can give you the steps (or just look in database.cpp -- get_recording_lastposition(). The intention is to pull the Resume field for the specific recording, based on the CmdURL, or coalesce it to zero if it doesn't exist.

The only trick to sqlite is you need to load the PVR DLL/so into sqlite to enable the extensions. The syntax is a little different on Windows vs. Linux.

[djp952]

I also wanted to post for you what my engine log looks like for comparison. I see a lot of skipping around, which Kodi is oft to do for recordings (I think ffmpeg is looking for the first/last PTS packets), but the main difference I see is that when I stopped the playback on the second system, I got the expected cmd set-resume again (Forgive my choice of program to stream, I wanted to pick something I knew nobody had been watching recently).

192.168.0.179 is a Firestick running 1.2.5, 192.168.0.174 is my main PC. For this test I used a Krypton Win32 (not app store) version client on the main PC.

20171129-03:00:48 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 0MB 20171129-03:00:48 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 3876MB 20171129-03:00:48 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 0MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 0MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 3877MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 3877MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 3877MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 3877MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 3877MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 276MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 342MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 335MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 332MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 332MB 20171129-03:00:51 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 starting at 332MB 20171129-03:00:58 Recording: stop Live channel 731 107efaa9 (result = 0x0000) 20171129-03:00:58 Recording: early end of Live channel 731 107efaa9 20171129-03:01:49 Status: Resource: nbk=1 dmk=952 20171129-03:01:49 Status: Disk read: MB=169 MB/s=522 worst=36 20171129-03:01:49 Status: Disk write: MB=79 MB/s=33 worst=186 20171129-03:01:49 Status: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 (82MB) 20171129-03:03:49 Status: Resource: nbk=1 dmk=952 20171129-03:03:49 Status: Disk read: MB=93 MB/s=3029 worst=1 20171129-03:03:49 Status: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.179 (176MB) 20171129-03:05:11 Playback: cmd set-resume: /mnt/HD/HD_a2/HDHomeRun/Movies/Tangled Before Ever After 20170101 [20170311-1800].mpg @ 560 20171129-03:05:49 Status: Resource: nbk=0 dmk=952 20171129-03:05:49 Status: Disk read: MB=68 MB/s=2150 worst=1 20171129-03:06:04 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 0MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 3876MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 0MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 3877MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 3877MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 3877MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 3877MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 3877MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 509MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 566MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 564MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 565MB 20171129-03:06:05 Playback: Streaming: Tangled Before Ever After 20170101 [20170311-1800].mpg to 192.168.0.174 starting at 565MB 20171129-03:06:14 Playback: cmd set-resume: /mnt/HD/HD_a2/HDHomeRun/Movies/Tangled Before Ever After 20170101 [20170311-1800].mpg @ 568

[djp952]

I'm spamming you, sorry. I opened an issue to track EDL support (https://github.com/djp952/pvr.hdhomerundvr/issues/18) and brain dumped how I think it would work. I thought of some challenges, none of which feel overwhelming.

[hunsra]

Glad you added the issue for EDL. I agree with the challenges. I'll continue to work on it and get back to you...

To answer some of your previous questions:

• I tried to get the response from http://192.168.0.6:60137/recorded_files.json but I received ERR_CONNECTION_REFUSED. Did I use the wrong URL? Let me know what I should use and I'll grab the info. Edit: The port number changed; I was able to get the information for the recording in question, and it has the correct Resume value: { "Category": "series", "ChannelImageURL": "http://img.hdhomerun.com/channels/US76739.png", "ChannelName": "BBCAPH", "ChannelNumber": "810", "EndTime": 1511022600, "EpisodeNumber": "S03E02", "EpisodeTitle": "The Enterprise Incident", "ImageURL": "http://img.hdhomerun.com/titles/C183885EN1GPX.jpg", "ProgramID": "EP000039850055", "RecordEndTime": 1511022630, "RecordStartTime": 1511018071, "RecordSuccess": 1, "Resume": 3149, "SeriesID": "C183885EN1GPX", "StartTime": 1511018100, "Synopsis": "A beautiful Romulan commander (Joanne Linville) woos Spock after capturing the Enterprise in the neutral zone.", "Title": "Star Trek", "DisplayGroupID": "C183885EN1GPX", "DisplayGroupTitle": "Star Trek", "PlayURL": "http://192.168.0.6:58983/recorded/play?id=2d2e4549", "CmdURL": "http://192.168.0.6:58983/recorded/cmd?id=2d2e4549" }
• Yes, I previoulsy removed the MySQL configuration settings from all of my Kodi instances' advancedsettings.xml files, thinking it might have an impact.

I put some specific logging in the add-on code and rebuilt everything. I updated the add-on on Windows 10 Kodi and an Android Kodi (FireTV 2nd-gen) instance with the new add-on. The Windows add-on seems to be correctly getting and setting the last played position, but the Android add-on is not behaving at all like I expected. I've attached the logs: Windows 10 Kodi: kodi(windows 10).log Android Kodi: kodi(android).log RECORD: 20171129.log

Here are the relevant details:

Windows 10 Kodi:

• At 06:48:27.926, it reports version 1.2.5.
• At 06:48:42.664, it reports successfully getting the last played position for a show, 3127
• At 06:49:17.782, it reports successfully setting the last played position for the show to 3149

Android Kodi:

• At 06:56:55.566, it reports version 1.2.3
• At 06:57:11.701 it streams the same show, but never reports getting or setting the last played position.

I've verified that both Kodi instances have been updated with the rebuilt add-on. Here are the ZIP files I used for each: Windows 10: zuki.pvr.hdhomerundvr-windows-win32-krypton-1.2.5.6541.zip Android: zuki.pvr.hdhomerundvr-android-aarch64-krypton-1.2.5.6541.zip

Both Kodi instances show v1.2.5 in the add-on details (see attached screen shots).

Have I rebuilt the add-on incorrectly? I've attached my build log below. Do I need to uninstall on Android before installing the newer add-on? I'm not sure what to do next. I think the code is correctly getting/setting position, but it appears that the add-on isn't being built or loaded properly.

Windows 10 Kodi:

Android Kodi:

Build log: buildlog.txt

[djp952]

I think I know what's going on here. On Android, in order to manually install from the .zip file you also have to mirror the .so file into the /data share. Android won't load binaries from the SD Card. So the "1.2.5" is coming from the addon.xml file on the SD Card, but the actual .so is being loaded from (sample path, may differ slightly): /data/app/org.xbmc.kodi-1/lib/{arch}

The version "1.2.3" in the log comes from a compiled-in string, so I'm hoping this is a simple case of a mistaken identity :) If you take the new .so (yours or mine or whomever's) and put into /data, I'll bet things start working more better! Android is a bit of a pain, but I understand their reasoning, executing code from the SD Card is a security disaster waiting to happen.

FWIW, I did write out some typical steps in the wiki, again the path on /data may differ slightly but since you're rooted you can also likely just find /data -name "libhdhomerundvr.so" to see where it is or if maybe there is more than one ... https://github.com/djp952/pvr.hdhomerundvr/wiki/Manual-Installation-on-Android. Suggestions on ways to improve that page are also welcome, it's far too tedious for most folks to endure.

adb shell cd /sdcard/Android/data/org.xbmc.kodi/files/.kodi/addons/pvr.hdhomerundvr cp libhdhomerundvr.so /data/app/org.xbmc.kodi-1/lib/{arch}/ cd /data/app.org.xbmc.kodi-1/lib/{arch}/ chown libhdhomerundvr.so system:system chmod 0755 libhdhomerundvr.so

PS - I made some stubs for EDL on a new branch, I'll try to keep you updated via that issue, you may need to post a "hello" or something to get notification e-mails from GitHub, though.

[hunsra]

Ah, that makes sense now. I normally use adbLink to manage installing APKs and copying files around, so I'm not very experienced with directly running adb. adbLink will start an adb shell on the connected Android system, so I was able to start that and try to copy the .so file as describe. Two problems happened:

• The /data/app/org.xbmc.kodi-1/lib/{arch}/ path doesn't exist. However, /data/app/org.xbmc.kodi-2/lib/{arch}/ does.
• When I try to copy the .so file, I receive the following error cp: /data/app/org.xbmc.kodi-2/lib/arm64/libhdhomerundvr.so: Permission denied

I tried su and sudo, but those don't seem to be available on Android. Is there some other way I can get permission to replace the file? Also, should org.xbmc.kodi-1 exist, or is org.xbmc.kodi-2 the right path?

Thanks, Randy

[djp952]

Unfortunately you do need root on Android to update/install a binary add-on into Kodi (like PVRs). At one point Team Kodi was working on a way to "chain" APKs so that binary addons could be separate APKs but that effort seems to have died out.

The only way to update anything executable about Kodi on Android without root is to install a whole new .APK. It's pretty annoying but sitting through an entire build process, the extended one that repacks the Kodi APKs is what I do all the time too. You get good at it, timing out when you'll run out of Coffee/Tea/Beer/etc so you have something to do :)

The good news is that I follow my own instructions fairly often so the README.md stuff is pretty accurate as to what you'd need extra to grab and repack the Kodi APKs. It's a bunch more stuff to download, you need the Android SDK and a recent version of Java available. You also need to generate a keystore file, which I copped out on and linked to Google's documentation for that. The good news is that it's not too horrible to set up, and at the end of the README.md is the extra-special build command to add APK generation:

msbuild msbuild.proj /t:PackageApk /p:Keystore={path_to_keystore};KeystorePassword={keystore_password}

If you try this method out, here is the command line I use here, without the actual password of course. I copy my keystore file to D:\ to make it less tedious to type out. You don't want to make the keystore file public, keep it offline.

msbuild msbuild.proj /t:PackageApk /p:D:\hdhomerundvr.keystore;KeystorePassword=xxxxxxxx

Now the BAD news is that any APK you generate will not be compatible with the real Kodi APKs or even mine because of the different keystore. So you also have to deal with uninstalling any existing Kodi before Android will allow you to install yours.

Fun, eh?

[hunsra]

Yeah, seems like fun :-) I'll give it a try. I already installed the SDKs per the build instructions, just haven't created a keystore yet. Thanks for the more detailed instructions.

One thought that came to mind is to support the "Auto-update" add-on feature (see screen shot). Would this help gt around the root issue?

[hunsra]

Installed your 1.2.6 APKs on two FireTV's and recording position getting/setting is working as expected. Thanks again :-)

[djp952]

Excellent! Marking this one as closed ...