djp952 / pvr.hdhomerundvr

Unofficial Kodi HDHomeRun DVR PVR Client
https://github.com/djp952/pvr.hdhomerundvr/wiki
Other
58 stars 9 forks source link

Extended Channel Names Not Displayed In Kodi (18.0 - RC3) #46

Closed gibxxi closed 5 years ago

gibxxi commented 5 years ago

Within the native HDHomeRun setup tool, the extended (real) proper-case channel names for scanned channels are displayed. Within Kodi (18.0 "Leia" - RC3) Only the truncated "Shortcut - All CAPS" channel names (abbreviations) are displayed.

My setup uses OTA guide data provision that's broadcast along with the channel data, and I'm based in the UK, using the Mendip Transmitter. I experience no such issues using any of the other HDHomeRun PVR add-ons available from the default Kodi Repo add-ons, but they don't provide recording functionality.

djp952 commented 5 years ago

Hi! I have zero experience with anything outside of the US (sorry!), can you help me out a bit more? If you go to your HDHomeRun tuner web page and select "Channel Lineup", are those the descriptions you want to see?

How it works today is that I use those channel names (the ones from the tuner) until / unless they become available via the EPG, then it will use those instead. It would not be hard to allow you to choose to keep the ones provided from the tuner, it shouldn't really affect anything since the channel numbers are used for things like scheduling recordings and whatnot.

If you see the names you want on the tuner page, let me know and I can add an option for you to keep those. Actually, I may as well do that anyway, it certainly wouldn't hurt :)

Thanks for using the PVR! I've never actually heard from anyone outside the US that used it. Other than this, does it generally work for you?

gibxxi commented 5 years ago

Hi,

If you want, I can provide you with screenshots of what i'm seeing in the SiliconDust App versus what I'm seeing in Kodi.

As for general functionality, Playing channels works fine. Did have a couple of minor stutters during playback last night, but it wasn't a major problem, and at this early stage, can't put that down to your add-on with any great degree of confidence anyhow. I will monitor it going forwards.

The system used for testing was a full-sized HTPC based on a Core-i5 4670K with a GTX950 graphics card and 16GB RAM, so I think it's more than capable of handling this. I'll follow up this post with some screenshots when I get a bit more time.

Essentially, the way OTA updates work in the UK, is that each transmitter has 4-5 muxes, and the channel names (and numbers) are broadcast separately to the channel streams, but on the same mux as the actual streams as far as I'm aware. This information contains the (full) extended channel names, where as the streams only contain abbreviated channel names, which are limited to about 4-8 characters. This makes it difficult to decipher what channel your looking at in Kodi versus the "full" channel names the SiliconDust app is able to decipher from the EPG data / mux data.

P.S: I did have a recording event i added that never started, but my NAS (QNAP) was setup with a QNAP Add-on that "pulled" recordings from the HDHomeRun hardware, as a pose to Kodi "pushing" the recordings via SiliconDust's software, or via Kodi. I've reverted back to stock config, and will try setting another timer and see what happens. I am a fully-paid-up DVR subscriber btw.

gibxxi commented 5 years ago

It would also be beneficial to us Kodi / GitHub users if the add-on content was in the root folder of your GitHub project, so we can sync changes / updates direct to Kodi via that, rather than downloading periodic releases. Not a big issue, but this would enable us to update changes / bug-fixes on the fly without having to wait for the next point release.

As it is, with the add-on content in a second-level subfolder, under the project folder Kodi ignores any add-on content in sub-folders so I can't use GitHub / SyncBackPro to sync your add-on directly to my Kodi install.

Not a big issue, but I am doing the above with a couple of my other Kodi add-ons that are "in development" and thought it was worth mentioning (as a potential feature request).

gibxxi commented 5 years ago

Here's a screenshot of the (extended) channel names, as SiliconDust's settings app sees them (from OTA EPG / Mux data):

https://imgur.com/nwwR8bm

And here's the first two pages of channels, as they appear with your add-on, within Kodi

https://imgur.com/u65vnpJ

and:

https://imgur.com/OupjRvf

If you would like me to also provide screenshots of the channels, as they appear in Kodi with Zoltan Csizmadia's HDHomeRun PVR add-on for Kodi (Which doesn't contain DVR / timer / recording functionality) I can do so, but i'd have to disable / remove yours and enable his first.

gibxxi commented 5 years ago

...Ignore the fact Channel 4 appears twice. The second one is from the Wenvoe (Welsh transmitter) that is farther away, to the north-west of my location, versus the closer Mendip transmitter (North East) so sometimes I get duplicates. The HDHomeRun is pretty good at biasing the channel list with the strongest signals first, but it doesn't always get it perfect. It's just a caveat, us down in Somerset (SW UK) have to put up with.

:)

djp952 commented 5 years ago

I got ya. It was indeed trivial to add.

edit: Sorry, it's not Kodi it's the tuner truncating it. Hopefully not an issue for your tuners, ATSC channels have sucky metdata :) The truncated descriptions I see match what I see in the source data. False alarm.

before: before

after: after

djp952 commented 5 years ago
      It would also be beneficial to us Kodi / GitHub users if the add-on content was in the root folder of your GitHub project, so we can sync changes / updates direct to Kodi via that, rather than downloading periodic releases. Not a big issue, but this would enable us to update changes / bug-fixes on the fly without having to wait for the next point release.

As it is, with the add-on content in a second-level subfolder, under the project folder Kodi ignores any add-on content in sub-folders so I can't use GitHub / SyncBackPro to sync your add-on directly to my Kodi install. Not a big issue, but I am doing the above with a couple of my other Kodi add-ons that are "in development" and thought it was worth mentioning (as a potential feature request).

I was thinking of setting up a Kodi repo at some point so the PVR can auto-update, if that's what you mean? Now that Android devices are valid targets for binary addon updates, it seems like the best plan.

gibxxi commented 5 years ago

I'd be able to work with that m8. If anything that would be easier (and better) than GitHub syncing as it is a bit of a kludge, even though it works.

I often forget to keep track of updates to software since I have so much I have to keep track of software-wise, so any automated / semi-automated solution you could provide would be a enormous help. 3rd party repos are not an issue for me, I already use several for other custom add-ons, made by community members (nothing illegal I might add, I don't use piracy add-ons in Kodi).

Dan / Gib.

gibxxi commented 5 years ago

With regards channel names, they're not just truncated. There's no spaces, and they're all in "All Caps" so I'm convinced they are coming from different sources on the HDHomeRun hardware.

Obviously, DVB-T/T2 (which is what we use) is different to ATSC (what you guys in the US use) but I'm assuming there should be something in the API SiliconDust provide that solves the issue for both cases?

Also, it's not just the UK. All of Europe uses DVB-T/T2, so if this add-on of yours becomes popular with HDHomeRun owning Kodi users like myself (No reason why it shouldn't), this issue will undoubtedly come up again I think.

djp952 commented 5 years ago

I think this is what you want, the descriptions in Kodi will match what you see in HDHomeRun Setup and the HDHomeRun App (confirmed both). There are only two places I can get it from - the EPG or the tuners, the new option works and allows you to use whichever you prefer! I've defaulted this to off, since this is the first time this has come up and the bulk of users are here in the US and as we've noted - our metadata is crap :)

One note about this -- you will still get the EPG (short, like "WBFF") channel names for recordings, provided the HDHomeRun RECORD engine isn't set up differently for DVB. The channel names I display for Recordings come from the recording metadata, it tells me what channel name to display, I don't do any type of lookup there. It's actually encoded into the start of the .MPG file, you can pop one open with a binary editor and see the metadata SD encodes (and change it, too).

I should get this out to everyone tonight, just doing some Leia housekeeping first, let me know if you think another option to try and marry the recording channel number back to the lineup to get the current channel name is warranted.

gibxxi commented 5 years ago

No, I'm fine with the truncated names in recordings, personally. Once an item is recorded I'm not really bothered what channel it came from. But browsing / watching TV is a different matter. I don't record much anyhow. Got an issue where the timer(s) set are working, but nothing ever gets recorded to the NAS even though the proper "recording started" and "recording completed" messages appear in Kodi.

This may be a permissions issue with the NAS, and it may be that the "HDHomeRun" share isn't on the first HD partition, but the second. I will have to do some more testing with this and check what's being written to the log. It used to work when my QNAP only had a single partition for data, but I've recently taken advantage of QNAP's advanced partition-management structure in order to segregate apps from media and data for security as well as fragmentation prevention.

Given the flakey nature of recent QNAP firmware it would not surprise me one little bit if these changes are causing a problem.

djp952 commented 5 years ago

OK. v1.3.8 is "live" for the grabbing -- https://github.com/djp952/pvr.hdhomerundvr/releases

The new option is called "Use channel names from tuner device(s) in channel lineups". I suggest restarting Kodi after changing this so that the EPG will also get the new names.

I've only tested it on Windows, will link it up properly and announce on SD forum after I try at least a couple more platforms.

gibxxi commented 5 years ago

Roger that. Will give it a test within the hour.

Thanks for your efforts.

:)

gibxxi commented 5 years ago

Well I've installed the add-on, enabled the option & restarted Kodi as advised...

https://imgur.com/vkOQBkV https://imgur.com/JljvrgJ

I think we can call this a success. Intriguingly, the startup channel scan seems a lot faster too. Before there was at least a .5 of a second between each channel appearing in the progress dialog at startup. It's now progressing MUCH faster.

Again, thanks for your efforts. Much apprieciated. Now to diagnose why nothing is being saved to the NAS when timers are set.

Dan / Gib.

djp952 commented 5 years ago

Anytime! Glad it was something I could turn around quickly for you. When you get the NAS working (and if you're ok with it) I would love to get my hands on a short DVB based recording file to run through the PVR. Being unable to test DVB I just want to make sure that the MPEG-TS packet filter isn't going to do any harm. I filter out a specific piece of data that breaks Kodi and use the stream data to do the timeshifting indicators. From what I've read there should be no impact to DVB but it would be nice to be sure :). Let me know of any other concerns as well. Hitting up SiliconDust's forums regarding the NAS may also bear fruit for you as well. Happy Holidays!!

gibxxi commented 5 years ago

Once I get it working, I'll record a short program and upload it to cloud storage and send you a link. I'm pretty sure I can find something (10 mins or so) that will suit your needs. Like I said, I have had this working before with your add-on, but that was on Krypton.

The issue with the channel names also existed on Krypton too, I just assumed it was a minor bug down to the fact the add-on was new, and / or maybe I was doing something wrong at the time. I didn't spend a lot of time using it until I updated to the Leia Betas, and then suddenly realised I couldn't use Zuki, because it wasn't ready for Leia at that point.

I'll repost here as soon as I have something for you. It might take a while, as I have 3 systems all capable of running Kodi here, all of which also have access to the HDHomeRun device. For clarity, mine's a HDHR-Connect (HDHR4-2DT) running 20180817 firmware at this time. I think starting with the native SiliconDust Windows app first will be the best starting point and work from there.

Happy Holidays & many thanks for your quick solution.

:)

Dan / Gib.

gibxxi commented 5 years ago

@djp952:

Please check your Kodi forum PMs m8, I've sent you a link to an DVB-T mpeg sample via PM (assuming I've got the right username). Let me know if/when you have it so I can delete it from my cloud storage.

Cheers,

Dan / Gib.

djp952 commented 5 years ago

Thanks again for the sample! I'm going to flag this particular issue as Closed to remove it from the list. If you have any other problems or requests please feel free to reopen or submit a new one!

gibxxi commented 5 years ago

Yep, no worries. I do have an issue where Radio channels and TV channels all appear in the "TV Channels" list, and the radio channels category (in Kodi) remains unpopulated, but this may equally be down to the way the HDHomeRun device works. Everything plays, so this is a minor issue, for which I may (or may not) raise a seperate issue for.

Thanks again for your help.

:)

djp952 commented 5 years ago

Interesting. I wonder if your metadata provides any differentiation between "Radio" and "TV" channels? If it does, I could actually move those channels over into the "Radio" area of Kodi for you. Here in ATSC/QAM land the audio-only channels come through as TV channels, there is nothing in the metadata to indicate if they are audio only or not.

For what it's worth, it seems that folks in the US with truly audio only channels have problems with the PVR and/or Kodi. Unfortunately my provider (Verizon FiOS) doesn't give me any pure audio only channels to play with. I get "Music TV" channels, there is a static video image on the stream.

If you are game, I would like to have a look at your discovery data to see if it includes any flags we might be able to key off of here. The easiest way would be to grab the "hdhomerundvr-v1.3.db" file from the Kodi userdata/addon_data/pvr.hdhomerundvr folder, that will have all the most recent discovery information in it. Another way is to interrogate the device directly from a web browser:

http://my.hdhomerun.com/discover, then find and navigate to the "LineupURL" listed to the tuner. You get back a bunch of JSON text that lists the channel lineups and all the metadata we might be able to use. I'm hoping for some unique flag on the audio-only channels or even just a lack of a "VideoCodec" flag.

gibxxi commented 5 years ago

I believe our setup here in Europe is the same as yours. We use DVB-T/T2 (SD/HD) with DAB (Digital Audio Broadcast) for radio stations. However, quite a few (if not all) the DAB radio stations as broadcast from the local transmitter are MPEG2 Video streams, MP2 audio, with a static picture as you say.

In fact it's not 'static' per se, it's the same picture re-broadcast at the given frame rate of the MPEG2 video stream, so for us, 25/50fps. I've never seen the video data displayed in any Kodi add-on for the HDHomeRun, not yours or any of the others. I think this was also the case when I used a dedicated TV Card & later a dongle, in the HTPC. At least not through Kodi. Said static image is visible via my dedicated Set-Top-Box DVR though (Toshiba RDXV60).

If you still want the data, I'll conjure up the items you've already asked for with a sample recorded stream from a DAB station and post it up to cloud storage as before.

An interesting note is that traditional radio (AM/FM) is not broadcast by the local transmitter AFAIK. Instead, each town has 'repeater' transmitters that output AM/FM to that locality so far as i'm aware. Whether this has always been the case, or only since the 'Digital Switchover' I do not know.

djp952 commented 5 years ago

Understood. I'm tinkering with a way to actually remove the video stream from these channels and move them into "Radio", but so far no success. The best I may be able to do is give you guys a way to manually specify that channels should be under "Radio" instead of "TV", but otherwise just leave them alone.

Seeing the discovery data would still be helpful, there probably isn't anything in there we can use to automatically move things around, but I can also ask SiliconDust if they would be willing to add something. I assume their EPG provider must know if a channel is TV or music in nature?

gibxxi commented 5 years ago

Rgr. I would say that if it ends up being a kludge to get it working, then leave it as it is. There are regional variations on broadcast radio channels in the UK, depending on the county you reside in (BBC Somerset, BBC London, BBC Wales, etc, etc. That's in addition to the regular Radio 1, 2, 3, 4, 5 Live, etc, etc.

At the end of the day, most skins provide the ability to hide unused home menu items, so it's not a major issue, just something that would be nice to have, like other TV cards, dongles, etc. The HDHomeRun is a bit unique in some aspects, but it's better than having to install TV reception hardware in all my networked PCs.

As I said, in Kodi, I never see the video stream in Kodi anyway. But I'm not sure if this is Kodi's doing, or that of the HDHomeRun device itself.

djp952 commented 5 years ago

Sorry for the delay in getting back to you here. First, thank you for the sample file and your PVR database, it was great to be able to see what kind of data is being used for DVB, and really great to finally have an audio only MPEG-TS sample stream!

In regard to breaking out the audio channels, I don't see anything in the data that would allow me to automatically make them 'Radio' channels, but I tinkered around with a few things. I think it still falls under the category of 'kludge', but I'll propose it anyway to see what your thoughts are:

I could add a table that tracks whether or not a channel should be considered "TV" or "Radio", defaulting all channels to "TV" at first. I can add right-click context menu items in the Kodi Channels view(s) that allow you to specify something like "Set channel as Radio" and/or "Set channel as TV". Selecting this would move the channel to the correct view as is appropriate. I can also add a PVR level "Client Setting" to let you reset that table to make everything "TV" again. I think this would be a bit awkward to use but still operates within the normal Kodi UI elements. It's also tucked away enough that people that don't care to use it shouldn't be bothered by it.

There are, of course, challenges to deal with. The first concern I foresee is that I would need to be able to differentiate between TV and Radio Recordings. Kodi asks for those separately. As long as the channel number metadata from the DVR matches the current channels database it's all great, but if there is no match the Recording will have to go to "TV" and show up there. The other main challenge I can see is that if I put this information into the PVR database you would have to set it up on every Kodi device you have individually, and if you uninstall the addon the mappings go away on you.

I can't think of a good solution for the Recordings pitfall possibility, but to overcome the PVR database being local to the machine one thing we could do instead would be to define a file format and use a flat file the user can specify instead. If this file were in a shared location, all Kodi instances could pick it up and read it.

I'm game to pursue this, I've already gotten most of it to work offline, and I recommend as a first try going with the option to use the PVR database as opposed to a flat file on the network. It would allow people to try it out without any real impact to those who don't care, and hopefully gather some constructive feedback for possibly better or more seamless ways of doing this.

Let me know what you think -- I have plenty of disabled "music" channels to play around with. Kodi doesn't seem to care if they have audio/video or just audio - it plays them regardless :)

gibxxi commented 5 years ago

Hi djp952,

While the end goal is to have a PVR (DVR) system that works for all, It worries me that an excessive amount of "workarounds" (for want of a better word) may result in in issues further down the line. Because it's not only the vagarities of the HDHomeRun devices and the supporting firmware / software we have to deal with here, but also the inherent vagarities of Kodi itself.

Since the main issue here seems to be the playback of radio channels (at this time) and recording doesn't seem to be a problem, why not just examine Zoltan's HDHomeRun add-on and see how he manages to get his add-on to work without issue, and compare his code to yours to see if there's a way we can bridge the gap? While I'm not an advocate of plagiarism or anything of that ilk, it does seem to me like we might be trying to "re-invent the wheel" here.

From my position, which probably differs from yours by quite a margin, as a simple user, my goal is to have a DVR add-on for Kodi & HDHomeRun devices that just works. It plays back all the channels I can receive, and it records all the channels I can receive, and the likelihood for it breaking at some point down the line is low. It's not my concern (as a simple user) how we get to that point, just that we do.

I understand this is your project, and you want it to be the best it can be, and also code it in such a way that your happy with it, and can stand behind it as a developer. I totally understand that. And if we have to force a kludge to get to that point, I'm not against that either (assuming it doesn't have a negative knock-on effect). But I do think it may be an idea to reach out and see if there's a chance of pooling resources here. Both to achieve the desired end result, but to also avoid you having to put in a ton of work that is at best, a 50/50 as to whether it proves to be a long-term fix / solution.

Also, since I last messaged you, I stumbled across this thread on the Kodi forums...

https://forum.kodi.tv/showthread.php?tid=339685

...look somewhat familiar to you? The OP's issue is based on completely different usage scenario to mine, but, the effect he's seeing is exactly the same as what I am. So it may not be an issue with your add-on after all. I'd hate for you to do a ton of work, then have it turn out that the issue lied elsewhere all along. I can only report what I'm "seeing". But while I can put 2 & 2 together, I'm not a developer or a coder, and what I'm seeing, and believing may not exactly tally up with what's actually happening under the hood.

Lastly, there used to be a thread on the Kodi forums for this add-on. Is it still active? Because I can't seem to find it now. I think this discussion would be better served, being posted there, so other potential users of your add-on can add their input, as well as advertise it's existence and get more people (testers / users) on board. Maybe my situation is unique, maybe it's not, but as a single user, I am not a big enough user-base to judge the issues (or lack thereof) that may exist in this add-on, to be fair.

If we can get to a point where it works without issue, I'll be happy with that, whatever you decide to do in the long run. I'm happy to test for you in any case, and report back my findings. But again, I re-iterate that my setup isn't exactly "stock" either, what with the NASes being the recording targets and all, and me utilizing Powerline technology. So i'm not exactly your average user to be basing code standards against IMHO.

Dan / Gib.

djp952 commented 5 years ago

It appears that I may have inadvertently insulted you, I assure you that was never my intention. I was trying to respond to the Radio vs. TV concern open on this issue, as opposed to the "audio only doesn't work" issue, which I have been treating separately (apologies). Regardless, I appreciate your feedback and concerns and also want to provide the best possible solution that works for as many people as possible :)

When compared to the mainline Kodi HDHomeRun addon, there are two primary differences that have to be taken into consideration. That addon hits the tuner device(s) directly as opposed to the accessing the DVR engine, which results in provably different MPEG-TS streams. The DVR engine adds lead-in metadata that the tuners do not, and also makes modifications to the PES (PMT) packets at a minimum, and there may be more alterations that I am unaware of. It also just sends a URL to Kodi to handle as opposed to handling the stream data directly, which until Kodi 18 "Leia" couldn't be made to work with the DVR engine properly and has no support for PVR timeshifting. It also has no way to deal with the "poison pill" SCTE packets we get here in the US that screw with ffmpeg to the point where channels won't stream at all.

The stream content difference can be tested by setting "Stream Live TV channels directly from tuner device(s)" to ON, which should provide the same behavior as the mainline addon does. You won't be able to seek on the stream, but may alleviate the concern. Unfortunately I don't currently have a way to completely remove the custom streaming implementation, but I have been assisting a developer that has been working to port my PVR to mainline Kodi to replace their existing addon and we did come up with a way to make that work. I am unaware of a release timeline there, he's been working on it for quite some time without any public releases.

I never had a specific thread on Kodi.TV for this addon, mainly because I "do my own thing" build-wise, but there is a thread on the SiliconDust.com forums (https://forum.silicondust.com/forum/viewtopic.php?f=88&t=65776). It kind of ebbs and flows activity-wise, lately has been kind of quiet. I hope that means it's all working pretty well, but it could also mean nobody is using it anymore!

I haven't had any problems with the sample file you provided, but as time allows I am trying to see what I can break. The linked post on kodi.tv does indeed sound exactly the same, I hope it's a Kodi/ffmpeg issue in the end since all I really do is take the data from the HDHomeRun and hand it to Kodi, but it's difficult to make a recorded MPEG-TS file appear to be "Live". Kodi/ffmpeg behave differently when they can seek anywhere in the stream as they appear to do when they cannot. It's a "process", for lack of a better term :)

I'll give up on the Radio vs. TV stuff since there doesn't appear to be anything I can do that isn't at least a little bit of a hack and try to double down on the root streaming issue here.

gibxxi commented 5 years ago

Hi djp952 / Mike,

You didn't offend me in any way m8. I greatly appreciate the work you have put in so far to get this add-on viable for use, even in it's current state. My message above was simply an attempt to clarify where I am, what I'm seeing, and ultimately where I'd (ideally) like to be.

I dabbled with Turbo C++ and Turbo Pascal many moons ago, and I know only too well how frustrating it is to spend a large amount of time on a project, only to realise that halfway in, you've been following the wrong path and have to go back and make major changes to the code because you failed to observe a critical flaw early enough. It's for this sole reason, I voiced my concerns. I'd hate for you to have to add a ton of code now that proves to be a wasted effort, later on down the road. That would not sit well with me. I would feel a ton of guilt about that. I hope this clarifies my reasoning well enough.

:o)

The native HDHomeRun "View" program isn't very user intuitive, and is pretty awkward to use in it's current form. So when you wish to make recordings from an armchair, armed with nothing except a remote control, it's use becomes almost impossible. Not to mention the fact, as a Kodi user, a solution that exists within Kodi, is the preferred choice anyway.

The above isn't true for all the systems I run Kodi on, but I like to have consistency in the way I do things. Just call me OCD, lol.

The thread link you posted may actually be the one I was thinking of, as I'm also a member of the SiliconDust forums. I probably got the two mixed up.

With regards the provided sample, I never anticipated you would get any issues with it, as the actual recording of streams from the HDHomeRun tuner is essentially, completely external to Kodi. Kodi initiates the recording, but the stream is passing from the tuner to the recording target directly (in my case, the NAS), and Kodi is only really involved when it comes to relaying status messages to the user.

I will try the "stream direct from tuner" option. The confusion for me comes from the fact that, under a vanilla install of Kodi (i.e: no 3rd-party add-ons) I get no issues. Load up my usual selection of add-ons, and it doesn't play ball.

I do have both the "Simple Cache" and "Common Cache" add-ons installed, and did wonder if one of these might be to blame. What's even more confusing (to me) is the fact that Video Channels playback fine (albeit with a slight delay before getting a picture & sound after the channel is selected). But radio doesn't. Like I said, I get no issue with the current official (unofficial) Kodi PVR add-on. I do also have SiliconDust's Video plug-in installed, but haven't used this in a long time.

I guess we'll have to wait and see what your contact can conjure up. It may be warranted to add a note about this issue to the UI for the "stream directly from tuner" option though (assuming I can verify it as a solution), in case any other European users like myself attempt to use this add-on, and experience the same issue.

Thanks for your work, and your friendly ear. Much appreciated.

:o)

Dan / Gib.

gibxxi commented 5 years ago

Update: Tried the "Stream Direct From Tuner" option. It has no effect. The audio stream still stops after 500ms. Any attempt to stop the stream playback, or back out of the Music Visualization screen causes Kodi to hard freeze, requiring termination of the process via Task Manager.

Dan / Gib.

gibxxi commented 5 years ago

Update 2: Tried another vanilla Kodi install by renaming my "portable_data" folder, and restoring your add-on + it's settings from the old add-on data folder.

Was able to get radio to play without issue. Stopped the stream, exited, patched back in some skin-related settings, and re-installed the skins they belonged to, and attempted another radio station playback. Again, playback without issue.

So now I've reversed the setup, and copied all the databases from the new install over the top of the old ones, and renamed back the portable_data folder.

I'm not going to attempt playback 3 until both the video & audio databases are fully populated, which at 1,400 films, 2,700 episodes and 48,000 music tracks, probably won't be before I hit the sack tonight. I'll leave it running overnight, and test again after work tomorrow night, just to see if there's a database corruption issue going on here.

I can't disable the "Common Cache" plug-in as that's a dependency of at least 5 of my installed add-ons. So at this stage, I hope it's some kind of database corruption.

Remember: I regularly manually copy my "portable_data" folder between machines, then tweak where I need to, to allow for variances between those systems. I don't want to have to administer 3 separate Kodi installs from the ground up, and the amount of changes required are fairly minimal between the three systems I run.

The theory here is, that a crash during testing on the desktop, may have corrupted something in the databases, or some entry about a played stream may not have ben properly cleared. Thus later on, the problem migrated to the other systems with the portable_data folder.

Time will tell. Leia has not been one of Team Kodi's better builds IMHO. It's been riddled with crashes and issues since I started using the early betas, and is still only borderline stable for me at RC5.2. However, the add-ons I have installed may be partly to blame too. Time will tell. I'm not running any piracy-based or dodgy add-ons, for the record.

Dan / Gib.

djp952 commented 5 years ago

I'm not home right now so I haven't read everything here yet (I will tonight), but as a quick note I started adding some entry point logging for us. It will be a bit insane for things like Read() but for troubleshooting very valuable. I'm hoping to see exactly what Kodi is asking for right before it stops. Guessing is only going to get us so far :). I'm doing it 'real' style as in making it a formal option under Advanced. More as I have it and after I can sift through all the info! Thanks again for your help and of course patience!!

djp952 commented 5 years ago

Now that I've read everything, I'll sit in a hold pattern and wait to hear back. Turns out adding entry point logging was a bad idea anyway, just the action of checking if it should be done had a measurable impact on the streaming functions and there is just no wiggle room there. Need to read my own comments from time to time - ha.

I truly appreciate the time and effort you have been putting in to get to the bottom of this. I was thinking that if it's truly all good after your databases are reloaded and reset, we could diff out the prior portable_data and the new one to see if we can find what was afoot here. I can definitely help there if privacy concerns aren't an issue (I'm a pretty honest guy, but of course both honest and dishonest people say that), and I can also provide you with a compatible version of the SQLite command-line tool we could use to deep dive into the Kodi (and PVR) databases. I don't ship that with the PVR, but I do build it for all platforms and use it here for diagnostics all the time.

While I of course hope it's fixed, if it turns out not to be there is another thing we can do as well. It's not too complicated to manually grab a tuner stream and dump it into an .MPG file. You can do it with a number of tools, "wget" works well. This is how I eventually found and fixed the SCTE issue, I hand-compared the streams from the tuners with the streams from the DVR engine using a tool called "MPEG-TS Packet Analyzer", and ultimately came upon the different PMT packets as being the poison pill that was killing Kodi.

Have a wonderful night (early morning, actually), let me know how Test #3 goes!

gibxxi commented 5 years ago

Alas, Kodi forced closed (again) during the night. No idea if the library had completed or not. I imagine so since the library scan at startup was fairly quick.

Alas, tuning the radio channel resulted in the usual behaviour. Seeing a lot of "failed to add packets to renderer" in the log, but no outright plugin errors.

So that rules out the core databases. Frustrating thing is, the elapsed duration is up at 14 hours once you get the music vis screen up. If I could isolate where it's getting that value from, I'd be well on the way to solving the issue.

It's not Common Cache either, as I cleared it's database.

Dan / Gib.

djp952 commented 5 years ago

Hi Dan! I apologize, my notification for this post was lost in a heap of spam. Can you show me a screen shot of where you see the 14 hours elapsed time? I can probably tell where Kodi is getting that based on where it's displayed.

The way things are supposed to work with the UI timers for live streams should be a combination of EPG data (program start and end times) and data I generate. If you haven't seeked on the stream at all and don't see the "Timeshifting" bar in Kodi that can eliminate one of the hokey parts of the code (figuring out the point in time you are actually viewing/listening to right now). There could very well be a problem with the data I'm generating for these streams, which could lead to UI behavioral issues :)

gibxxi commented 5 years ago

Hi Mike,

Here's a screenshot I took last night. It seems to be accruing all playback time ever encountered (since last reset). The value on the right, is the current playback duration, and it seems the one on the left is total playback time (cumulative). If you choose the "Stream direct from tuner option" only the current session playback time is then shown. Both values increment on a per-second basis. Also notice that the program displayed in the top-right seems to be inferred from this as well, as the "current" program title in this screen shot is actually the next program in the EPG. The reason for it only showing about 2 hours, as a pose to the 14 hours I reported, is that this screenshot was taken on the desktop, not the HTPC. But rest assured, the 2 hours is the same symptom as the 14 on the HTPC, since at the time this screenshot was taken, the stream had only been active for about 7 minute.s (as indicated by the duration to the right). Usually these values in other streaming add-ons, are switched in terms of position (if it matters).

https://imgur.com/B8lmdJO

If I wipe out my "addon_data" folder (or simply rename it), I don't get an issue with the stream freezing, but if I put it back, it freezes. I tried disabling most options (aside from channel names from back-end) in the HDHomeRun plug-in, it worked on one Kodi run, froze again on the next. Tried disabling / removing add-on settings for some of the other suspect add-ons in that folder one by one with similar effect.

I've since remembered something. My Powerline adapters (4x) are two TP-Link TL-PA9020p (V1) kits. TP-Link enforces QoS on these things with no option to turn it off. You can only select between 4 different modes. I have mine currently set to "Media". So it may be the case that the stream is being blocked by them initially. I do get a 30 second delay when using a "Save As" dialog via any program (on any of the PCs) to either NAS, but not when browsing shares via Windows Explorer. However, the HTPC is connected to the same Ethernet switch as the HDHomeRun device, so if this is indeed the case, I would expect issues on the desktop & laptop, but not on the HTPC. Unless the switch is so cheap (also TP-Link) it doesn't do smart routing to other devices on the same switch like Netgear consumer switches seem to do.

EDIT: Scratch that. The Powerline adapters are rated at 1200Mbps (across the two Ethernet connectors they feature). Both the HTPC and laptop are connected to Port 1, on their respective Powerline adapters, the switches at those locations, are connected to port 2 on each. The desktop and primary NAS (TS453-Pro) are connected to a Netgear GS108Tv2 managed switch which is then connected to the Powerline adapter at that location.

It may also be down to a buffer size limitation, but I can't compare with the other HDHomeRun addon as I don't think it gives you the option to tweak the buffer size.

I can give you a list of add-ons I have installed that automatically re-populate the addon_data folder automatically, and thus are likely to be suspects if it is an add-on conflict. My money was on script.common.plugin.cache and/or script.module.simplecache, but have since changed my mind. The crux of the issue certainly seems to be network related, with regards to the "failed to add packets to renderer" error messages being reported by CDDVDPlayer in the Kodi log, but the cause still eludes me.

What's even more frustrating is the fact that TV playback on your add-on, and all playback on Zoltan's plug-in seem to work without issue.

One thing I can confirm, is that Kodi is processing radio channels as audio-only streams, not video streams (with audio). I use the Aeon Madnox skin, and have it set to go to the Music Vis screen on playback start. When starting a radio channel, it does just that. Milkdrop2 is visible in the background, and I get a pop-up about CU-LRC Lyrics attempting to find lyrics for "-random hash value here-.pvr" and obviously failing.

Dan / Gib.

djp952 commented 5 years ago

Hi sir! I haven't necessarily been ignoring you here, some other things came up. Both real work and this work. Just wanted to ping you quick to let you know I didn't abandon ya :)

gibxxi commented 5 years ago

No worries. I've switched my two daily-driver systems (HTPC & Laptop) back to the other HDHomeRun PVR add-on for the time being. Being able to play channels without Kodi falling on it's face is more important ATM, than recording functionality.

I've also installed Deamonrik's "HDHomeRun DVR UI" add-on on one of the NASes, so that I can at least queue recordings, albeit outside of Kodi.

The setup on the desktop has not been altered, as that install is meant for testing purposes only anyway.

From what I've been seeing locally, there is certainly an issue with regards buffering / streaming of radio shows. I did an extended playback test on one of the occasions it played without instantly losing the stream. It lasted for about an hour before failing. Bringing up the HDHomeRun Tuner status webpage showed the tuner in use, but no data being pulled / pushed to / from my desktop PC.

Dan / Gib.

gibxxi commented 5 years ago

...Also, I tried changing the "priority" mode of the Powerline adapters to a option I don't personally make use of.

Out of the 4 options available (Music & Video, Internet, VoIP & Online Game) I chose VoIP, since I do not own or use any VoIP-based tech at this time. The rationale being that if it's set to that, and no VoIP traffic is detected, other things will not suffer as a result, as there will be no need to apply the brakes on other traffic types to prioritise VoIP. This is obviously assuming that these adapters don't reserve bandwidth for the chosen option on a permanent basis.

It's had no effect, either positive or negative that I can tell at this time.

Dan / Gib.

djp952 commented 5 years ago

Great information, thank you. I would discount the network and/or QoS as a root cause. If the in-built PVR is working without a hitch, it has to be my code. The regular PVR uses Kodi's cURL "file" code to stream as opposed to doing it on it's own, which leads me to ....

I have a new theory about the audio-only playback based on the additional details. What I think might be happening here is buffer starvation. This really never happens with video streams, there is so much data to be processed if none comes in within a reasonable timeout window it can be assumed that it's dead. A significantly lower bandwidth stream could, theoretically, actually have no more data available when asked. I definitely am not accounting for a scenario like that. I'm going to poke around Kodi's cURL implementation and see if I can get any pointers there. I also need to come up with a way to try that out here. (This entire problem probably wouldn't exist 2+ years in if I had one of these channels!)

As for the other issue with the UI reporting weird progress, I also have a theory there. I'll start with that I installed Aeon Madnox on Leia, it doesn't look like your screen shot at all - what skin are you using that shows the progress like that - I'd like to see it first-hand if possible? Regardless, what I think may be going on based on the screen shot is that the skin is showing you the "Timeshift" progress as opposed to the "Realtime" progress. Stock Leia (Estuary) has 2 individual progress bars. I only report "Timeshift" progress to Kodi if you are streaming from the DVR engine, since you can seek on that stream. When streaming direct from a tuner I tell Kodi to pound sand when it asks for this, since it can't be timeshifted. It would fit the symptoms, at least.

Timeshift progress is supposed to tell you a few things: 1) Are you timeshifting? 2) How far back does the timeshift buffer go? and 3) What time is being played right now? To get this information is hard and in my case probably a little hokey. I scan the input stream for the arbitrary timestamps inside of it and use that information to answer all three questions. It's imperfect, but works most™ of the time. It's also designed to shut itself off if the answers become unreasonable, like if the timestamps suddenly go backwards.

If we can prove that the skin is using the Timeshift progress instead of the realtime, that can easily be made into an option to just turn it off or we can talk to the skin author about it. I don't think many of the in-built PVR addons try to report timeshifting.

Let me know if you can about the skin used for the screenshot, and I'll start tinkering around with what happens if a stream temporarily has no more data to give but hasn't actually ended. This may tie into an older issue I'd love to fix which is how I can reconnect to a dropped stream automatically. Should be similar, if not identical, concepts.

Thanks again for all the info, and of course your patience :)

gibxxi commented 5 years ago

Hi Mike,

I am indeed using the Aeon Madnox (Leia Version) skin. The screenshot I sent you was from the EPG screen from the Arctic Zephyr skin I also have installed, but as the same values are displayed in both, didn't think it would matter. I am not active in the thread for it on the Kodi forums for personal reasons. I can't find the correct forum thread at the moment, but you can sync it via GitHub from here:

https://github.com/mikesilvo164/skin.aeon.madnox

The original thread for this skin (mod) can be found below, but be advised: That thread is only for the "Jarvis / Krypton" builds of Aeon Madnox, not the Leia build. The guy responsible for Aeon Nox SiLVO is also concurrently maintaining the "Leia" build of Aeon Madnox (on behalf of Mad_Mike_Doc) and providing support / bugfixes (but no enhancements) for that build. It is entirely possible I've been banned from the thread, thus I can't see it, as I was pretty scathing towards the maintainer (MikeSiLVO) about his attitude and practices with regards the maintenance of the Leia version of this skin.

https://forum.kodi.tv/showthread.php?tid=230821&highlight=Aeon+Madnox

Be advised: "Aeon Nox SiLVO" is a fork of the "Official" Aeon Nox 5 skin, and is nothing to do with Aeon Madnox. MikeSiLVO is maintaining both on behalf of the community (for Leia).

I'm also beta testing for another skinner, called "Mr V." who is in the process of recoding Aeon Madnox from the ground up, for Leia and beyond. His version is simply called "Madnox (2.0)". I can't provide you a link to that, as it's by invitation only at this time. Regardless, I've not used that version with your add-on anyway (AFAIK).

I have used time-shifting on my systems, both mistakenly (pressing wrong key to stop stream) and on purpose. However I would expect this to be reset at the point Kodi is exited / restarted. or the stream is stopped by the proper means and then restarted. The weird time values might suggest that this is not happening.

I have had issues with Live TV channels freezing too, but put this down to decoding, as the render mode selected (Auto vs. DXVA) was also having an issue with de-interlacing not being applied. When changing this option from "Auto" to "DXVA" explicitly, the problem seemed to disappear.

It may be the case that the same thing is happening in both cases, as you don't get on screen duration info with TV channels, so any time-shifting issues wouldn't be so instantly recognizable. However, I'm not experiencing any instant lock-ups with TV streams either (at this time), so maybe not. One thing is for sure, Kodi is treating Radio streams as Audio-only (music) streams, and processing them as such.

At the point the stream is stopped manually (even if time-shifted), then the HDHomeRun deletes the "Live TV" folder from the NAS's recording share, and any streams it contains. This may be the reason we're getting freezes here.

The add-on could be looking for packets of data from a (time-shifted) stream that no longer exists physically.

Dan / Gib.

gibxxi commented 5 years ago

Had an interesting experience today, using Zoltan Csizmadia's HDHomeRun PVR Add-on on the laptop...

...It lost the stream in exactly the same way as happens when I use your add-on. The difference? Unknown to me at the time, Acronis TrueImage was performing a cloud backup / check from the same machine.

Now I'm going to say right now, that I don't think Acronis TrueImage is the definite cause. Reason? 99% of the time, I've had no issue with the other PVR add-on for streaming, and I'm reasonably certain, since Acronis backups occur on all my systems at set times, that the constant failure of streams on your add-on cannot be attributed to it directly. Also, for note, backups it does run are set to the lowest priority I can in the software, specifically to avoid this kind of issue.

The reason for posting this message, is that I have a theory as to what might be occurring here, and it's not the fault of your add-on or Zoltan's (specifically) either.

I lay the blame at the HDHomeRun device and it's handling of streams. As I noted above, upon the disconnection of a stream, with your add-on, which does support time-shifting, it appears if the connection is lost, the device will "clean up" automatically. It deletes the time-shift buffer and stops sending packets of data, or attempting to send packets of data to the remote IP. In the case of Kodi, it's then waiting for data it will never receive, as the connection has been lost and the HDHomeRun device has already deleted the buffer.

Also, Kodi, through either PVR add-on, seems to be unable to recover the lost stream once it has failed, and I believe it's due to the above issue on the HDHomeRun device itself. Or rather the way the two interact with each other.

The fact is, that while Kodi has lost the stream, and is not outputting any audio at this stage, it's not frozen / crashed at that stage. The visualization is still active, and the on-screen channel icon (& spinning CD) are still quite happily being displayed / rotating. The instant you perform any action that requires a GUI update though, and Kodi will freeze, waiting for a response from the HDHomeRun box it's never going to get. That includes attempting to stop playback, using the Backspace or ESC keys to back out of that screen, or even the "backslash" key to switch between full-screen or windowed modes. At that time, no more entries will be added to the log, and it's necessary to task-kill Kodi and start again.

For me, with my Powerline setup, it's entirely possible, that other processes (within Kodi) or other services on the system are taking bandwidth that cause the Powerline adapters to put the brakes on the stream, because it's not being recognized by them as "audio or video", and it's then prioritizing those other transactions.

Powerline is like Wi-Fi and Ethernet combined. The Mbps connection rate can fluctuate wildly, although generally not as wildly as Wi-Fi, nor as much. But the other caveat to it is, AFAIK, only one node can transmit or receive data per cycle, much like non-Mimo Wi-Fi, so even if services on the current system aren't affecting the connection, other systems connected to other adapters might well be doing so.

Either way, that's somewhat irrelevant. The relevant thing is, that (broken) streams from the HDHomeRun appear to be impossible to recover once failed, and some kind of timeout needs to exist to tell Kodi to stop / give up attempting to retrieve packets from a stream that is now essentially dead. Doing so is far from ideal, but is much better than Kodi persisting in the attempt to get data from a non-recoverable stream, so it doesn't lock up the entire system.

Also, the time-shifting status needs to be reset at the point we determine the stream is non-recoverable or it has been manually ended, thus doing a "clean-up" of the connection that was broken / stopped. Assuming Kodi isn't hung / frozen by the PVR add-on, we can always retune or attempt a reconnect a cleanly terminated connection. While a PITA, it's better than having to force-close the whole app each time.

Unfortunately I cannot offer you access to my local setup to test, Sorry.

Dan / Gib.

EDIT: I'd also like to point out at this point, while I did have to force-close Kodi, using Zoltan's add-on, since it does not support time-shifting, once the bandwidth restriction was removed, upon a restart of Kodi, attempting to connect to a radio station worked as usual, without issue. Once the same thing happens with your add-on, Kodi is still under the belief that it should attempt to connect to the previous (time-shifted) buffer, which it appears to do (while still playing 500ms of audio from the current tuner output) before falling on it's face.

Therefore it might be an idea, to clear any add-on time-shift cache / buffer content at add-on start as well, as at the end of playback, in order to be safe. Since a time-shifted stream shouldn't exist across multiple runs of Kodi, and Kodi does have a tendency to crash often, this will prevent issues for users who run more "standard" setups than me, who have the misfortune of Kodi crashing on them (for an unrelated issue) while time-shifting using your add-on.

djp952 commented 5 years ago

I was thinking much along the same lines, actually. This morning I found a nice little tool called "Clumsy" that allows me to simulate various network problems (https://jagt.github.io/clumsy). I've only tinkered with it so far (today is daddy daughter day), but it seems like it will be quite valuable.

My intention is to sit down with it and see what types of behaviors I can mimic effectively, and how cURL and Kodi react. The main thing I need to see the differences for is in-progress recordings; they appear as "Live" until they are done, but if you don't seek I don't get new HTTP headers to tell me it now has a length. That can be worked with :)

I think you're onto something with the buffer disappearing on the DVR. Stopping and restarting the stream would take care of that, the timeshift information will be reset, since it needs to be assumed that you can't go back any farther than when you started.

Kodi is indeed very fussy about this stuff, and Leia has been doubly annoying. Too many changes in one release in my opinion.

At a minimum I should be able to get you something with at least some additional logging points to see if we can come up with the proper methodology for when to give up on a stream and try to start it over again. I will also be adding some logging points to indicate the timestamps being used, what will be reported for timeshifting, etc. Honestly it will probably take a few iterations to come up with the right series of "stuff" to smooth this all out.

(Or I can send you a spool of Cat 6 and a 1000-BaseT switch - bright blue wiring strewn across the floor solves everything! LOL)

gibxxi commented 5 years ago

I already have a 15m CAT5/6 cable that can be utilized if necessary, and The HTPC area, and laptop area are both connected to the Powerline Adapters via TP-Link switches, so it is possible for me to "create" a direct connection. Not easy, due to the way things are laid out here in my 1-bed flat (apartment), but doable nonetheless. I'll PM you (on the Kodi forums) some photos I already have of my setup(s) when I get time.

I can't remember off the top of my head how the desktop connects. My GS108Tv2 is sat behind my NAS in a small cubby-hole in the desk, and getting at it is a problem. I have a feeling though, that the switch connects to the Powerline adapter there, and to the router, as well as the primary NAS and desktop PC. There isn't any issue with network loops though, it was all planned out in the right way before assembly.

Both the primary NAS and the desktop PC use LACP teaming, but it's broken (again) on the desktop at the moment due to ongoing issues with Intel's Ethernet driver compatibility (ANS) with the recent RS5 Windows release (October Update).

I'll download Clumsy and get it installed on the desktop. If you have any specific things you'd like me to try, let me know.

With regards remote connections, it's not that I have anything to hide per-se, but I wouldn't feel comfortable allowing anyone I don't know personally accessing my local system(s) without me being there to monitor what they're doing, and since I only get two days a week (on average) to dedicate any time to this "project", and the fact that our active times are off by a minimum of 6 hours, it's not doable for me (at least at this time).

If I book some holiday from work in the next few months, that scenario might change. If it does, and we're still at an impasse, I'll be sure and let you know, and we can work something out. I have a TS3 server running here, and TeamViewer, so I'm sure we can work something out at a time more suitable.

FWIW: I notice that a lot of people on the Kodi forums posting issues about PVR and stuttering and performance issues with Kodi 18. Some using the ServerWMC back-end, some using NextPVR (I used this when I had a TV card in the HTPC), and some issues with Nvidia's Shield device (Android).

I think we can however rule out our initial assumption that it was something to do with the stream itself or the way it was constructed. I will need to do some research on how DAB Radio is packaged. I know it's an MP2 stream (MPEG-1, Layer 2), but I do not know if it's also encapsulated in a MPEG2 transport stream like the TV channels are. It's not possible to trust the Live recordings from the HDHomeRun, as there's probably on-the-fly transcoding going on there anyhow, for playback consistency.

I also want to do some testing with Live TV and time-shifting, to see if this issue is indeed only related to radio channels, or whether my previous issues were down to the same problem. Unlikely to get time to look at that in any depth until next weekend though. If I see anything that I think I ought to report, I'll do just that.

:)

Dan / Gib.

gibxxi commented 5 years ago

https://forum.kodi.tv/showthread.php?tid=334453

Might suggest that my suspicion that there's a Kodi issue with the PVR service is not misplaced (See last post by eddieleong).

Dan / Gib.

djp952 commented 5 years ago

Thanks for the link! It does sound familiar :)

I have a couple questions that may sound dumb to you, but I have to ask anyway (this thread has gotten very long). 1) What version of my PVR are you using right now on Leia, have you updated to the latest (v1.3.12)? 2) What platform / OS were you running when you took the screen shot from Arctic: Zephyr?

I ask because what I'm seeing in Arctic: Zephyr is different than what you saw. What I see are the raw stream times being reported, as in "00:00->XX:XX", where "XX:XX" is the number of seconds since the live stream was opened. There was a bug on 32-bit platforms for a couple versions that would send the "end time" being reported into negative territory. Regardless of that, the stock Kodi skin actually uses the stream time data in conjunction the EPG to figure out where you really are. What I can do here is play with the values I'm reporting to see if I can get the starting point to be accurate. The ending point is hard, if the skin doesn't check the EPG it will have to go on what the PVR tells it, and I have to base this answer on when "now" is. I'm not done with this analysis but wanted to ask you these questions.

What I have really been looking at is the drop/timeout conditions. I believe I have a bug here that I need to address. When simulating a drop-out I found that my code will go into a generally infinite loop trying to satisfy Kodi's read request. What it should do is decide at some point that there is no more data to be had and hand over what data is available to Kodi, or just send it zero bytes which will kill the stream after Kodi retries a couple times.

I have not yet duplicated a server-side network concern, but I watched the HDHomeRun DVR share's Live TV folder when the client is having a problem and found that the HDHomeRun DVR will keep the live stream file out there until it thinks the client(s) have disconnected. Since I'm sitting in a loop waiting for data, the connection is technically still open, and the file isn't going away until I kill Kodi.

Please let me know on the addon version as well as the platform / OS versions so I can be sure to replicate exactly what you are seeing skin-side to be sure I'm barking up the right tree. In the meantime I will see how the skins react to changes in the time(s) being reported.

No actions requested on the loop problem - I clearly see it and don't think it's a difficult fix. I just need to pick a method to use.

Thanks again for the time and effort you have been putting into this. Trying to get the software 'perfect' is the main priority, and from my chair you being banned from that thread wasn't necessary. :)

More as I have it.

gibxxi commented 5 years ago

Hi Mike,

At the point of writing this message, I have version 1.3.9 of the HDHomeRun PVR DVR add-on installed. Which was downloaded at the point you added the back-end channel names fix I believe.

All my systems are running Windows 10 Pro - RS-5 (October 2018 Update - Build: 17763.rs5_release.180914-1434). I'm running a registry hack that displays the version info on the desktop.

You do realise we've been communicating in the wrong thread all this time too don't you? Having said that, once we get a solution, I'd feel happier if these posts are deleted in preference to a "summary" in the right thread, as I've done a lot of waffling here, and divulged more details that I should of on a public, open platform such as this.

The version of Arctic Zephyr I'm running is BeatmasterRS' Leia Mod (At this time). I also have the vanilla Arctic Zephyr installed as well, now that Jurialmunkey has finally made it Leia compatible. It's not in the Kodi repo at this time though (neither are).

BeatmasterRS - Arctic Zephyr (Leia Mod): https://forum.kodi.tv/showthread.php?tid=337862

Jurialmunkey - Arctic Zephyr (Original Version - Updated for Leia): https://forum.kodi.tv/showthread.php?tid=217174

For Beatmaster's version, I'm using his repo for updates. For jurialmunkey's version, he doesn't provide one, so I'm syncing to the Kodi folder with a combination of GitHUB Desktop and SyncBack Pro. (I also use this combination to keep Audio Addict, Aeon Madnox, and Mr.V's Madnox 2.0 - Pre Alpha updated). I de-select the syncing of any GitHub related files/folders, and de-select all other add-ons that reside in the "portable_data/add-ons" folder so that SyncBack Pro won't attempt to remove them during sync (because they exist at the destination folder, but not at the source). The profile used in SyncBack Pro is the "Mirror" profile.

SyncBack Pro is available from here (30-day trial): https://www.2brightsparks.com/syncback/sbpro.html

Hope this helps.

:)

DM.

djp952 commented 5 years ago

I know it's technically the 'wrong' place, but there is little need to follow protocol here :) I do that stuff all day. My goal is to try to solve the problems, how that gets done isn't that important to me. I see no need to close it out. We did solve the actual problem for this issue a ways back, but let's keep it going.

I'll reinstall 1.3.9 tonight on both x86 and x64 Leia and see if that changes things, display-wise. That bug shouldn't have affected Windows builds regardless of architecture, the Visual C++ compiler behaved differently than GCC for that.

If it's OK I'm going to look at fixing what I found with never allowing a dead stream to close out first, as that will enable me to test the ability to actually retry connecting to the server/tuner again, which has been an issue for a number of folks. Rumor has it that the FireStick 4K still changes IP addresses every so often, which is really killing people since the DVR disconnects the client.

But really, no worries about where and when and what to post. You should see my incessant ramblings! Together we could fill our own forum! LOL

gibxxi commented 5 years ago

FWIW, I also tried the latest version, with no difference.

I have, at several points earmarked several add-ons that may be responsible for generating enough traffic to cause the problem, by renaming the "portable_data" folder, and adding add-ons back in one at a time, between runs, to attempt to identify a suspect.

Unfortunately, each time I think I have a winner, and remove it (and / or it's settings folder), it has no effect. Once there's breakage, it carries over between runs of Kodi, regardless of the presence of the suspect add-on or not and I have to start from scratch.

At this stage I've done a full wipe of the "portable_data" folder and will attempt a complete setup from scratch. The only issue with this is Kodi doesn't allow you to see / download dependencies directly any more, so installing / restoring those add-ons I sync via GitHub will be a problem, as side-loading them will not trigger their dependencies, and i'll have to check each one manually, and force-download them through the dependencies menu item.

Even though I plan to do this, i'm not confident it will improve the situation any. Ideally, going back to Krypton would be the ideal test, as I have run this add-on on Krypton (for a short time) but cannot remember much about the experience.

Dan / Gib.

gibxxi commented 5 years ago

I believe there's an issue with the Time-shift buffer and the HDHomeRun add-on and device.

Starting with only a bare-bones install of Kodi, plus the skins (and all dependencies) and nothing else, radio streams play back fine, until you attempt a time-shift / seek action. At that point you lose audio and the install is borked.

On this setup, for me, the only (installed) add-ons that have created addon_data folders are:

peripheral.joystick plugin.program.autocompletion pvr.hdhomerundvr script.module.metadatautils script.module.simplecache script.openweathermap.maps script.skinshortcuts skin.arctic.zephyr.mod skin.estuary weather.openweathermap.extended

Out of the above, the only add-ons I can imagine that would save any data relating to PVR, are pvr.hdhomerundvr and possibly script.module.simplecache. I've deleted the contained databases from both, and restarted Kodi, then attempted to re-tune the previous radio station. Usual scenario persists. I get about a second of audio, and then the stream quits.

Checking the tuner status will yield the following for that tuner on the HDHomeRun device's web page:

https://imgur.com/ZMshMHs

Attempting to do absolutely anything in Kodi from this point, will result in a freeze, as Kodi is waiting for packets from the HDHomeRun device it will never receive, meaning a force-close is required. Attempt a restart or reboot, and the same will happen, but only for radio shows.

So with the above being true, data about the current time-shift status must be being sent to SiliconDust's cloud servers, as is the case for recordings / timers. There's nothing that I can find in my Kodi install, that I can clear, that will result in a resolution of the issue. At this point in time, even renaming the addon_data folder is not making any difference.

If this was simply an add-on or buffer / cache issue, then clearing out the addon_data folder should do the trick. AFAIK Kodi won't store session / user specific data anywhere else, so the duration of the stream (which is also incorrect) must be coming from the HDHomeRun device or SiliconDust's cloud server. This data must be cleared at add-on start to prevent carry-overs from previous ill-fated runs IMHO.

It would seem that the way DAB radio is broadcast differs somewhat from the way the TV signal is broadcast. I do know that in the UK, the local antenna for your area (In my case, Mendip) does not broadcast FM/AM Radio. This is handled instead by a number of regional sub-transmitters in each town. Can't remember where I read this, but read it I did. It does however broadcast DAB radio. What I do not know, is how these streams are encapsulated. There must be some difference to the TV stations, otherwise there wouldn't be the performance difference we're currently seeing with TV vs. Radio channels.

However, there does seem to be a definite difference between DAB Radio (UK, Europe, etc.) and HD Radio (US). I found a page that mentions this specifically.

https://www.toptenreviews.com/electronics/articles/how-digital-radio-works/

All this does however assume that there isn't some kind of inherent problem with Kodi itself. As there are at least a half-dozen threads on the Kodi forums where people are having issues with (various) PVR setups, but at the same time, a major portion of these users also seem to be having issues on Android devices like the Nvidia Shield, specifically.

It might be the case, that until you start advertising this mod on the Kodi forums, your not going to get any Europeans using it to corroborate what I'm seeing. IMHO, the SiliconDust forums are too niche to give this add-on enough exposure (for those seeking to use it with Kodi, who also reside in the UK/EuroZone), again, IMHO.

That's all for now.

Dan / Gib.

EDIT: I think this is specific to the NextPVR PVR Add-on for Kodi, but the symptoms described seem to fit with what we're seeing. How this bears relevance, since your add-on makes no special provisions for radio channels (vs. TV channels) I cannot say, but worth a look in any case...

https://forum.kodi.tv/showthread.php?tid=340436

djp952 commented 5 years ago

There are a few competing factors at play here, and I think I have a build that will help figure out where things are going wrong, or at least maybe help finally point the finger into or out of the PVR.

First off, I can assure you that the PVR makes no changes to the installed state of Kodi whatsoever. Everything it does is private to itself, there are absolutely no changes being made to the Kodi configuration or other addons or anything. The data it uses is 100% dynamic, but it does cache it off in a small database to be able to limit the number of times it has to go ask for it. You can even delete the PVR database completely (there is an option for that too, but it's hard to find in the Kodi UI) and it just picks up where it left off since all of the data is completely volatile.

Now, as for the changes I've made …

There were a couple bugs to eliminate that would occur if the connection was completely dropped. The main one was that the PVR would allow Kodi to continue to try and read from a dead stream indefinitely. This would cause Kodi to pretty much lock up since the PVR would just sit there in a loop waiting for data that would never come. Fixed that one. There was also an inherant inability to detect a dropped stream, which exacerbated the first problem quite a bit. That is also (apparently) fixed, but there is a side-effect. Instead of waiting forever for something that will never happen, a dropped/errored connection has to be reported via the logs and a notification banner only - Kodi does not obey it's own API and will continue to try and read until the stream gives it back a few zeros, it ignores the "error happened" result. It's also not immediate - I had to tell cURL to stop and error out if it sees less than 1 byte per second over a 5 second sample period. In testing, this ended up more like 6-8 seconds before cURL would error out for me, and it has to be a complete drop -- either the client or the server has to be good and dead.

Resuming a dropped stream is seemingly impossible with Kodi, it has to close the stream on it's own and start it up again (which it doesn't do). Truly resetting things and starting over with the same URL sends Kodi into la-la land -- all the metadata and properties it had are now invalid. I tried this in response to the theory that the PVR was reporting some manner of "all time" data, but that just isn't the case. Every stream is unique and all properties/metadata about it are calculated when the stream is opened.

For the special build I'm going to link to below, I added some extra logging so we can find out for certain where the weird times come from. It has a mainline change that will now log the properties of a new stream (Live or Recorded) when you start it that we can examine, but I also added one-off logging to periodically report the times being reported to Kodi via the new Leia API call, and they will be reported as both Unix epoch (time_t) and converted into readble local time. There is also a special set of logging for Live TV seek operations -- this build will report the current stream position when seek was called, and what it ended up at. These are byte positions, so I will need to compare what Kodi asked for with what it will get to see if there is anything weird there. I don't think there will be, a problem in seek would affect everyone.

If you get a timeout or some other HTTP/cURL error with this test build, the stream will stop completely and you will get a banner notification. Remember it will take a few seconds, but once a stream is dead it will be truly dead. This behavior change applies to both seek and read operations.

So when you can, if you wouldn't mind installing this build on a system and making it break, there should be enough evidence in Kodi log to determine what the next step will be :)

Link to test build (all platforms, you can use whatever one you'd like): https://1drv.ms/f/s!AgEGEEVzGNq-i9Usd4MUjS-3WjooDQ Let me know if you have trouble accessing it, it's just shared from OneDrive

I hope this will resolve enough and provide enough evidence to keep going!!

gibxxi commented 5 years ago

Hi Mike,

Gave it a try, but no change in behaviour from before within the Kodi UI. I've attached the Kodi log from that run for you to examine (minus advancedsettings.xml section, as this contains personal data, paths, passwords, etc.).

https://1drv.ms/u/s!Am5WO9nIAax8gZRcm1AmgW4qlM0yrA

Dan / Gib.