Missing channels in Guide

[jcastilloalonso]

Hi,

I noticed that the amount of channels that are shown in Guide is not complete. Guide has 36 channels while Locast shows 44 available channels. Not sure what I can provide you to identify why some are missing.

Thanks!

[tharris9d]

I just tested zap2it and I'm getting tomorrow in the xmltv date. http://127.0.0.1:5006/api/xmltv?method=get&source=zap2it <programme start="20201211010000 +0000" stop="20201211043000 +0000" Ram's game

[deathbybandaid]

Just going to talk my way through this:

1) we pull data from https://api.locastnet.org/api/watch/epg/DMA_NUMBER_HERE

2) This is a list of dictionaries, the dictionaries contain channel information (same place we get channel info to begin with)

3) each channel dictionary then has a list of event/program dictionaries listings.

4) each listing has an epoch timestamp startTime and a duration in seconds (endtime is calculated from this info as well)

5) epoch timestamps should be UTC, and are then converted to xmltv format. "%Y%m%d%H%M%S +0000"

[tharris9d]

I got tvtv working but the most of the content thumbnails are not coming through in the webui.

[deathbybandaid]

yeah, tvtv still needs some work

[jcastilloalonso]

Sorry don't know python so can't help much, thinking on logic wise, there might be a mistake in the calculation from UTC that does not occur in your timezone. Is there any dump I can provide you to analyze?

[deathbybandaid]

not really,,, there shouldn't be any time conversion whatsoever

[jcastilloalonso]

If epoch timestamps are UTC, how do we know what time to show as current?

[deathbybandaid]

https://www.unixtimestamp.com/index.php

[deathbybandaid]

if you copy a start time from Locast API into there, you'll get the UTC time output

[jcastilloalonso]

right, then why does it show the future shows as present? I'll try to dig into this this evening.

[deathbybandaid]

Plex/Emby should be the only thing doing any time conversion

[jcastilloalonso]

How about the web guide in fHDHR?

[deathbybandaid]

do either of you use another fHDHR varient? can you see if those schedules look fine?

[deathbybandaid]

web guide simply gets uses utcnow() to find what's currently on, intentionally not listing a local time

[deathbybandaid]

it loops through the listings per channel and finds the first entry that meets the criteria if start_time <= nowtime <= end_time:

[deathbybandaid]

I'm also finding that Locast misses a few channels in their EPG, so I'm coding in some Blocks scheduling to fill it in

[deathbybandaid]

Looks like Locast has channels with EPG data, but not for every hour?

[tharris9d]

http://127.0.0.1:5006/guide?source=tvtv Missing some thumbnails but the Content Title and Description are accurate. http://127.0.0.1:5006/guide?source=origin Origin is still 8 hours off.

[deathbybandaid]

The hard thing to diagnose is that I changed nothing about how the timestamps work since a prior build.

If you install the last release, does it work?

[tharris9d]

I updated fHDHR_PlutoTV to the latest Main version and the web ui guide is correct. I compared with the PlutoTV Roku app.

[deathbybandaid]

Okay guys, here's some interesting development,,, I'm coding in the ability to detect missing timeslots, and generate Program information for those times. I also coded in some stuff to create block scheduling for missing channels.

here's where it gets interesting. I added some debug print lines for my testing, and here's what happens with my Locast New York DMA 501, 49 channels:

[deathbybandaid]

I think I'm going to change some EPG handling stuff, and store epoch style timestamps instead of datetime style timestamps, and do the conversion to xmltv format just for the xmltv generation

[jcastilloalonso]

http://127.0.0.1:5006/guide?source=tvtv Missing some thumbnails but the Content Title and Description are accurate. http://127.0.0.1:5006/guide?source=origin Origin is still 8 hours off.

I actually get an 500 internal Server Error using tvtv. zap2it only shows 6 channels.

Okay guys, here's some interesting development,,, I'm coding in the ability to detect missing timeslots, and generate Program information for those times. I also coded in some stuff to create block scheduling for missing channels.

here's where it gets interesting. I added some debug print lines for my testing, and here's what happens with my Locast New York DMA 501, 49 channels:

That looks weird... I like your idea of moving the conversion over to xmltv generation. Hope it helps

[deathbybandaid]

just a thought,,, if you set your server time to UTC, what happens?

[deathbybandaid]

if that fixes it, I may try to find something to try to override system time with UTC

[tharris9d]

FYI: My Windows 10 server is set to UTC Pacific. Origin is still getting 8 hours off. 8 hours is weird.

[jcastilloalonso]

I actually get an 500 internal Server Error using tvtv. zap2it only shows 6 channels.

Here's the exception I got when trying tvtv:

fHDHR - ERROR - Exception on /guide [GET] Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/opt/fHDHR_Locast/fHDHR_web/pages/guide_html.py", line 15, in call return self.get(*args) File "/opt/fHDHR_Locast/fHDHR_web/pages/guide_html.py", line 28, in get for channel in self.fhdhr.device.epg.whats_on_allchans(source): File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 88, in whats_on_allchans epgdict = self.get_epg(method) File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 117, in get_epg self.update(method) File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 177, in update programguide = self.epg_handling[method].update_epg() File "/opt/fHDHR_Locast/alternative_epg/tvtv/init.py", line 52, in update_epg cached_items = self.get_cached(dates_to_pull) File "/opt/fHDHR_Locast/alternative_epg/tvtv/init.py", line 112, in get_cached self.get_cached_item(str(datesdict["start"]), url) File "/opt/fHDHR_Locast/alternative_epg/tvtv/init.py", line 128, in get_cached_item result = resp.json() File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 900, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/lib/python3.7/json/init.py", line 348, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[jcastilloalonso]

if that fixes it, I may try to find something to try to override system time with UTC

Switching to UTC fixed origin for me. tvtv still sending the same Internal Error 500.

$ timedatectl Local time: Fri 2020-12-11 12:33:23 CST Universal time: Fri 2020-12-11 18:33:23 UTC RTC time: Fri 2020-12-11 18:33:23 Time zone: America/Chicago (CST, -0600) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: no Last DST change: DST ended at Sun 2020-11-01 01:59:59 CDT Sun 2020-11-01 01:00:00 CST Next DST change: DST begins (the clock jumps one hour forward) at Sun 2021-03-14 01:59:59 CST Sun 2021-03-14 03:00:00 CDT $ timedatectl set-timezone UTC $ timedatectl Local time: Fri 2020-12-11 18:34:02 UTC Universal time: Fri 2020-12-11 18:34:02 UTC RTC time: Fri 2020-12-11 18:34:02 Time zone: UTC (UTC, +0000) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: n/a

[deathbybandaid]

@tharris9d in your config set

[tvtv]
postalcode = 
lineuptype = ota

@jcastilloalonso

[jcastilloalonso]

Changing timezone requires fHDHR to be restarted for guide to be fixed. Otherwise, it stays broken even if I purge cache.

[jcastilloalonso]

@tharris9d in your config set

[tvtv]
postalcode = 
lineuptype = ota

@jcastilloalonso

Fixed tvtv, derp on my end... tvtv shows correct show in both timezones.

[deathbybandaid]

it's on my list to figure out hos to manage the threads from the webUI, so that you can completely manage from the webUI, which is one of the key factors in reaching version 1.0

[deathbybandaid]

Guys I'm about to drop a major overhaul of the EPG system, and it may fix all the issue, or at least some of them

[deathbybandaid]

pushed some sweet sweet code

[deathbybandaid]

turns out some of my code to dedupe entries was potentially part of the problem, and I've implemented something to counter that

[deathbybandaid]

If the most recent push doesn't fix your issues with offset, I'll look into a way for the script environment to lock to UTC

[jcastilloalonso]

Nice work! Just pulled and restarted and it's day and night, channel order fixed, correct time regardless of timezone (tried both) for origin, yet zap2it and tvtv sending 500 Internal Error. http://plexserver:5009/guide?source=zap2it:

fHDHR - ERROR - Exception on /guide [GET] Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/opt/fHDHR_Locast/fHDHR_web/pages/guide_html.py", line 15, in call return self.get(*args) File "/opt/fHDHR_Locast/fHDHR_web/pages/guide_html.py", line 47, in get whatson = self.fhdhr.device.epg.whats_on_allchans(source) File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 99, in whats_on_allchans whatson = self.whats_on_now(channel_number, method) File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 70, in whats_on_now if listing["time_start"] <= nowtime <= listing["time_end"]: TypeError: '<=' not supported between instances of 'str' and 'float'

http://plexserver:5009/guide?source=tvtv : fHDHR - ERROR - Exception on /guide [GET] Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/opt/fHDHR_Locast/fHDHR_web/pages/guide_html.py", line 15, in call return self.get(*args) File "/opt/fHDHR_Locast/fHDHR_web/pages/guide_html.py", line 47, in get whatson = self.fhdhr.device.epg.whats_on_allchans(source) File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 99, in whats_on_allchans whatson = self.whats_on_now(channel_number, method) File "/opt/fHDHR_Locast/fHDHR/device/epg/init.py", line 70, in whats_on_now if listing["time_start"] <= nowtime <= listing["time_end"]: TypeError: '<=' not supported between instances of 'str' and 'float'

[deathbybandaid]

push the update button for those on the xmltv page

[jcastilloalonso]

push the update button for those on the xmltv page

And that fixed it, didn't think of the cache after a rebuild haha

Now tvtv is going to the past :(

Channel Name	Channel Number	Channel Thumbnail	Content Title	Content Thumbnail	Content Description	Start Time (UTC)	End Time (UTC)	Content Remaining Time
Daystar (KDTN) Dallas, TX	2.1		Kenneth Copeland Ministries		The Kenneth Copeland Ministries teach the Christian principles of the Bible including faith, hope and love.	2020-12-11 15:00:00	2020-12-11 15:30:00	364 days 18 hours 11 seconds

[deathbybandaid]

Zap and origin are good though?

[jcastilloalonso]

Zap and origin are good though?

Origin good, Zap and tvtv also wrong with America/Chicago timezone, UTC timezone fixes it.

Content Remaining Time calculation is wrong for all 3

[tharris9d]

Origin is all correct now in the latest main version web ui. Zap2it and TVTV are missing thumbs titles and descriptions. Time zone is now correct. Thanks! I will test in Plex now too.

[tharris9d]

Using Plex the channels all load using http://127.0.0.1:5005/api/xmltv?method=get&source=origin as my xmltv link. However the guide doesn't load in Plex. 0 information. I tried restarting the Plex Server, clearing cache, updating, rebooting.

[deathbybandaid]

Try deleting the dvr in plex, restart plex, refresh browser, and readd

[deathbybandaid]

anybody want to test fHDHR/fHDHR_Locast#132 with a fresh db?

[jcastilloalonso]

anybody want to test fHDHR/fHDHR_Locast#132 with a fresh db?

Origin still working correctly except the Content Remaining Time still showing pretty much a yr. tvtv still showing 8hrs ago content zap2it not pulling anything from guide, just channels. I have the following in my config.ini for it:

[zap2it] postalcode = 750xx (do have full zip code just took off the last 2 for privacy, but can provide if needed)

[tharris9d]

Tested new code. Origin pulls channels into plex but nothing shows on Plex Guide. Everything shows fine in the fHDHR_locast web-ui. @jcastilloalonso are all the origin channels with content showing up in your plex guide? Using the http://127.0.0.1:port/api/xmltv?method=get&source=origin link?

I deleted db, restarted app, restarted plex, cleared cache, updated.

[jcastilloalonso]

@jcastilloalonso are all the origin channels with content showing up in your plex guide? Using the http://127.0.0.1:port/api/xmltv?method=get&source=origin link?

Yeah it seems to be working over here

[tharris9d]

I have tried everything but the channels with airing data will not load into Plex using the latest version. It starts with 0% then just finishes without loading data. When you hit view guide nothing is there. I tried downloading the xml file and loading it directly with the exact same results. I then tried an older release 4.5 and it works fine except for the 8 hour time difference. I zipped up both xml files to see if something is different. Please take a look. FYI: Everything works fine in the Web UI. locast_xmltv.zip

[deathbybandaid]

@tharris9d I really don't know what to do to fix it for you. Try ALL of the below steps, IN ORDER, and report back

1) Delete DVR in plex 2) stop your plex service

3) stop fHDHR_Locast 4) Delete database file

4.5) Make sure you have the newest version of the code

5) start fHDHR_Locast 6) start plex service

7) add the dvr to plex