New build has lost XMLTV internal grabber for Schedules Direct

[Gronda74]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

Since installing the latest build of e2ae8f4e-ls121 the XMLTV internal grabber for Schedules Direct JSON API has vanished.

Expected Behavior

I expect to see an XMLTV internal grabber for Schedules Direct JSON API as in the previous build 09a2c71a-ls116.

Steps To Reproduce

1. Install build e2ae8f4e-ls121

Environment

- OS:
- How docker service was installed:

CPU architecture

x86-64

Docker creation

HOME    /config
LSIO_FIRST_PARTY    true
PATH    /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PGID    100
PS1 $(whoami)@$(hostname):$(pwd)\$
PUID    1029
S6_CMD_WAIT_FOR_SERVICES_MAXTIME    0
S6_STAGE2_HOOK  /docker-mods
S6_VERBOSITY    1
TERM    xterm
TZ  Europe/London
VIRTUAL_ENV /lsiopy

Container logs

2023-05-21 11:31:36.589 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S09E15 The Petrified Florist.TV.mkv'
2023-05-21 11:31:36.592 [  ERROR] dvr: unable to stat file '/recordings/Grand Designs/Grand Designs - S21E03 - South Lincolnshire.mkv'
2023-05-21 11:31:36.594 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S06E11 - The M_A_S_H Olympics-1.mkv'
2023-05-21 11:31:36.595 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother - S02E18 - Moving Day.mkv'
2023-05-21 11:31:36.597 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother - S09E13 - Bass Player Wanted.mkv'
2023-05-21 11:31:36.598 [  ERROR] dvr: unable to stat file '/recordings/The Best of Top Gear/The Best of Top Gear Series 29 and 30.TV.mkv'
2023-05-21 11:31:36.600 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother - S09E02 - Coming Back.mkv'
2023-05-21 11:31:36.601 [  ERROR] dvr: unable to stat file '/recordings/Hunt for the Wilderpeople/Hunt for the Wilderpeople - -.mkv'
2023-05-21 11:31:36.603 [  ERROR] dvr: unable to stat file '/recordings/Man With a Plan/Man With a Plan - S02E11 - Guess Who's Coming to Breakfast, Lunch and Dinner-1.mkv'
2023-05-21 11:31:36.605 [  ERROR] dvr: unable to stat file '/recordings/Professor Marston and the Wonder Women/Professor Marston and the Wonder Women [720].mkv'
2023-05-21 11:31:36.606 [  ERROR] dvr: unable to stat file '/recordings/The Simpsons/The Simpsons S29E14 Fears of a Clown - 720.TV.mkv'
2023-05-21 11:31:36.608 [  ERROR] dvr: unable to stat file '/recordings/Devs/Devs - S01E06 - Episode 6.mkv'
2023-05-21 11:31:36.614 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S06E11 Town Father.TV.mkv'
2023-05-21 11:31:36.617 [  ERROR] dvr: unable to stat file '/recordings/Top Gear/Top Gear - --2.mp4'
2023-05-21 11:31:36.620 [  ERROR] dvr: unable to stat file '/recordings/Automata/Automata [720].mkv'
2023-05-21 11:31:36.622 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother - S06E07 - Canning Randy.mkv'
2023-05-21 11:31:36.624 [  ERROR] dvr: unable to stat file '/recordings/Planet of the Apes/Planet of the Apes - 720.TV.mkv'
2023-05-21 11:31:36.625 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S07E17 - Inga.mkv'
2023-05-21 11:31:36.630 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S07E16 From the Horse's Mouth.TV-1.mkv'
2023-05-21 11:31:36.633 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S06E07 - The Light That Failed.mkv'
2023-05-21 11:31:36.636 [  ERROR] dvr: unable to stat file '/recordings/Galactica 1980/Galactica 1980 S01E06 Spaceball.TV.mkv'
2023-05-21 11:31:36.638 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother - S01E11 - The Limo.mkv'
2023-05-21 11:31:36.639 [  ERROR] dvr: unable to stat file '/recordings/Rebecca/Rebecca.TV.mkv'
2023-05-21 11:31:36.641 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H S01E05 The Moose.TV-1.mkv'
2023-05-21 11:31:36.643 [  ERROR] dvr: unable to stat file '/recordings/The Day of the Locust/The Day of the Locust.mkv'
2023-05-21 11:31:36.646 [  ERROR] dvr: unable to stat file '/recordings/Breaking In/Breaking In - 720.TV.mkv'
2023-05-21 11:31:36.650 [  ERROR] dvr: unable to stat file '/recordings/Airwolf/Airwolf S02E22 Short Walk to Freedom.TV-1.mkv'
2023-05-21 11:31:36.652 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S05E17 - Ping Pong.mkv'
2023-05-21 11:31:36.655 [  ERROR] dvr: unable to stat file '/recordings/Battlestar Galactica/Battlestar Galactica S01E24 The Hand of God.TV.mkv'
2023-05-21 11:31:36.657 [  ERROR] dvr: unable to stat file '/recordings/Grand Designs/Grand Designs S17E01 - 720.TV.mkv'
2023-05-21 11:31:36.660 [  ERROR] dvr: unable to stat file '/recordings/Battlestar Galactica/Battlestar Galactica S04E05 The Road Less Traveled.TV.mkv'
2023-05-21 11:31:36.665 [  ERROR] dvr: unable to stat file '/recordings/This Time with Alan Partridge/This Time with Alan Partridge S02E06 - 720.TV.mkv'
2023-05-21 11:31:36.666 [  ERROR] dvr: unable to stat file '/recordings/Louis Theroux/Louis Theroux S07E02 - Life on the Edge - The Dark Side of Pleasure [720].mkv'
2023-05-21 11:31:36.668 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S05E19 Double Exposure.TV.mkv'
2023-05-21 11:31:36.669 [  ERROR] dvr: unable to stat file '/recordings/Battlestar Galactica/Battlestar Galactica S01E02 Saga of a Star World.TV-1.mkv'
2023-05-21 11:31:36.671 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S08E09 The Committee.TV.mkv'
2023-05-21 11:31:36.673 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S07E09 Ballad for a Blue Lady.TV-1.mkv'
2023-05-21 11:31:36.674 [  ERROR] dvr: unable to stat file '/recordings/All Creatures Great and Small/All Creatures Great and Small - S01E03 - Andante.mkv'
2023-05-21 11:31:36.676 [  ERROR] dvr: unable to stat file '/recordings/Top Gear/Top Gear Top Fails, Volume 1.TV.mkv'
2023-05-21 11:31:36.681 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother S01E07 Matchmaker.TV.mkv'
2023-05-21 11:31:36.684 [  ERROR] dvr: unable to stat file '/recordings/What We Do in the Shadows/What We Do in the Shadows S03E08 The Wellness Center - 720.TV.mkv'
2023-05-21 11:31:36.685 [  ERROR] dvr: unable to stat file '/recordings/Crawl/Crawl - 720.TV.mkv'
2023-05-21 11:31:36.687 [  ERROR] dvr: unable to stat file '/recordings/Top Gear/Top Gear - S28E03 -.mkv'
2023-05-21 11:31:36.689 [  ERROR] dvr: unable to stat file '/recordings/Star Trek_ Enterprise/Star Trek_ Enterprise S01E16 Fusion.TV.mkv'
2023-05-21 11:31:36.690 [  ERROR] dvr: unable to stat file '/recordings/The Cleaner/The Cleaner S01E05 The Influencer - 720.TV.mkv'
2023-05-21 11:31:36.695 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S02E05 - Dr Pierce and Mr Hyde.mkv'
2023-05-21 11:31:36.700 [  ERROR] dvr: unable to stat file '/recordings/Murder, She Wrote/Murder, She Wrote S07E19 Thursday's Child.TV-1.mkv'
2023-05-21 11:31:36.703 [  ERROR] dvr: unable to stat file '/recordings/The Deer Hunter/The Deer Hunter [720].mkv'
2023-05-21 11:31:36.704 [  ERROR] dvr: unable to stat file '/recordings/How I Met Your Mother/How I Met Your Mother - S01E09 - Belly Full of Turkey.mkv'
2023-05-21 11:31:36.708 [  ERROR] dvr: unable to stat file '/recordings/The Old Man & the Gun/The Old Man & the Gun.TV.mkv'
2023-05-21 11:31:36.711 [  ERROR] dvr: unable to stat file '/recordings/Battlestar Galactica/Battlestar Galactica - S02E16 - Sacrifice.mkv'
2023-05-21 11:31:36.712 [  ERROR] dvr: unable to stat file '/recordings/Mock the Week/Mock the Week - S12E01 -.mkv'
2023-05-21 11:31:36.714 [  ERROR] dvr: unable to stat file '/recordings/Baptiste/Baptiste S02E02 - 720.TV.mkv'
2023-05-21 11:31:36.716 [  ERROR] dvr: unable to stat file '/recordings/NCIS/NCIS - -.mkv'
2023-05-21 11:31:36.717 [  ERROR] dvr: unable to stat file '/recordings/Itch/Itch - S01E08 - Friends and Enemies.mkv'
2023-05-21 11:31:36.719 [  ERROR] dvr: unable to stat file '/recordings/Stieg Larssons Millennium/Stieg Larssons Millennium S01E02 The Girl With The Dragon Tattoo Part 2 - 720.TV.mkv'
2023-05-21 11:31:36.720 [  ERROR] dvr: unable to stat file '/recordings/Star Trek_ Enterprise/Star Trek_ Enterprise S01E21 Vox Sola.TV.mkv'
2023-05-21 11:31:36.722 [  ERROR] dvr: unable to stat file '/recordings/Top Gear/Top Gear S30E01 - 720.TV.mkv'
2023-05-21 11:31:36.724 [  ERROR] dvr: unable to stat file '/recordings/Airwolf/Airwolf S02E10 Once A Hero.TV-1.mkv'
2023-05-21 11:31:36.725 [  ERROR] dvr: unable to stat file '/recordings/Battlestar Galactica/Battlestar Galactica S01E21 Baltar's Escape.TV.mkv'
2023-05-21 11:31:36.728 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S08E02 - Are You Now, Margaret_.mkv'
2023-05-21 11:31:36.733 [  ERROR] dvr: unable to stat file '/recordings/Fred Claus/Fred Claus [720].mkv'
2023-05-21 11:31:36.735 [  ERROR] dvr: unable to stat file '/recordings/All Creatures Great & Small/All Creatures Great & Small - S01E05 - Out of Practice.mkv'
2023-05-21 11:31:36.736 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S04E05 - The Late Captain Pierce.mkv'
2023-05-21 11:31:36.738 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S04E22 - Smilin' Jack.mkv'
2023-05-21 11:31:36.746 [  ERROR] dvr: unable to stat file '/recordings/Paris Police 1900/Paris Police 1900 S01E02 - 720.TV.mkv'
2023-05-21 11:31:36.747 [  ERROR] dvr: unable to stat file '/recordings/Star Trek_ Enterprise/Star Trek_ Enterprise S03E04 Rajiin.TV.mkv'
2023-05-21 11:31:36.752 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S01E23 - Cease-Fire.mkv'
2023-05-21 11:31:36.759 [  ERROR] dvr: unable to stat file '/recordings/What We Do in the Shadows/What We Do in the Shadows - S02E05 - Colin's Promotion.mkv'
2023-05-21 11:31:36.760 [  ERROR] dvr: unable to stat file '/recordings/Deceit/Deceit S01E03 - 720.TV.mkv'
2023-05-21 11:31:36.766 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S04E19 - Hawkeye.mkv'
2023-05-21 11:31:36.767 [  ERROR] dvr: unable to stat file '/recordings/Galactica 1980/Galactica 1980 S01E09 Space Croppers.TV-1.mkv'
2023-05-21 11:31:36.770 [  ERROR] dvr: unable to stat file '/recordings/Hellraiser/Hellraiser - -.mkv'
2023-05-21 11:31:36.772 [  ERROR] dvr: unable to stat file '/recordings/NCIS/NCIS - S14E24 - Rendezvous.mkv'
2023-05-21 11:31:36.774 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H S10E03 Identity Crisis.TV-1.mkv'
2023-05-21 11:31:36.778 [  ERROR] dvr: unable to stat file '/recordings/Apocalypse Now/Apocalypse Now [720].mkv'
2023-05-21 11:31:36.782 [  ERROR] dvr: unable to stat file '/recordings/Anchorman 2_ The Legend Continues/Anchorman 2 - The Legend Continues [720].mkv'
2023-05-21 11:31:36.786 [  ERROR] dvr: unable to stat file '/recordings/Spy/Spy - 720.TV.mkv'
2023-05-21 11:31:36.792 [  ERROR] dvr: unable to stat file '/recordings/Battlestar Galactica/Battlestar Galactica S04E21 Daybreak - 720.TV.mkv'
2023-05-21 11:31:36.802 [  ERROR] dvr: unable to stat file '/recordings/The Handmaid's Tale/The Handmaid's Tale S02E09 Smart Power.TV.mkv'
2023-05-21 11:31:36.805 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H S07E06 The Billfold Syndrome.TV.mkv'
2023-05-21 11:31:36.808 [  ERROR] dvr: unable to stat file '/recordings/Pirates of the Caribbean_ Salazar's Revenge/Pirates of the Caribbean_ Salazar's Revenge - -.mkv'
2023-05-21 11:31:36.811 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H S11E07 Settling Debts.TV-1.mkv'
2023-05-21 11:31:36.813 [  ERROR] dvr: unable to stat file '/recordings/NCIS/NCIS S16E12 The Last Link.TV.mkv'
2023-05-21 11:31:36.815 [  ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S05E24 - Post Op.mkv'
2023-05-21 11:31:37.097 [   INFO] dvr: Purging obsolete autorec entries for current schedule
2023-05-21 11:31:37.098 [ NOTICE] START: HTS Tvheadend version 4.3-2120~g18effa8ad started, running as PID:143 UID:1029 GID:100, CWD:/run/s6-rc:s6-rc-init:BJaLGI/servicedirs/svc-tvheadend CNF:/config
2023-05-21 11:31:37.100 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)
2023-05-21 11:31:37.461 [   INFO] scanfile: DVB-S - loaded 1 regions with 116 networks
2023-05-21 11:31:37.461 [   INFO] scanfile: DVB-T - loaded 46 regions with 1134 networks
2023-05-21 11:31:37.461 [   INFO] scanfile: DVB-C - loaded 20 regions with 82 networks
2023-05-21 11:31:37.461 [   INFO] scanfile: ATSC-T - loaded 2 regions with 13 networks
2023-05-21 11:31:37.461 [   INFO] scanfile: ATSC-C - loaded 1 regions with 5 networks
2023-05-21 11:31:37.461 [   INFO] scanfile: ISDB-T - loaded 2 regions with 1297 networks
2023-05-21 12:31:26.346 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)
2023-05-21 13:31:26.169 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)
2023-05-21 14:31:26.331 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)
2023-05-21 15:31:26.213 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)
2023-05-21 16:31:26.987 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)
2023-05-21 17:31:26.554 [   INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68)

[github-actions[bot]]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[thespad]

It looks like the Alpine repo build doesn't enable it https://build.alpinelinux.org/buildlogs/build-3-18-x86_64/community/xmltv/xmltv-1.2.1-r0.log

The simplest fix would be to open an issue against the package and ask them to build it with the SD JSON API enabled; they're usually pretty responsive. If they can't/won't change the build, we might be able to look at reverting to building it ourselves but we stopped because it was causing us issues and so I'd rather avoid it if we can.

In the meantime you can pin to 09a2c71a-ls116 so that you don't lose any functionality.

[Gronda74]

Thanks. I've put the request in and it looks like this has been done so hopefully the next time this build updates it will be there too :)

[Gronda74]

Thanks for pushing the updated build through. The internal grabber is now there but having enabled it when I run the grabber I get the following error:

2023-05-22 08:35:41.623 [ ERROR] dvr: unable to stat file '/recordings/M_A_S_H/M_A_S_H - S05E24 - Post Op.mkv' 2023-05-22 08:35:41.900 [ INFO] dvr: Purging obsolete autorec entries for current schedule 2023-05-22 08:35:41.901 [ NOTICE] START: HTS Tvheadend version 4.3-2120~g18effa8ad started, running as PID:142 UID:1029 GID:100, CWD:/run/s6-rc:s6-rc-init:chIIEe/servicedirs/svc-tvheadend CNF:/config 2023-05-22 08:35:41.903 [ INFO] iptv: m3u parse: 0 new mux(es) in network 'TV' (total 68) 2023-05-22 08:35:44.011 [ INFO] scanfile: DVB-S - loaded 1 regions with 116 networks 2023-05-22 08:35:44.011 [ INFO] scanfile: DVB-T - loaded 46 regions with 1134 networks 2023-05-22 08:35:44.011 [ INFO] scanfile: DVB-C - loaded 20 regions with 82 networks 2023-05-22 08:35:44.011 [ INFO] scanfile: ATSC-T - loaded 2 regions with 13 networks 2023-05-22 08:35:44.011 [ INFO] scanfile: ATSC-C - loaded 1 regions with 5 networks 2023-05-22 08:35:44.011 [ INFO] scanfile: ISDB-T - loaded 2 regions with 1297 networks 2023-05-22 08:37:32.000 [ INFO] xmltv: /usr/bin/tv_grab_zz_sdjson: grab /usr/bin/tv_grab_zz_sdjson 2023-05-22 08:37:32.002 [ INFO] spawn: Executing "/usr/bin/tv_grab_zz_sdjson" 2023-05-22 08:37:33.155 [ ERROR] spawn: Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl) at /usr/bin/tv_grab_zz_sdjson line 57. 2023-05-22 08:37:33.155 [ ERROR] spawn: BEGIN failed--compilation aborted at /usr/bin/tv_grab_zz_sdjson line 57. 2023-05-22 08:37:33.164 [ ERROR] xmltv: /usr/bin/tv_grab_zz_sdjson: no output detected 2023-05-22 08:37:33.164 [WARNING] xmltv: /usr/bin/tv_grab_zz_sdjson: grab returned no data

[thespad]

Can you try docker exec tvheadend apk add perl-json and then test again. You may also need to install perl-json-xs, but try without first.

If that fixes it I can sort out a PR to add the package(s) to the image.

[Gronda74]

Where do I try that? Sorry, I've got very limited experience of docker / linux. I'm using docker as pre-installed on my Synology NAS. I do have Portainer running on that but when I use Portainer to connect to the console for the tvheadend package and type that in I get an error message of bash: docker: command not found

[thespad]

If you're connecting to the container console then it's just apk add perl-json

[Gronda74]

Fantastic, that's fixed it. I ran the command apk add perl-json and got the result as follows:

fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/1) Installing perl-json (4.10-r0)
OK: 379 MiB in 268 packages

From there I ran the grabber and it duly grabbed and processed the data.

[thespad]

OK great, I'll knock up a PR to add the package to the image.

[the-jeffski]

Just to note, I found this issue when my guide data dropped. I noticed this added in the SD JSON API but we still missed the tv_grab_zz_sdjson_sqlite version. Requested to be added to the alpine package which is now done, so next time this image is rebuilt I assume it will also pull through.