epg-grabber stops without saying why

[benbelgium]

Hi

When I'm grabbing telenet.tv the grabbing suddenly stops without telling why.... I do first get a cookie error, and 'usually' it doesn't take long to crash afterwards, but I can't say that cookie error is the problem, as sometimes it also continues after this error (see below log) I started running the program with a log file and a log-level=debug but this didn't add anything extra (I also tried level=trace but then the program didn't want to start, so I don't thing trace exists...)

[2023-01-15T16:27:48.441Z] INFO: Starting... [2023-01-15T16:27:48.442Z] INFO: Loading 'sites/telenet.tv/telenet.tv.config.js'... [2023-01-15T16:27:48.444Z] INFO: Loading 'sites/telenet.tv/telenet.tv.channels.xml'... [2023-01-15T16:27:56.563Z] INFO: [1/728] telenet.tv - 2MInternational.ma - Jan 15, 2023 (42 programs) [2023-01-15T16:28:04.045Z] INFO: [2/728] telenet.tv - 2MInternational.ma - Jan 16, 2023 (49 programs) [2023-01-15T16:28:12.418Z] INFO: [3/728] telenet.tv - 2MInternational.ma - Jan 17, 2023 (49 programs) [2023-01-15T16:28:20.882Z] INFO: [4/728] telenet.tv - 2MInternational.ma - Jan 18, 2023 (50 programs) [2023-01-15T16:28:27.958Z] INFO: [5/728] telenet.tv - 2MInternational.ma - Jan 19, 2023 (49 programs) [2023-01-15T16:28:35.410Z] INFO: [6/728] telenet.tv - 2MInternational.ma - Jan 20, 2023 (48 programs) [2023-01-15T16:28:42.760Z] INFO: [7/728] telenet.tv - 2MInternational.ma - Jan 21, 2023 (43 programs) [2023-01-15T16:28:49.568Z] INFO: [8/728] telenet.tv - AB3.be - Jan 15, 2023 (28 programs) [2023-01-15T16:28:52.755Z] INFO: [9/728] telenet.tv - AB3.be - Jan 16, 2023 (0 programs) [2023-01-15T16:28:52.755Z] ERROR: Cookie not in this host's domain. Cookie:horizon.tv Request:static.spark.telenet.tv [2023-01-15T16:28:58.985Z] INFO: [10/728] telenet.tv - AB3.be - Jan 17, 2023 (30 programs) [2023-01-15T16:29:05.311Z] INFO: [11/728] telenet.tv - AB3.be - Jan 18, 2023 (31 programs) [2023-01-15T16:29:12.223Z] INFO: [12/728] telenet.tv - AB3.be - Jan 19, 2023 (25 programs) [2023-01-15T16:29:17.878Z] INFO: [13/728] telenet.tv - AB3.be - Jan 20, 2023 (32 programs) [2023-01-15T16:29:23.996Z] INFO: [14/728] telenet.tv - AB3.be - Jan 21, 2023 (33 programs) [2023-01-15T16:29:30.119Z] INFO: [15/728] telenet.tv - AlJazeeraEnglish.qa - Jan 15, 2023 (40 programs) [2023-01-15T16:29:36.185Z] INFO: [16/728] telenet.tv - AlJazeeraEnglish.qa - Jan 16, 2023 (40 programs) [2023-01-15T16:29:43.375Z] INFO: [17/728] telenet.tv - AlJazeeraEnglish.qa - Jan 17, 2023 (39 programs) [2023-01-15T16:29:50.423Z] INFO: [18/728] telenet.tv - AlJazeeraEnglish.qa - Jan 18, 2023 (39 programs) [2023-01-15T16:29:56.486Z] INFO: [19/728] telenet.tv - AlJazeeraEnglish.qa - Jan 19, 2023 (39 programs) [2023-01-15T16:30:02.673Z] INFO: [20/728] telenet.tv - AlJazeeraEnglish.qa - Jan 20, 2023 (40 programs) [2023-01-15T16:30:08.818Z] INFO: [21/728] telenet.tv - AlJazeeraEnglish.qa - Jan 21, 2023 (40 programs) [2023-01-15T16:30:14.678Z] INFO: [22/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 15, 2023 (33 programs) [2023-01-15T16:30:20.894Z] INFO: [23/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 16, 2023 (30 programs) [2023-01-15T16:30:26.922Z] INFO: [24/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 17, 2023 (31 programs) [2023-01-15T16:30:32.596Z] INFO: [25/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 18, 2023 (30 programs) [2023-01-15T16:30:38.746Z] INFO: [26/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 19, 2023 (30 programs) [2023-01-15T16:30:45.520Z] INFO: [27/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 20, 2023 (30 programs) [2023-01-15T16:30:51.570Z] INFO: [28/728] telenet.tv - AnimalPlanetNetherlands.nl - Jan 21, 2023 (31 programs) [2023-01-15T16:31:21.549Z] INFO: [29/728] telenet.tv - BabyTV.uk - Jan 15, 2023 (354 programs) [2023-01-15T16:31:51.689Z] INFO: [30/728] telenet.tv - BabyTV.uk - Jan 16, 2023 (356 programs) [2023-01-15T16:32:19.698Z] INFO: [31/728] telenet.tv - BabyTV.uk - Jan 17, 2023 (357 programs) [2023-01-15T16:32:49.801Z] INFO: [32/728] telenet.tv - BabyTV.uk - Jan 18, 2023 (355 programs) [2023-01-15T16:33:23.059Z] INFO: [33/728] telenet.tv - BabyTV.uk - Jan 19, 2023 (354 programs) [2023-01-15T16:33:54.300Z] INFO: [34/728] telenet.tv - BabyTV.uk - Jan 20, 2023 (355 programs) [2023-01-15T16:34:25.709Z] INFO: [35/728] telenet.tv - BabyTV.uk - Jan 21, 2023 (350 programs) [2023-01-15T16:34:34.186Z] INFO: [36/728] telenet.tv - BBCEntertainment.uk - Jan 15, 2023 (39 programs) [2023-01-15T16:34:39.861Z] INFO: [37/728] telenet.tv - BBCEntertainment.uk - Jan 16, 2023 (37 programs) [2023-01-15T16:34:46.278Z] INFO: [38/728] telenet.tv - BBCEntertainment.uk - Jan 17, 2023 (38 programs) [2023-01-15T16:34:52.699Z] INFO: [39/728] telenet.tv - BBCEntertainment.uk - Jan 18, 2023 (38 programs) [2023-01-15T16:34:58.746Z] INFO: [40/728] telenet.tv - BBCEntertainment.uk - Jan 19, 2023 (36 programs) [2023-01-15T16:35:06.498Z] INFO: [41/728] telenet.tv - BBCEntertainment.uk - Jan 20, 2023 (39 programs) [2023-01-15T16:35:12.812Z] INFO: [42/728] telenet.tv - BBCEntertainment.uk - Jan 21, 2023 (40 programs) [2023-01-15T16:35:18.002Z] INFO: [43/728] telenet.tv - BBCFirstBenelux.uk - Jan 15, 2023 (30 programs) [2023-01-15T16:35:23.281Z] INFO: [44/728] telenet.tv - BBCFirstBenelux.uk - Jan 16, 2023 (27 programs) [2023-01-15T16:35:28.866Z] INFO: [45/728] telenet.tv - BBCFirstBenelux.uk - Jan 17, 2023 (29 programs) [2023-01-15T16:35:33.882Z] INFO: [46/728] telenet.tv - BBCFirstBenelux.uk - Jan 18, 2023 (31 programs) [2023-01-15T16:35:54.132Z] INFO: [47/728] telenet.tv - BBCFirstBenelux.uk - Jan 19, 2023 (31 programs) [2023-01-15T16:35:59.964Z] INFO: [48/728] telenet.tv - BBCFirstBenelux.uk - Jan 20, 2023 (29 programs) [2023-01-15T16:36:04.952Z] INFO: [49/728] telenet.tv - BBCFirstBenelux.uk - Jan 21, 2023 (30 programs) [2023-01-15T16:36:11.026Z] INFO: [50/728] telenet.tv - BBCOneEngland.uk - Jan 15, 2023 (37 programs) [2023-01-15T16:36:17.307Z] INFO: [51/728] telenet.tv - BBCOneEngland.uk - Jan 16, 2023 (38 programs) [2023-01-15T16:36:24.435Z] INFO: [52/728] telenet.tv - BBCOneEngland.uk - Jan 17, 2023 (36 programs) [2023-01-15T16:36:30.818Z] INFO: [53/728] telenet.tv - BBCOneEngland.uk - Jan 18, 2023 (37 programs) [2023-01-15T16:36:34.076Z] INFO: [54/728] telenet.tv - BBCOneEngland.uk - Jan 19, 2023 (0 programs) [2023-01-15T16:36:34.076Z] ERROR: Cookie not in this host's domain. Cookie:horizon.tv Request:static.spark.telenet.tv [2023-01-15T16:36:40.772Z] INFO: [55/728] telenet.tv - BBCOneEngland.uk - Jan 20, 2023 (40 programs) [2023-01-15T16:36:46.742Z] INFO: [56/728] telenet.tv - BBCOneEngland.uk - Jan 21, 2023 (36 programs) [2023-01-15T16:36:51.456Z] INFO: [57/728] telenet.tv - BBCTwoEngland.uk - Jan 15, 2023 (19 programs) [2023-01-15T16:36:56.235Z] INFO: [58/728] telenet.tv - BBCTwoEngland.uk - Jan 16, 2023 (27 programs) [2023-01-15T16:37:01.424Z] INFO: [59/728] telenet.tv - BBCTwoEngland.uk - Jan 17, 2023 (23 programs) [2023-01-15T16:37:06.097Z] INFO: [60/728] telenet.tv - BBCTwoEngland.uk - Jan 18, 2023 (25 programs)

[PopeyeTheSai10r]

Try using the --log argument.

https://github.com/freearhey/epg-grabber#cli

[freearhey]

The same problem occurred when updating the telenet.tv guide in iptv-org/epg: https://github.com/iptv-org/epg/actions/runs/3935704430/jobs/6731621597

I will look into the issue as soon as I have some free time.

[bratekarate]

I'm pretty sure the issue is cause by this error:

ERROR: Cookie not in this host's domain. Cookie:horizon.tv Request:static.spark.telenet.tv

It's clearly the one thing all my failed builds have in common. This issue is crazy hard to debug. I was only able to reproduce it in a CI.

The first thing I would actually recommend in a pipeline is to check for the existence of the created file after running the grabber and exit if it does not exist. Otherwise, the pipeline will always be green. E.g. test -f <outfile> || exit 1.

Locally I am running Linux on node 16.19.1 and npm 8.19.2. Tried many times, never had this issue. In my CI pipeline, I first used node:16.19.1-alpine, which failed with said error. Funny thing though, locally it works with the exact same container!

I then switched to node:16.19.1-slim (debian I guess) and the pipeline worked for some time. However, as I tried multiple builds, it stopped working again. Proceeded to try node:16.19.0-slim, worked for one build, then it stopped again.

Now I went down to node:16.18.0, because I found out that it uses npm 8.19.2, the same version I use locally outside of docker. Same story, works for one build, then dies.

I am out of ideas for now, except lowering the log level and see if something helpful appears. I wonder why an invalid cookie is set.

[bratekarate]

TL;DR: Ignoring garbage cookies solves it. Can be configured in axios via ignoreInvalidCookies.

I don't think it's worth trying anything else, as the sites just return weird, out of spec cookies.

I will open a PR now, feel free to close it if it doesn't hit the mark. Maybe it's helpful to at least see where to configure it.

If anyone needs a quick fix, just use my fork in your package.json:

"epg-grabber": "git+https://github.com/bratekarate/epg-grabber.git",

[freearhey]

Fixed in v0.33.0