Some timezone problems – Pull request added

[Moonbase59]

Thanks for making this! When playing around with it today, I noticed a few quirks which should be fixed in https://github.com/willemw12/xmltv2rss/pull/1:

• In larger (mostly aggregated) XMLTV EPGs, you’ll see a mixture of programme items with different UTC offsets, or even none, which is perfectly valid but produced an error due to using strptime with the %z option.
• The Air Date and Air Time was always shown in the original XMLTV EPG programme item’s timezone, not in local time (as other fields like pub date are).
• The test-xmltv2rss.sh test produced lots of diff’s if you were in a timezone different from the one in which the test output files were created in.

My pull request fixes these, and adds a timezones.xml test file for verification:

• fix reading XML programmes using no UTC offset for start and stop times
• display airdate, airtime in local timezone instead of EPG programme timezone
• add timezones.xml test files to test:
  • Programme within timezone UTC+0200.
  • Programme crossing end of daylight savings time (UTC+0200 to UTC+0100). This has an actual duration of 7:00:00!
  • Programme with no UTC offset given; should assume UTC.
• add TZ to test-xmltv2rss.sh so results are comparable even when in different timezone

Would be happy if you accepted the PR so we can use the same codebase again.

[willemw12]

Thanks for this. Looks great.

Does the -h/--help text need to be changed in any way?

Tests are failing for me. I am still getting diff outputs and the exit code is 1. Should be: an empty output text and exit code 0:

==> input/simple.xml
--- /tmp/test-xmltv2rss.sh.Y1Rx0V-expected  2023-10-28 19:36:18.179261134 +0200
+++ /tmp/test-xmltv2rss.sh.Y1Rx0V-output    2023-10-28 19:36:18.175927713 +0200
@@ -6 +6 @@
-    <pubDate>Sat, 28 Oct 2023 14:50:47 +0000</pubDate>
+    <pubDate>Sat, 28 Oct 2023 17:06:26 +0000</pubDate>
==> input/timezones.xml
--- /tmp/test-xmltv2rss.sh.Y1Rx0V-expected  2023-10-28 19:36:18.269263494 +0200
+++ /tmp/test-xmltv2rss.sh.Y1Rx0V-output    2023-10-28 19:36:18.265930073 +0200
@@ -6 +6 @@
-    <pubDate>Sat, 28 Oct 2023 15:55:42 +0000</pubDate>
+    <pubDate>Sat, 28 Oct 2023 17:14:26 +0000</pubDate>

There is a small typo: timzeone.

[Moonbase59]

oops, let me check (and thanks for the superfast response!)

[Moonbase59]

Fixed the typo, updated & tested the test data. Added "Fallback" in help, if you like this better (I think I do). Updated PR.

Here’s my console output:

matthias@e6510:~/Projekte/xmltv2rss$ ./xmltv2rss.py --help
usage: xmltv2rss.py [-h] [--feed-date-format FEED_DATE_FORMAT]
                    [--feed-language FEED_LANGUAGE]
                    [--feed-time-format FEED_TIME_FORMAT]
                    [--feed-title FEED_TITLE] [--feed-url FEED_URL]
                    [--xmltv-datetime-format XMLTV_DATETIME_FORMAT]
                    [<file>]

Generate an RSS feed from an XMLTV (tvguide) listing. Print the result to
standard output.

positional arguments:
  <file>                XMLTV input file

options:
  -h, --help            show this help message and exit
  --feed-date-format FEED_DATE_FORMAT, -d FEED_DATE_FORMAT
                        examples: "%Y-%m-%d", "%a %d %B, %Y", "%x"
  --feed-language FEED_LANGUAGE
                        RSS feed language. Default: "en"
  --feed-time-format FEED_TIME_FORMAT, -t FEED_TIME_FORMAT
                        examples: "%H:%M", "%I:%M %p", "%X"
  --feed-title FEED_TITLE
                        RSS feed title
  --feed-url FEED_URL   RSS feed link
  --xmltv-datetime-format XMLTV_DATETIME_FORMAT
                        XMLTV date and time format. Default: "%Y%m%d%H%M%S
                        %z", Fallback: "%Y%m%d%H%M%S"

For information about date and time format strings ("%Y", "%H", etc.), search
for "datetime" on https://docs.python.org.
matthias@e6510:~/Projekte/xmltv2rss$ TZ=UTC ./xmltv2rss.py tests/input/simple.xml > tests/output/simple.xml 
matthias@e6510:~/Projekte/xmltv2rss$ TZ=UTC ./xmltv2rss.py tests/input/timezones.xml > tests/output/timezones.xml 
matthias@e6510:~/Projekte/xmltv2rss$ tests/test-xmltv2rss.sh 
matthias@e6510:~/Projekte/xmltv2rss$ 

Did I miss anything? And could you retry, please?

[Moonbase59]

Oh, and I added a newline conversion for the EPG programme <desc> (the only field where newlines are allowed). Looks like this when using the timezones.xml as input to my Nextcloud RSS:

[Moonbase59]

Did a quick test running my version on the real radio server, using a much larger EPG that has different start and stop time variants. Here’s the marked entry from the testfile above "in real" (including the line break change), and operating from UTC+0200:

Daylight savings time ends on Oct 29 at 03:00, clocks turned back to 02:00.

Here’s the corresponding XMLTV entry:

  <programme start="20231029000000 +0200" stop="20231029060000 +0100" channel="niteradio.example.com">
    <title lang="en">Nuit électronique (mit Musikwünschen)</title>
    <sub-title lang="en">Euer Sender, eure Musik!</sub-title>
    <desc lang="en">Playlist: Nuit électronique

Dieses Programm könnt ihr durch zur Sendung passende Musikwünsche mitgestalten.
Geht dazu auf https://example.com/public/niteradio, klickt auf »Lied wünschen« und sucht euren Lieblingstitel aus.

Programm Copyright © 2023 Nite Radio — Non-public test &amp; evaluation server only
Besucht uns auf https://example.com/public/niteradio</desc>
    <credits/>
    <category lang="en">Music</category>
  </programme>

[willemw12]

Yes, tests succeed following your instructions:

TZ=UTC ./xmltv2rss.py tests/input/simple.xml > tests/output/simple.xml 
TZ=UTC ./xmltv2rss.py tests/input/timezones.xml > tests/output/timezones.xml 
tests/test-xmltv2rss.sh 

But the files committed in this pull request branch fail for me (as mention above).

The new help line should be split into two lines. I can do that later, if needed.

[Moonbase59]

That’s odd. I mean, when building the output test file, we must of course use the same TZ=UTC as the verify script will. What platform are you running this on (I tried with Linux Mint 21.2 and Ubuntu 22.04.3 LTS)?

The first 2 lines (with the TZ=UTC) are actually just needed to re-generate the output test files. Hmmm.

Can you show me the diff again? Maybe I made a stupid mistake somewhere like using an non-break space (that’s shift+space on my keyboard, so it could have happened).

Yeah, would be nice if you do the help stuff, thanks.

[Moonbase59]

Guess I found the culprit with the diffs: We use the input testfile’s mtime to generate the feed’s pubDate. Unfortunately, even a simple git pull or git clone modifies the mtime so the expected pubDate in line 6 isn’t matching anymore.

Maybe not only exclude lastBuildDate but also this first pubDate when comparing?

Btw, this also happens with the unmodified master branch:

matthias@e6510:~/test$ git clone https://github.com/willemw12/xmltv2rss
Klone nach 'xmltv2rss' …
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (18/18), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 29 (delta 3), reused 17 (delta 3), pack-reused 11
Empfange Objekte: 100% (29/29), 33.79 KiB | 2.11 MiB/s, fertig.
Löse Unterschiede auf: 100% (5/5), fertig.
matthias@e6510:~/test$ cd xmltv2rss/
matthias@e6510:~/test/xmltv2rss$ tests/test-xmltv2rss.sh 
==> input/simple.xml
--- /tmp/test-xmltv2rss.sh.W7VpVT-expected  2023-10-28 23:40:46.679134504 +0200
+++ /tmp/test-xmltv2rss.sh.W7VpVT-output    2023-10-28 23:40:46.675134448 +0200
@@ -6 +6 @@
-    <pubDate>Mon, 11 Sep 2023 10:46:49 +0200</pubDate>
+    <pubDate>Sat, 28 Oct 2023 23:40:36 +0200</pubDate>
matthias@e6510:~/test/xmltv2rss$ 

[Moonbase59]

@willemw12 Should be good now, in #1 I modified the test-xmltv2rss.sh to also ignore the first occurrence of <pubDate/> (the feed’s pubDate).

Please test & merge if all is good :-)

[Moonbase59]

Thanks for merging and the other updates—glad to see you keep this up-to-date!