tv_grab_fi configuration fails in windows pre-built binary

[VattuV]

XMLTV Version?

Pre-built binary xmltv-1.1.2-win32: https://github.com/XMLTV/xmltv/releases/tag/v1.1.2

XMLTV Component?

tv_grab_fi

Perl Version 5.28.1

Operating System

Windows 10 22H2

What happened?

tv_grab_fi configuration fails to run

Did you see any warnings/errors?

fetchRaw(): could not fetch https://areena.yle.fi/tv/opas, error: 403 Forbidden , aborting at C:\Users\username\AppData\Local\Temp\par-4b61797474616a6161\cache-2fcf189c5e125e3ed42d1163af5687b2e53fd2f0\inc/script/tv_grab_fi line 1634.

What steps are needed to reproduce this issue?

1. Extract prebuilt binary zip
2. Run in cmd xmltv.exe tv_grab_fi --configure

Please attach your config file below:

# -*- coding: utf-8 -*-

Any other information?

Temporary solution could be to disable yle.fi source in yle.pm line 15-17: BEGIN { our $ENABLED = 0; }

Same issue affects tv_grab_fi_sv, but above "easy fix" would not be usable alone with it as there is no alternative data source ready in tv_grab_fi_sv.

[stefanb2]

Duplicate of PR #189 fixed by 14a38faa6cec21929f31be2e349a1c4a32394779. Closing

Unfortunately there hasn't been a new release since then.

[honir]

@VattuV This fix isn't available in the latest official Windows build. There is an interim Windows build at 'Alpha' Windows exe but unfortunately that is broken at the moment, and hasn't been updated since 2022-09-03.

. (@rmeden can you check out the Windows build failure?)

[stefanb2]

(@rmeden can you check out the Windows build failure?)

Judging by the action log the SSH password/key for uploading the alpha binary needs to be updated?

[honir]

I think it's a file permissions issue - when the SCP upload tries to overwrite the existing file it can't write?

[stefanb2]

I think it's a file permissions issue - when the SCP upload tries to overwrite the existing file it can't write?

Nope, it fails before that:

Run scp -o StrictHostKeyChecking=no -o HostKeyAlgorithms=+ssh-rsa -P *** xmltv.exe @***:***
Warning: Permanently added '***' (RSA) to the list of known hosts.
Permission denied, please try again
Permission denied, please try again.
runner@***: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
lost connection
Error: Process completed with exit code 1.

[rmeden]

~xmltv-github and ~xmltv-github/httpdocs were owned by root, but still had world execute.. should have worked.   I changed them to be owned by github

drwxr-x--x 11 xmltv-github xmltv-github 4096 Sep  4 12:08 /home/xmltv-github drwxr-xr-x 3  xmltv-github xmltv-github 4096 Sep  4 12:08 /home/xmltv-github/httpdocs drwxrwsr-x 2  xmltv-github xmltv-github 4096 Feb  2 20:29 /home/xmltv-github/httpdocs/alpha -rw-rw-r-- 1  xmltv-github xmltv-github 27521427 Sep  4 12:09 /home/xmltv-github/httpdocs/alpha/xmltv.exe

On 2/13/2023 3:35 AM, Stefan Becker wrote:

I think it's a file permissions issue - when the SCP upload tries
to overwrite the existing file it can't write?

Nope, it fails before that:

|Run scp -o StrictHostKeyChecking=no -o HostKeyAlgorithms=+ssh-rsa -P xmltv.exe @: Warning: Permanently added '' (RSA) to the list of known hosts. Permission denied, please try again Permission denied, please try again. runner@***: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). lost connection Error: Process completed with exit code 1. |

— Reply to this email directly, view it on GitHub https://github.com/XMLTV/xmltv/issues/197#issuecomment-1427620862, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBCLNUZSMTZ6357N724UXDWXH55NANCNFSM6AAAAAAUZHSOL4. You are receiving this because you were mentioned.Message ID: @.***>

[stefanb2]

~xmltv-github and ~xmltv-github/httpdocs were owned by root, but still had world execute.. should have worked. I changed them to be owned by github

I re-ran the failed builds, but upload still fails. Error message is different though.

Run scp -o StrictHostKeyChecking=no -o HostKeyAlgorithms=+ssh-rsa -P *** xmltv.exe @***:***
kex_exchange_identification: read: Connection reset by peer
Connection reset by 207.58.129.***0 port ***
lost connection
Error: Process completed with exit code 1.

[rmeden]

Stefan, give me a message to troubleshoot...

rmeden    gmail,  gmail chat, skype... may other things :)

Robert On 2/13/2023 1:25 PM, Stefan Becker wrote:

~xmltv-github and ~xmltv-github/httpdocs were owned by root, but
still had world execute.. should have worked. I changed them to be
owned by github

I re-ran the failed builds, but upload still fails. Error message is different though.

|Run scp -o StrictHostKeyChecking=no -o HostKeyAlgorithms=+ssh-rsa -P xmltv.exe @: kex_exchange_identification: read: Connection reset by peer Connection reset by 207.58.129.0 port *** lost connection Error: Process completed with exit code 1. |

— Reply to this email directly, view it on GitHub https://github.com/XMLTV/xmltv/issues/197#issuecomment-1428528816, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBCLNUPRAFUTRRGALYQC2TWXKDEHANCNFSM6AAAAAAUZHSOL4. You are receiving this because you were mentioned.Message ID: @.***>

[stefanb2]

Newer SSH trying to talk to old SSHD: no matching algorithms. Robert is setting up a route through a different server. SCP to the new server already works.

[stefanb2]

@VattuV the Windows Alpha executable should be is updated to the latest commit now. Please download and try it.

[rmeden]

as someone with lots of production legacy systems, I find the new OSs blocking old stuff by default (unless you specifically enable legacy) quite annoying.   Yes, in a perfect world everyone is running the latest OS, but that's not the case!

Hopefully alpha-xmltv will update automatically now!

Robert

[garybuhrmaster]

Newer SSH trying to talk to old SSHD: no matching algorithms. Robert is setting up a route through a different server. SCP to the new server already works.

Unless the newer client/server have the algorithms explicitly compile time disabled, it is usually possible to use older compatible ciphers (that have typically been deprecated for many many years, but most people don't pay attention to those changes) using configuration or command line overrides (I have to do that for certain embedded targets), but ssh is very particular about file ownership and permissions, and has been so for essentially forever.

As an aside, as soon as the fixes for #196 are committed it might be time for a formal next release (1.1.3, or all the way to 1.2.1?).

[VattuV]

@VattuV the Windows Alpha executable ~should be~ is updated to the latest commit now. Please download and try it.

Thank you, tested now this alpha and seems to run fine, configuration works as well as grabbing finnish epg data. Though gives two errors:

ERROR: Date::Manip unable to determine Time Zone. GMT will be used. at C:\Users\Username\AppData\Local\Temp\par-446174757261\cache-f715706c2a46aa8a614f6f9544e0a837e709caff\inc\lib/Date/Manip/DM6.pm line 65. which refers to $dateUT = new Date::Manip::Date;

ERROR: Date::Manip unable to determine Time Zone. GMT will be used. at C:\Users\Username\AppData\Local\Temp\par-446174757261\cache-f715706c2a46aa8a614f6f9544e0a837e709caff\inc\lib/Date/Manip/DM6.pm line 68. which refers to $date = new Date::Manip::Date;

[stefanb2]

man page for Date::manip::TZ says

DETERMINING THE SYSTEM TIME ZONE
...
       The default methods for Windows systems are:

          main     TZ
          env      zone TZ
          registry
          gmtoff

So what tv_grab_fi does (time zone stored in $ENV{TZ}} should work under Windows. Maybe the timezone files are missing on your machine?

You could try to run this command in the directory where the Alpha release has been unpacked to:

perl -Ilib -MDate::Manip::TZ -e '$ENV{TZ}="Europe/Helsinki"; print new Date::Manip::TZ->curr_zone(), "\n"';

It should print out Europe/Helsinki.

Although, now that I look at the code I see thatsetTimeZone() is only called during grabbing, not configure or list channels. Do the error messages go away when you move that call further up into the soure, e.g. after these lines

  setDebug($Option{debug});
  setQuiet($Option{quiet});
  setTimeZone();                       # MOVED HERE...

[VattuV]

You could try to run this command in the directory where the Alpha release has been unpacked to:

perl -Ilib -MDate::Manip::TZ -e '$ENV{TZ}="Europe/Helsinki"; print new Date::Manip::TZ->curr_zone(), "\n"';

It should print out Europe/Helsinki.

Although, now that I look at the code I see thatsetTimeZone() is only called during grabbing, not configure or list channels. Do the error messages go away when you move that call further up into the soure, e.g. after these lines

  setDebug($Option{debug});
  setQuiet($Option{quiet});
  setTimeZone();                       # MOVED HERE...

Hi, suggested perl command gave me this error

Can't locate Date/Manip/TZ.pm in @INC (you may need to install the Date::Manip::TZ module) (@INC contains: lib C:/Perl64/site/lib C:/Perl64/lib).
BEGIN failed--compilation aborted.

I tried also moving setTimeZone(); with no effect on the error complaining missing module. Finally I did find manip from temp files and copied it to C:\Perl64\lib\Date and after that the error no longer appears, but yeah I guess for me it was only a visual problem. Thanks anyway!