search

Ascoware / get-iplayer-automator

The goal of Get iPlayer Automator is to allow iTunes and your Mac to become the hub for your British Television experience regardless of where in the world you are. Currently, Get iPlayer Automator allows you to download and watch BBC and ITV shows on your Mac. Series-Link/PVR functionality ensures you will never miss your favourite shows. Programmes are fully tagged and added to iTunes automatically upon completion. It is simple and easy to use, and runs on any machine running Mac OS X 10.7 or later. And since the shows are in iTunes, it is extremely easy to transfer them to your iPod, iPhone, or Apple TV allowing you to enjoy your shows on the go or on your television.
GNU General Public License v3.0
144 stars 27 forks source link

Cannot Download from ITV #372

Closed sxmpete closed 2 years ago

sxmpete commented 2 years ago

Cannot download Good Karma Hospital. Get "WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg". DL then appears to start, but Keep getting error messages about "Invalid Timestamps". This is the same problem as Issues 353 and 363. Mac OS 10.14, GIA 1.21.13, same problems using both ExpressVPN and Nord VPN. Verbose log attached GIA 1.21.13 ITV Fail 26Jan22.docx

sleaux-meaux commented 2 years ago

Those messages can often be ignored if the download finishes successfully. Let it run to completion and then check the file.

skovatch commented 2 years ago

This happens for me on an M1 MacBook because the crypto library wasn't built for the same architecture as the included version of python. Youtube-dl can't handle decrypting the video segments, so it passes it off to ffmpeg, which is what generates all of those warnings. The video still downloads correctly if you let it run to completion.

I'm guessing, though, that you don't have one of the newer Macs?

sxmpete commented 2 years ago

2019 iMac running Mojave (for app compatibility reasons). I gave up after 32 log pages of errors, but will rty to leave it longer to see if it does indeed finish...

sxmpete commented 2 years ago

Let attempted DL run for 40 minutes. It did not complete, there was no longer any network activity and the log had stopped updating after 1286 pages.

sleaux-meaux commented 2 years ago

It could be that slow if downloading at or below realtime. It's not unheard of with a crummy VPN - you would have to check the file to see how much of the programme was downloaded. However, it would be more profitable to address the underlying problem. I see now that you have spliced in python3 for GiA, which I suspect is the root cause. It will not be able to load modules from pycrypto package included with GiA, which is compiled for Python 2.7, so youtube-dl will fail over to ffmpeg for downloading. The Mojave-supplied Python 2.7 should work OK. so remove the link to python3 (I assume you have followed #277).

sxmpete commented 2 years ago

Thanks! Yes I followed 277 suggestion and it worked, for a while. I will remove link and give it a try

On Sat, Jan 29, 2022 at 1:23 PM sleaux-meaux @.***> wrote:

It could be that slow if downloading at or below realtime. It's not unheard of - you would have to check the file to see how much of the programme was downloaded. However, it would be more profitable to address the underlying problem. I see now that you have spliced in python3 for GiA, which I suspect is the root cause. It will not be able to load PyCrypto module included with GiA, which is compiled for Python 2.7, so youtube-dl will fail over to ffmpeg for downloading. The Mojave-supplied Python 2.7 should work OK. so remove the link to python3 (I assume you have followed

277 https://github.com/Ascoware/get-iplayer-automator/issues/277).

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/372#issuecomment-1024962326, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGZX3OOSFX3QKNP72AXISM3UYQWDXANCNFSM5NAVZABA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- Regards, Peter

sxmpete commented 2 years ago

Per sleaux-meaux suggestion, reverted to MacOS version of Python (2.7.10). Attempted DL started ok (no hand-off to ffmpeg this time) but failed on what appears to be a python related error, log attached GIA ITV Fail 30Jan22.docx :

sleaux-meaux commented 2 years ago

Similar problem already reported upstream: https://github.com/ytdl-org/youtube-dl/issues/30544

There is nothing GiA can do in the face of network/server problems like that. Keep retrying the download. It should pick up where previous attempt left off and eventually finish.

nicacy commented 2 years ago

It will not be able to load modules from pycrypto package included with GiA, which is compiled for Python 2.7, so youtube-dl will fail over to ffmpeg for downloading. The Mojave-supplied Python 2.7 should work OK. so remove the link to python3 (I assume you have followed #277).

Hijacking this thread slightly, to query specifically this response. I've been having no luck downloading anything for the past few days, and as all happened to be ITV programmes I looked to this thread. But I'm running macOS 12.3 beta, in which the already deprecated Python 2.7 has been removed. I'm guessing this is the reason that Get iPlayer Automator no longer works (for me)?

skovatch commented 2 years ago

Hijacking this thread slightly, to query specifically this response. I've been having no luck downloading anything for the past few days, and as all happened to be ITV programmes I looked to this thread. But I'm running macOS 12.3 beta, in which the already deprecated Python 2.7 has been removed. I'm guessing this is the reason that Get iPlayer Automator no longer works (for me)?

This is correct -- Apple removed Python 2.7, and GiA looks for "/usr/bin/python" when starting youtube-dl for ITV downloads.

This is now a minor fire drill for me, since this breaks GiA in a couple of ways, and not just for ITV. I'm building a Python 3.x distribution that I can include inside of GiA, but the process isn't documented very well. I'm convinced it's doable, though. :-)

sleaux-meaux commented 2 years ago

For now, you can just install Python 2 from python.org and splice it into GiA as in https://github.com/Ascoware/get-iplayer-automator/issues/277#issuecomment-585271723. Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

skovatch commented 2 years ago

Yes, PyCryptodome does work with python3. I verified that part. I was able to create a python3 distribution and bundle it, but it leaves hard-coded paths in random places causing it to fail when I put the app on another machine.

I don't mind if you want to install Python yourself, but you shouldn't have to. Python 2.7 is outdated and I'd really like to avoid requiring any kind of additional installation, as it goes against the philosophy of dragging one thing to the Applications folder and launching the app.

sleaux-meaux commented 2 years ago

Sorry, that comment was for @nicacy

cgareth commented 2 years ago

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

To sleaux-meaux - I've just followed your advice to reread the comment link, and thank you. I remain on OSX10.12 with the latest GiA and have accepted not being able to download from ITV for over a year. I think I'd messed around with updating and reverting Python too much and had just broken something.

For what it's worth, I followed the instructions in the post I (having forgotten I'd ever done it) dumbed down and requoted from Steps39 on Feb 14th 2020. I don't know how to link directly to it from here.

Thanks again Scott, and to all you others who remain invaluable.

sleaux-meaux commented 2 years ago

@skovatch Consider switching to yt-dlp. It is patched for ITV and one of their build scripts could be used to generate a standalone executable (via pyinstaller --onedir) that presumably could be dropped into GiA. You may need to build it yourself for compatibility with older macOS (unless all dependencies are supplied via wheels). Also, their build was broken in latest release 2022.01.21, though previous release 2021.12.27 would be OK for ITV. I don't think you need to invoke python directly since it is no longer used to convert ITV subtitles.

skovatch commented 2 years ago

@sleaux-meaux yt-dlp does look like a good option. I'm finding that building a python distribution on macOS is not for the faint of heart, particularly when it comes to backwards compatibility.

sleaux-meaux commented 2 years ago

FWIW, I've verified that it's possible to build yt-dlp with pyinstaller to run on a clean 10.10 (Intel) system if built with python.org Python 3. The release build doesn't seem to work on macOS earlier than 10.15, which I think is probably due to their use of Xcode-supplied Python 3.

skovatch commented 2 years ago

Yup, I have a build now with a PyInstaller-built yt-dlp from a 3.9 python I built myself. It pulls in openssl and pycryptodomex and is built for arm64 and x86_64. You have to set MACOSX_DEPLOYMENT_TARGET=10.10

skovatch commented 2 years ago

I tested on my M1 MacBook pro and my work laptop which is an intel MBP running Big Sur. I can't put my VPN on my work laptop so I couldn't completely test but it did invoke yt-dlp and fetch page metadata.

nicacy commented 2 years ago

I'm building a Python 3.x distribution that I can include inside of GiA, but the process isn't documented very well. I'm convinced it's doable, though. :-)

Fingers crossed!

nicacy commented 2 years ago

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

Thanks very much for the response. I installed P2 and tried following the info in the other thread, but nothing would make GiA work with ITV downloads. I have zero experience with Python — so no idea how to go about troubleshooting — so for the time being at least it looks as though only BBC downloads are in my future!

cgareth commented 2 years ago

For now, you can just install Python 2 from python.org and splice it into GiA as in #277 (comment). Python 3 would also probably work as long as you install the pycryptodome package to avoid delegation to ffmpeg for downloading (unverified).

Thanks very much for the response. I installed P2 and tried following the info in the other thread, but nothing would make GiA work with ITV downloads. I have zero experience with Python — so no idea how to go about troubleshooting — so for the time being at least it looks as though only BBC downloads are in my future!

I was in the same boat as you until a couple of days ago. This works for me (I'm on Sierra and the newest GiA)

https://github.com/Ascoware/get-iplayer-automator/issues/277#issuecomment-586251184

nicacy commented 2 years ago

I was in the same boat as you until a couple of days ago. This works for me (I'm on Sierra and the newest GiA)

#277 (comment)

Encouraged by this, I tried again today on a different Mac (also running 12.3) using those step-by-step instructions, and it worked — ITV downloads are back 😊.

sxmpete commented 2 years ago

Latest workaround suggested in issue 354 fixes this problem on older Mac Os's up to Mojave by installing python 2.7.17 which contains TLS 1.02. ITV downloads are back!

cgareth commented 2 years ago

OS10.12 Latest GiA Tried downloading Trigger Point.

Having worked again last week, ITV isn't downloading again for me; I'd wouldn't normally question this but this part of the log caught my eye -

_youtube-dl version 2021.04.26-gia_01 [debug] Python version 2.7.10 (CPython) - Darwin-16.5.0-x8664-i386-64bit [debug] exe versions: ffmpeg 4.3.2-tessus [debug] Proxy map: {} [debug] Using fake IP 25.240.109.211 (GB) as X-Forwarded-For.

My python version is in fact 2.7.16 (I checked this on Terminal). I'm not using a fake IP.

I've tried the identical download on the previous two versions of GiA, to no avail. Don't know if this is of interest / any use, but one never knows.

skovatch commented 2 years ago

OS10.12 Latest GiA Tried downloading Trigger Point.

My python version is in fact 2.7.16 (I checked this on Terminal). I'm not using a fake IP.

This means that your environment that is started from Terminal.app is finding your 2.7.16 version of python in your path, while the system environment is finding /usr/bin/python first, which is 2.7.10.

What does /usr/bin/env python give you?

cgareth commented 2 years ago

Hi Scott, Do ignore my first reply below. I've reinstalled the necessary Python and all is good (I presume installing the updates to GiA mucked things up for me).

Every day here is a lesson!

This means that your environment that is started from Terminal.app is finding your 2.7.16 version of python in your path, while the system environment is finding /usr/bin/python first, which is 2.7.10.

What does /usr/bin/env python give you?

Hi Scott, It gives me this:

Python 2.7.17 (v2.7.17:c2f86d86e6, Oct 19 2019, 16:24:34) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

(I haven't done anything (to my knowledge at least) to GiA directly to re-halt ITV downloads.) Is that reply notable to any degree?

sleaux-meaux commented 2 years ago

Every time you update GiA, you need to re-create symbolic link for python as in https://github.com/Ascoware/get-iplayer-automator/issues/277#issuecomment-586251184 for 10.12 and below

skovatch commented 2 years ago

I'm closing this as fixed in v1.23.0