Open darfielddx opened 6 years ago
Same problem, it is stuck on a show I am trying to download:
https://www.bbc.co.uk/iplayer/episode/b0b6ssl8/trooping-the-colour-highlights-2018
b0b6ssl8: Trooping the Colour: Highlights 2018: Retrieving Metadata For PID
INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used
Downloading Show 1/2:
BBC Download (ID=b0b6ssl8): Downloading Trooping the Colour: Highlights 2018
It seems it is now working again!
From the get_iplayer FAQ:
"What does it mean when I see a message similar to "WARNING: Unexpected size for file segment [99]"?
This means a segment of the requested programme was incomplete. get_iplayer will make several attempts to re-download the segment. This may be due to server problems or upstream connectivity problems - it is not a get_iplayer problem. These problems are usually transient, so delete any partial download and retry your download at a later time."
Sorry, that's the best answer I can give you.
@skovatch
Hello again... :smile: From the reports in https://github.com/Ascoware/get-iplayer-automator/issues/183 and the new one posted here, it appears that it is (only?) the "bidi" CDN which manifests the hickup; unfortunately, the bidi hosted mode is the first one GiA tries to fetch...
I would suspect the VPN used by the OP being at some kind of fault with that specific CDN, but that same issue has also been recently reported in the get_iplayer mailist list here by a member I know is located inside the UK (I used to be very active there in the past), so that leads me to believe it's not a VPN related issue...
In any case, something just does not add up: if the first attempt to fetch from bidi is unsuccessful, then GiP should
get_iplayer will make several attempts to re-download the segment.
however we're not seeing this in the logs posted...
Perhaps you should request a verbose GiA log is generated and posted by @darfielddx, so one can have a clearer understanding of where those failures occur (... the last time I asked a verbose log from a troubled GiA user I received verbal abuse/profanity and accused of being things I most certainly am not 😞 ...).
More importantly, I queried
Perhaps explicitly request the same quality variant (hvfhd) from a different CDN (akamai | limelight), is this somehow possible in GiA?
but no response that I saw was offered (?)
In the standalone GiP perl script, one can use the flag --exclude-supplier=bidi
to throw that CDN out of the equasion, or explicitly request the (2|3) mode variants
for a programme (e.g. --tvmode=hvfxsd2) to by-pass download attempts from the CDN at fault...
My gut feeling is the OP will succeed if tries to fetch from Akamai or Limelight...
... And FYI, the maintainer of GiP has recently released a macOS installer (package) for GiP 3.14:
https://github.com/get-iplayer/get_iplayer_macos https://github.com/get-iplayer/get_iplayer_macos/releases
Now, I won't just advise one to go and install/try that, for fear it interferes with an existing GiA installation, maybe you (i.e. Scott) can inspect/try it and, if all's good, give the go-ahead for GiA users to try it with their BBC failing downloads (selecting in it another CDN that hopefully works...).
Thanks and regards!
Hi Vangelis66. I am not about to issue profane foul rubbish at you for your request! That sort of person is contemptible!
I only understood a fraction of what you wrote. In short what do you require of me? What should I download for you to have a look at? What is a verbose log?
On 12 Jun 2018, at 12:30 AM, Vangelis66 notifications@github.com wrote:
@skovatch
Hello again... 😄 From the reports in #183 and the new one posted here, it appears that it is (only?) the "bidi" CDN which manifests the hickup; unfortunately, the bidi hosted mode is the first one GiA tries to fetch...
I would suspect the VPN used by the OP being at some kind of fault with that specific CDN, but that same issue has also been recently reported in the get_iplayer mailist list here by a member I know is located inside the UK (I used to be very active there in the past), so that leads me to believe it's not a VPN related issue...
In any case, something just does not add up: if the first attempt to fetch from bidi is unsuccessful, then GiP should
Retry fetching the segment with wrong filesize (I think a maximum of three times), as also mentioned in the wiki FAQ you quoted: get_iplayer will make several attempts to re-download the segment.
however we're not seeing this in the logs posted...
Fall back to alternate CDNs like Akamai (ak) and Limelight (ll); why is that not happening in GiA? Perhaps you should request a verbose GiA log is generated and posted by @darfielddx, so one can have a clearer understanding of where those failures occur (... the last time I asked a verbose log from a troubled GiA user I received verbal abuse/profanity and accused of being things I most certainly am not 😞 ...).
More importantly, I queried
Perhaps explicitly request the same quality variant (hvfhd) from a different CDN (akamai | limelight), is this somehow possible in GiA?
but no response that I saw was offered (?) In the standalone GiP perl script, one can use the flag --exclude-supplier=bidi to throw that CDN out of the equasion, or explicitly request the (2|3) mode variants for a programme (e.g. --tvmode=hvfxsd2) to by-pass download attempts from the CDN at fault... My gut feeling is the OP will succeed if tries to fetch from Akamai or Limelight...
... And FYI, the maintainer of GiP has recently released a macOS installer (package) for GiP 3.14:
https://github.com/get-iplayer/get_iplayer_macos https://github.com/get-iplayer/get_iplayer_macos/releases
Now, I won't just advise one to go and install/try that, for fear it interferes with an existing GiA installation, maybe you (i.e. Scott) can inspect/try it and, if all's good, give the go-ahead for GiA users to try it with their BBC failing downloads (selecting in it another CDN that hopefully works...).
Thanks and regards!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
I set GiP-A to recording. I left it for a lengthy time (12 minutes). Eventually, as you said it started recording. However, it obviously had its problems. This is the log of recording until I had to stop the VPN so that I could do some work. (My VPN is not working at home for some reason).
INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads
Downloading Show 1/1:
BBC Download (ID=b0b6vgbl): Downloading Poldark: Series 4 - Episode 1 INFO: 1 total programmes WARNING: A UK TV licence is required to access BBC iPlayer TV content legally INFO: Downloading tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd1/bi) [audio+video] WARNING: Unexpected size for file segment [1] WARNING: Expected: 1976632 Downloaded: 1260499 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 1.98 MB (00:00:08) @ 0.36 Mb/s (hvfxsd1/bi) [audio+video] WARNING: Unexpected size for file segment [2] WARNING: Expected: 1510392 Downloaded: 359370 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 1.51 MB (00:00:07) @ 0.31 Mb/s (hvfxsd1/bi) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1653702 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd1/bi) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1686470 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd1/bi) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 211909 WARNING: Retrying download INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd2/ll) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1751962 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd2/ll) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1162138 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd2/ll) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 752538 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd2/ll) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1407898 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd2/ll) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1342362 WARNING: Retrying download INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd3/ak) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 1553592 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd3/ak) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 148550 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd3/ak) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 617032 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd3/ak) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 576312 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd3/ak) [audio+video] WARNING: Unexpected size for file segment [3] WARNING: Expected: 1814200 Downloaded: 246672 WARNING: Retrying download INFO: Downloaded: 1.42 MB (00:01:59) @ 0.50 Mb/s (dvfxsd1/ll) [audio] WARNING: Unexpected size for file segment [32] WARNING: Expected: 45964 Downloaded: 31727 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 1.93 MB (00:02:41) @ 0.50 Mb/s (dvfxsd1/ll) [audio] WARNING: Unexpected size for file segment [74] WARNING: Expected: 46104 Downloaded: 31726 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 2.43 MB (00:03:23) @ 0.50 Mb/s (dvfxsd1/ll) [audio] WARNING: Unexpected size for file segment [127] WARNING: Expected: 46042 Downloaded: 3618 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 3.40 MB (00:04:44) @ 0.51 Mb/s (dvfxsd1/ll) [audio] WARNING: Unexpected size for file segment [201] WARNING: Expected: 45858 Downloaded: 31731 WARNING: Retrying download WARNING: Retrying tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 1.33 MB (00:01:51) @ 0.51 Mb/s (dvfxsd1/ll) [audio] WARNING: Unexpected size for file segment [230] WARNING: Expected: 45717 Downloaded: 15730 WARNING: Retrying download INFO: Downloaded: 2.80 MB (00:03:54) @ 0.83 Mb/s (dvfxsd2/ak) [audio] WARNING: Unexpected size for file segment [291] WARNING: Expected: 45892 Downloaded: 33676 WARNING: Retrying download BBC Download (ID=b0b6vgbl): Download Cancelled
When I restarted it, it took a similarly long time to get going with the same messages.
On 12 June 2018 at 07:05, Scott Kovatch notifications@github.com wrote:
Reopened #187 https://github.com/Ascoware/get-iplayer-automator/issues/187.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/187#event-1675071708, or mute the thread https://github.com/notifications/unsubscribe-auth/Ahz30telcGqg9YYBOheRlh5t5KvcU0dJks5t7vemgaJpZM4UiItV .
This is helpful, thanks. In the preferences, on the General tab, you'll see "Verbose Mode". Check that, then start a download. That will give us some additional information to work with.
As for the get_iplayer_macos distribution, that's just another way of packaging a get_iplayer distribution in a macOS-friendly installer. GiA also contains all of these components (except for the man pages), but includes them in a self-contained application bundle. You could install the get_iplayer macOS distribution without interfering with your Get iPlayer Automator.app installation.
@darfielddx, open up a Terminal window and paste in this line:
"/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer" --profile-dir "~/Library/Application\ Support/Get\ iPlayer\ Automator" --prefs-add --exclude-supplier=bidi
That will tell the embedded get_iplayer to ignore that supplier as @Vangelis66 suggests.
First off here is the Log for the download with Verbose checked. Sorry it is a bit long.
I will then try what you suggested above. Get iPlayer Automator 1.13.8 Initialized. <AppController: 0x7f88656261d0>: Updating Programme Index Feeds...
<AppController: 0x7f88656261d0>: Index was Up-To-Date.
INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads
Downloading Show 1/1:
BBC Download (ID=b0b6vgbl): Downloading Poldark: Series 4 - Episode 1
/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer
--profile-dir=/Users/forsyth.sm/Library/Application Support/Get iPlayer
Automator
--nocopyright
--nopurge
--atomicparsley=/Applications/Get iPlayer
Automator-3.app/Contents/MacOS/AtomicParsley
-e604800000000
--output=/Users/forsyth.sm/Movies/TV Shows
--subdir
--logprogress
--modes=radiobest,radiobetter,tvbest,tvbetter
--pid
b0b6vgbl
--whitespace
--attempts=5
--thumbsize=640
--ffmpeg=/Applications/Get iPlayer Automator-3.app/Contents/MacOS/ffmpeg
--verbose
--file-prefix=
I did as instructed and then when I hit return I got this message
-bash: /Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer: No such file or directory
I find this a bit strange!
I am about to start it up and see what happens.
This is what happened after. At the end of 5min 39 it still hadn’t started downloading, although it had created a file under in my “Movies” file. I see mentions of bidi still there.
Get iPlayer Automator 1.13.8 Initialized. <AppController: 0x7fccf9527fe0>: Updating Programme Index Feeds...
<AppController: 0x7fccf9527fe0>: Index was Up-To-Date.
INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads
Downloading Show 1/1:
BBC Download (ID=b0b6vgbl): Downloading Poldark: Series 4 - Episode 1
/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer
--profile-dir=/Users/forsyth.sm/Library/Application Support/Get iPlayer
Automator
--nocopyright
--nopurge
--atomicparsley=/Applications/Get iPlayer
Automator-3.app/Contents/MacOS/AtomicParsley
-e604800000000
--output=/Users/forsyth.sm/Movies/TV Shows
--subdir
--logprogress
--modes=radiobest,radiobetter,tvbest,tvbetter
--pid
b0b6vgbl
--whitespace
--attempts=5
--thumbsize=640
--ffmpeg=/Applications/Get iPlayer Automator-3.app/Contents/MacOS/ffmpeg
--verbose
--file-prefix=
INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads
Downloading Show 1/1:
BBC Download (ID=b0b6vgbl): Downloading Poldark: Series 4 - Episode 1
/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer
--profile-dir=/Users/forsyth.sm/Library/Application Support/Get iPlayer
Automator
--nocopyright
--nopurge
--atomicparsley=/Applications/Get iPlayer
Automator-3.app/Contents/MacOS/AtomicParsley
-e604800000000
--output=/Users/forsyth.sm/Movies/TV Shows
--subdir
--logprogress
--modes=radiobest,radiobetter,tvbest,tvbetter
--pid
b0b6vgbl
--whitespace
--attempts=5
--thumbsize=640
--ffmpeg=/Applications/Get iPlayer Automator-3.app/Contents/MacOS/ffmpeg
--verbose
--file-prefix=
I did as instructed and then when I hit return I got this message
bash: /Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer: No such file or directory
Oh, I think you renamed the application. Try this instead:
"/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer" --profile-dir "~/Library/Application\ Support/Get\ iPlayer\ Automator" --prefs-add --exclude-supplier=bidi
Then try downloading again.
Am away until Sunday. Will try Sunday evening.
On 14/06/2018, at 7:02 AM, Scott Kovatch notifications@github.com wrote:
I did as instructed and then when I hit return I got this message
bash: /Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer: No such file or directory
Oh, I think you renamed the application. Try this instead:
"/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer" --profile-dir "~/Library/Application\ Support/Get\ iPlayer\ Automator" --prefs-add --exclude-supplier=bidi
Then try downloading again.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Hello,
I did as instructed and pasted the code above into Terminal and it worked. By the way the application ahd the 3 tacked on the end when I downloaded it.
I tried downloading a program and it didn’t work. I am pasting below the log for the ending of the download.
I am also attaching a screenshot of what was downloaded.
The Semi-Final (b0b6trzt) [original]' INFO: Downloaded: 7.69 MB (00:08:03) @ 0.99 Mb/s (dvfsd2/ak) [audio] WARNING: Unexpected size for file segment [787] WARNING: Expected: 61065 Downloaded: 17090 WARNING: Retrying download WARNING: Retrying tv: 'Britain's Best Home Cook: Series 1 - 7. The Semi-Final (b0b6trzt) [original]' INFO: Downloaded: 3.66 MB (00:03:50) @ 1.01 Mb/s (dvfsd2/ak) [audio] WARNING: Unexpected size for file segment [847] WARNING: Expected: 61185 Downloaded: 928 WARNING: Retrying download WARNING: Retrying tv: 'Britain's Best Home Cook: Series 1 - 7. The Semi-Final (b0b6trzt) [original]' INFO: Downloaded: 2.01 MB (00:02:06) @ 1.07 Mb/s (dvfsd2/ak) [audio] WARNING: Unexpected size for file segment [880] WARNING: Expected: 61139 Downloaded: 706 WARNING: Retrying download WARNING: Retrying tv: 'Britain's Best Home Cook: Series 1 - 7. The Semi-Final (b0b6trzt) [original]' INFO: Downloaded: 1.51 MB (00:01:36) @ 0.93 Mb/s (dvfsd2/ak) [audio] INFO: Finished downloading to file: /Users/forsyth.sm/Movies/TV Shows/Britains Best Home Cook Series 1/Britains Best Home Cook Series 1 -
Regarding the app name, if you rename it once for whatever reason I believe Sparkle will maintain the name you gave it and just update the insides of the application bundle. So, not a big deal; you could rename it back.
I'm starting to think your VPN is doing something to corrupt the downloaded packets, since it's not just the 'bi' CDN that has the problem -- 'll' and 'ak' (Akamai?) are also not working. Are you able to watch video in a browser?
I would look into another provider. That might be the easiest fix here.
I have another VPN. Do I need to negate the code so that bidi is not excluded?
"/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer" --profile-dir "~/Library/Application\ Support/Get\ iPlayer\ Automator" --prefs-add --exclude-supplier=bidi
If so, what could I just swap exclude for include?
On 14 June 2018 at 08:02, Scott Kovatch notifications@github.com wrote:
I did as instructed and then when I hit return I got this message
bash: /Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer: No such file or directory
Oh, I think you renamed the application. Try this instead:
"/Applications/Get iPlayer Automator-3.app/Contents/Resources/get_iplayer" --profile-dir "~/Library/Application\ Support/Get\ iPlayer\ Automator" --prefs-add --exclude-supplier=bidi
Then try downloading again.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/187#issuecomment-397125172, or mute the thread https://github.com/notifications/unsubscribe-auth/Ahz30nIcf9jMbRK2aScbReZt3IXMHRB9ks5t8agagaJpZM4UiItV .
No, if your VPN provider is at fault it shouldn't matter if you are excluding bidi. But, if you want, you can remove it with:
/Applications/Get\ iPlayer\ Automator-3.app/Contents/Resources/get_iplayer --profile-dir ~/Library/Application\ Support/Get\ iPlayer\ Automator --prefs-del --exclude-supplier=bidi
Even with 1.13.8 I still cannot download BBC programs.
The Log says Downloading Show 1/3:
BBC Download (ID=b0b6vgbl): Downloading Poldark: Series 4 - Episode 1 INFO: 1 total programmes WARNING: A UK TV licence is required to access BBC iPlayer TV content legally INFO: Downloading tv: 'Poldark: Series 4 - 1. Episode 1 (b0b6vgbl) [original]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (hvfxsd1/bi) [audio+video] WARNING: Unexpected size for file segment [1] WARNING: Expected: 1976632 Downloaded: 64472 WARNING: Retrying download
I tried with 25 fps checked and unchecked and got the same message. The Higher Quality Audio button is checked as well.
I have better and best selected as download formats.
The 'bi' mentioned in a previous posting is still there as is the Unexpected Size for File Segment message.
What version of GiA are you using? On what version of OS X? 1.13.8 on OSX 10.13.5
Are you using a web proxy? DNS proxy? VPN? VPS? VPN
What program are you attempting to download (full title, series and episode)? Provide only ONE program as an example.
I was trying to download the latest Poldark
Please provide any additional information below.