search

bvanheu / pytoutv

TOU.TV client library and user interfaces written in Python 3
96 stars 23 forks source link

Can't download single movie #118

Closed gplaurin closed 6 years ago

gplaurin commented 6 years ago

When I try to download a movie on MAC OS X like https://ici.tou.tv/les-mondes-perdus or https://ici.tou.tv/arthur-laventurier-au-costa-rica there is always an error : "Unknown exception: <class 'TypeError'>: 'NoneType' object is not iterable".

Most probably because of the fact there is no Seasons or Episodes?

toutv fetch 'les-mondes-perdus'

Unknown exception: <class 'TypeError'>: 'NoneType' object is not iterable

simark commented 6 years ago

I get the same. Here is the backtrace:

$ toutv -v fetch https://ici.tou.tv/les-mondes-perdus
DEBUG:ShelveCache:Trying to open shelve at /home/simark/.cache/toutv/.toutv_cache
_parse_show_episode_from_args returns: emission les-mondes-perdus and episode None
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): ici.tou.tv
DEBUG:requests.packages.urllib3.connectionpool:https://ici.tou.tv:443 "GET /presentation//les-mondes-perdus?v=2&excludeLineups=False&d=android HTTP/1.1" 200 3939
Unknown exception: <class 'TypeError'>: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/home/simark/src/pytoutv/toutvcli/app.py", line 110, in run
    args.func(args)
  File "/home/simark/src/pytoutv/toutvcli/app.py", line 509, in _command_fetch
    overwrite=overwrite)
  File "/home/simark/src/pytoutv/toutvcli/app.py", line 773, in _fetch_emission_episodes
    episodes = self._toutv_client.get_emission_episodes(emission, True)
  File "/home/simark/src/pytoutv/toutv/client.py", line 101, in get_emission_episodes
    episodes = self._transport.get_emission_episodes(emission, short_version)
  File "/home/simark/src/pytoutv/toutv/transport.py", line 138, in get_emission_episodes
    for season in seasons:
TypeError: 'NoneType' object is not iterable
simark commented 6 years ago

"Les mondes perdus" now works, but there are two episodes. Arthur l'aventurier still doesn't , as there is a single episode. Indeed, the SeasonLineups attribute is None in that case.

It seems like we can still download the video if we treat the top-level object as if it was an episode. I have a potential fix in this branch, are you able to build and test it?

https://github.com/simark/pytoutv/tree/arthur

gplaurin commented 6 years ago

Yes your fix works I was able to download the video!

leye0 commented 6 years ago

@simark Could this fix be pull-requested in the @bvanheu repo?

simark commented 6 years ago

I've created a pull request (#120), I'll give people a few days to review it if they want to.

jfcomeau commented 6 years ago

I am still having issues downloading movies.

Any clue?

$ toutv -v fetch https://ici.tou.tv/j-ai-tue-ma-mere
DEBUG:ShelveCache:Trying to open shelve at /Users/###/.cache/toutv/.toutv_cache
_parse_show_episode_from_args returns: emission j-ai-tue-ma-mere and episode None
Unknown exception: <class 'AttributeError'>: 'LooseVersion' object has no attribute 'version'
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pytoutv-3.0.1-py3.6.egg/toutvcli/app.py", line 110, in run
    args.func(args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pytoutv-3.0.1-py3.6.egg/toutvcli/app.py", line 509, in _command_fetch
    overwrite=overwrite)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pytoutv-3.0.1-py3.6.egg/toutvcli/app.py", line 780, in _fetch_emission_episodes
    for episode in App._sort_episodes(episodes):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pytoutv-3.0.1-py3.6.egg/toutvcli/app.py", line 819, in _sort_episodes
    return sorted(episodes, key=episode_sort_func)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/version.py", line 52, in __lt__
    c = self._cmp(other)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/version.py", line 335, in _cmp
    if self.version == other.version:
AttributeError: 'LooseVersion' object has no attribute 'version'
simark commented 6 years ago

@jfcomeau, I pushed a patch to fix that particular issue, which was caused by the "episode" of the movie not having a SeasonAndEpisode tag. It now fails later with:

Error: cannot fetch "Épisode 01": Aucun DRM n'est supporté pour cette plateforme
Traceback (most recent call last):
  File "/home/simark/src/pytoutv/toutvcli/app.py", line 788, in _fetch_emission_episodes
    self._fetch_episode(episode, output_dir, bitrate, quality, overwrite)
  File "/home/simark/src/pytoutv/toutvcli/app.py", line 733, in _fetch_episode
    qualities = episode.get_available_qualities()
  File "/home/simark/src/pytoutv/toutv/bos.py", line 554, in get_available_qualities
    playlist, cookies = self.get_playlist_cookies()
  File "/home/simark/src/pytoutv/toutv/bos.py", line 542, in get_playlist_cookies
    url = self._get_playlist_url()
  File "/home/simark/src/pytoutv/toutv/bos.py", line 532, in _get_playlist_url
    raise RuntimeError(response_obj['message'])
RuntimeError: Aucun DRM n'est supporté pour cette plateforme

There seems to be something different with this media than the usual tou.tv shows. When loading it in Firefox, I had to accept/enable some DRM thing, which I don't get usually with other shows. So one would need to investigate what's different about it, analyze the traffic, etc. We usually have good success when analyzing the Android or iPhone application. I don't plan to have time to do this in the near future, but if you manage to capture some traffic, it would be useful if you could share it. If you need help to get started (e.g. which tools to use), feel free to ask.

A precision about the behavior of pytoutv: note that after failing to download the movie, it then proceeds to download the second "episode", "Entrevue avec les comédiens". This is because https://ici.tou.tv/j-ai-tue-ma-mere is an URL that we consider to be a series URL, because it doesn't have the last component that would identify the episode. When you pass a series URL, pytoutv downloads all the available episodes. So here, it fails to download the first one and starts downloading the second (which I guess is not the one you want). If this pattern becomes common, we'll need to find a way to just download the "main feature", i.e. the movie.

ehoffman2 commented 6 years ago

Having issue with 'Le commerce du sexe':

toutv -v fetch -q MAX https://ici.tou.tv/le-commerce-du-sexe DEBUG:ShelveCache:Trying to open shelve at /home/ehoffman/.cache/toutv/.toutv_cache _parse_show_episode_from_args returns: emission le-commerce-du-sexe and episode None DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): ici.tou.tv DEBUG:urllib3.connectionpool:https://ici.tou.tv:443 "GET /presentation//le-commerce-du-sexe?excludeLineups=False&v=2&d=android HTTP/1.1" 200 3964 Unknown exception: <class 'TypeError'>: 'NoneType' object is not iterable Traceback (most recent call last): File "/home/ehoffman/.local/lib/python3.5/site-packages/toutvcli/app.py", line 110, in run args.func(args) File "/home/ehoffman/.local/lib/python3.5/site-packages/toutvcli/app.py", line 509, in _command_fetch overwrite=overwrite) File "/home/ehoffman/.local/lib/python3.5/site-packages/toutvcli/app.py", line 773, in _fetch_emission_episodes episodes = self._toutv_client.get_emission_episodes(emission, True) File "/home/ehoffman/.local/lib/python3.5/site-packages/toutv/client.py", line 101, in get_emission_episodes episodes = self._transport.get_emission_episodes(emission, short_version) File "/home/ehoffman/.local/lib/python3.5/site-packages/toutv/transport.py", line 138, in get_emission_episodes for season in seasons: TypeError: 'NoneType' object is not iterable

Thanks, Eric

ehoffman2 commented 6 years ago

It turns out that applying simark fix did the trick. I taught that this was merged in this 'mainline' :-)

Note that once applied, I also needed to delete ~/.cache/toutv/.toutv_cache for this fix to work, otherwise the 'emission' data was pulled from cache instead of being re-generated.

So, this is my validation that simark patch did work for me... thanks!

Eric

simark commented 6 years ago

Thanks for testing. I created a new 3.0.2 release that includes this patch. Can you confirm that it also works with the release on PyPi?

ehoffman2 commented 6 years ago

Thanks simark

I verified with tag v3.0.2

All worked fine.

There might be specific movies or episodes which give DRM errors, as you noted, but this would be due to DRM, and would not be related to this issue here (that would be issue #124).

Merci! Eric

Edit: I verified with 'pip3 install pytoutv', so this is from the latest PiPI repos. Regards