Bug report/feature request: Continue downloading other videos when one fails with a permanent error

[heinrich5991]

I don't like the language in your issue template. Do you want a bug report or do you not? The tool said "please report this issue to https://github.com/bibanon/tubeup/issues". If you don't, maybe delete this message from the exception handler.

I have up-to-date tubeup (from git repository), yt-dlp and ia:

$ ia --version
3.2.0
$ yt-dlp --version
2023.03.04
$ git rev-parse HEAD
8c6fd019f97e6cb3690a14a33bbcd46438ba071d

$ python -m tubeup https://youtu.be/05LRSGq8nT4 https://youtu.be/0Ts-fXAkj8s
Teeworlds - Pros play TW 144: The Drunk coach
There are no annotations to write.
ERROR: [youtube] 0Ts-fXAkj8s: Private video. Sign in if you've been granted access to this video
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/youtube.py", line 4018, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1172, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

An exception just occured, if you found this exception isn't related with any of your connection problem, please report this issue to https://github.com/bibanon/tubeup/issues
Traceback (most recent call last):
  File "__main__.py", line 99, in main
    for identifier, meta in tu.archive_urls(URLs, metadata,
  File "TubeUp.py", line 417, in archive_urls
    downloaded_file_basenames = self.get_resource_basenames(
  File "tubeup/TubeUp.py", line 172, in get_resource_basenames
    if info_dict.get('_type', 'video') == 'playlist':
AttributeError: 'NoneType' object has no attribute 'get'

When one of the videos in the command line does not exist or is private, this aborts the whole download process. This is unwanted in my use case, because I want to archive all the Youtube videos that a certain website references; some of them might be gone or made private.

[vxbinaca]

Do not submit issues without the template answered because they're there for a reason. Follow the issue template and re-submit please.

[vxbinaca]

What Tubeup version is that?

[vxbinaca]

We recommend you install from pip. The latest changes to Tubeup don't involve the script. If you've modified Tubeup, consider this my request for a PR of your changes.

Take your URLs and do them one by one. This way the private videos fail and don't affect the rest. If you don't like Tubeups behavior please feel free to submit a PR or perhaps Bran will work on it. Follow the issue submission guidelines to a T or you get your issues closed.

You will be the last solitary issue submitter I will answer who doesn't follow the guidelines. After this issue it's close on sight and lock without comment. Too many issues submitted to this project don't follow basic things like updating your software. Furthermore you submitted this issue and admitted one of the URLs isn't live.

[heinrich5991]

What Tubeup version is that?

It's the latest master (8c6fd019f97e6cb3690a14a33bbcd46438ba071d), I have made no modifications. I can confirm that this also a problem with the latest pip version:

$ yt-dlp --version
2023.03.04
$ ia --version
3.0.2
$ tubeup --version
0.0.35
$ tubeup https://youtu.be/05LRSGq8nT4 https://youtu.be/0Ts-fXAkj8s

:: Item already exists. Not downloading.
Title: Teeworlds - Pros play TW 144: The Drunk coach
Video URL: https://www.youtube.com/watch?v=05LRSGq8nT4

ERROR: [youtube] 0Ts-fXAkj8s: Private video. Sign in if you've been granted access to this video
  File "/tmp/tu/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "/tmp/tu/lib/python3.10/site-packages/yt_dlp/extractor/youtube.py", line 4018, in _real_extract
    self.raise_no_formats(reason, expected=True)
  File "/tmp/tu/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1172, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

An exception just occured, if you found this exception isn't related with any of your connection problem, please report this issue to https://github.com/bibanon/tubeup/issues
Traceback (most recent call last):
  File "/tmp/tu/lib/python3.10/site-packages/tubeup/__main__.py", line 99, in main
    for identifier, meta in tu.archive_urls(URLs, metadata,
  File "/tmp/tu/lib/python3.10/site-packages/tubeup/TubeUp.py", line 417, in archive_urls
    downloaded_file_basenames = self.get_resource_basenames(
  File "/tmp/tu/lib/python3.10/site-packages/tubeup/TubeUp.py", line 172, in get_resource_basenames
    if info_dict.get('_type', 'video') == 'playlist':
AttributeError: 'NoneType' object has no attribute 'get'

Furthermore you submitted this issue and admitted one of the URLs isn't live.

Yes, this was kind of the point of the issue though; that the current behavior if one of the URLs is offline is not optimal for my use case.

Take your URLs and do them one by one.

Yes, that's a suitable workaround.

[vxbinaca]

Yes, that's a suitable workaround.

No it's not a 'workaround' because your issue isn't a problem, don't feed dead URLs to Tubeup and then run to us - bypassing the list of issue creation requirements that would have stopped this issue in it's tracks before being made because you 'don't like the language' - telling us we have a problem. YT-DLP which Tubeup depends can't rip dead URLs and I don't care about it's behavior outside tubeup.

I would direct you to look at the front page about live videos, there are behaviors in yt-dlp that Tubeup devs cannot control nor even yt-dlp devs. I'd say this is one of them. Specifically the parts where Pukkaden talks about ripping order is out of his control.

It takes a few seconds to check if a video is live. Failing that, get used to ripping one video at a time so failures are isolated.

[brandongalbraith]

I will take a look at improving this logic when time permits.