Open holta opened 2 months ago
@deldesir
Is it possible the video's long length/duration (1:56:49) might be part of the problem?
https://dpaste.com/4D4SEN6F6 Ubuntu 24.04 here:
Lines 1110 and 1111 pasted in below — this issue appears to be the same as:
[2024-07-05 23:59:45,842] ERROR {cps.tasks.download:132} An error occurred during the subprocess execution: 'NoneType' object is not subscriptable
[2024-07-05 23:59:45,843] ERROR {cps.services.worker:202} no such column: error
Traceback (most recent call last):
File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 93, in run
requested_file = conn.execute("SELECT path FROM media WHERE webpath = ? AND path NOT LIKE 'http%'", (self.media_url,)).fetchone()[0]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/calibre-web-py3/cps/services/worker.py", line 199, in start
self.run(*args)
File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 133, in run
self.message = f"{self.media_url_link} failed to download: {self.read_error_from_database()}"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 149, in read_error_from_database
error = conn.execute("SELECT error FROM media WHERE webpath = ?", (self.media_url,)).fetchone()[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such column: error
@deldesir do you understand this bug @nzola reported about a week ago?
Yes, that means the video got stuck due to unavailable fragments.
means the video got stuck due to unavailable fragments
Dozens of "Some fragments are taking longer than expected to download" messages need to be explained better in "Tasks" view for everyday users:
means the video got stuck due to unavailable fragments
Dozens of "Some fragments are taking longer than expected to download" messages need to be explained better in "Tasks" view for everyday users:
- What does this really mean?
This is an unrecoverable error happened mysteriously with some videos. During the download some fragments just gone missing and the download is stuck. Nothing to see with Calibre-Web, xklb and yt-dlp. The expected fragments just don't leave YouTube servers.
- Being told dozens of times to "please wait" is not believable — is this truly the best that can be done?
Fixed in #223
- Conversely, worst case if there's absolutely nothing that can be done in this case and such cases, the root cause yt-dlp issue(s) need to be mentioned!
There is nothing we can do actually. This is an external issue yt-dlp cannot do anything about
I have done my best to get rid of the warning editbooks.py: SAWarning: Object of type <Books> not in session, add operation along 'Authors.books' won't proceed
. There is still work to do to ship a successful PR.
There is nothing we can do actually. This is an external issue yt-dlp cannot do anything about
Ok. So please now provide a link in "Tasks" view to a yt-dlp (or similar!) ticket explaining the context. Thanks.
Explain here why the error message is being repeated, so everyone can understand the root cause, and what it means.
Whereas suppressing error messages is almost always neither useful nor healthy.
(Ref: https://github.com/iiab/calibre-web/pull/223#issuecomment-2231708420)
Where is the work below? Thanks!
I have done my best to get rid of the warning
editbooks.py: SAWarning: Object of type <Books> not in session, add operation along 'Authors.books' won't proceed
. There is still work to do to ship a successful PR.
I have new understanding of the issue. The video was not stuck at yt-dlp level. That means it actually reached 100% progress. Another video that I tested (which was previously stuck) recovered successfully. I am looking into xklb's media check for hints. I have a suspicion memory might be an issue because the video that was recovered did not use over 4G which is set as the maximum in the media check code here at L48
Great progress! 💯
Explaining to everyone a bit better what's happening is indeed the goal, as intuitively/clearly as we can — on PR #223 and/or similar, so these can be merged quickly — as soon as any of these appear to be reasonable and safe (no matter how small!) steps forward! 🏗️
@nzola & @deldesir
Can you please help retest &/or recap? Now that these PR's are merged...
...thanks!! To help everyone deepen our understanding of next steps, e.g.:
@nzola, this issue should have been fixed by #191.
@nzola can you restest & confirm? (On July 24th I wrote...)
@nzola & @deldesir
Can you please help retest &/or recap? Now that these PR's are merged...
@nzola & @deldesir
I just tested "Download to IIAB" with https://youtu.be/4BL65HElOPg using the latest yt-dlp and the latest xklb, and it works:
Thanks @nzola for reporting the inability to download this formerly live video:
https://dpaste.com/4D4SEN6F6 Ubuntu 24.04 here:
https://dpaste.com/5CJPTTJRM Raspberry Pi 4 here:
ASIDE: Does the repetition in this "Tasks" view UX need to be clarified, or tightened up a bit?
Related:
188
202