search

ArchiveTeam / youtube-grab

Archiving all metadata from YouTube (everything except videos themselves due to size)
The Unlicense
23 stars 2 forks source link

Issue when downloading thumbnail/comments "Lua runtime error: youtube.lua:630" #14

Open Medowar opened 2 months ago

Medowar commented 2 months ago 
Starting CheckIP for Item 
Finished CheckIP for Item 
Starting GetItemFromTracker for Item 
Received item 'v2:n0-Q9daIxSc' from tracker
Starting PrepareDirectories for Item v2:n0-Q9daIxSc
Finished PrepareDirectories for Item v2:n0-Q9daIxSc
Starting WgetDownload for Item v2:n0-Q9daIxSc
Archiving item v2:n0-Q9daIxSc.
1=200 https://www.youtube.com/watch?v=n0-Q9daIxSc  
getting comments
Lua runtime error: youtube.lua:630: attempt to index field 'videoDetails' (a nil value)
stack traceback:
    youtube.lua:630: in function <youtube.lua:189>.
Process WgetDownload returned exit code -6 for Item v2:n0-Q9daIxSc
Failed WgetDownload for Item v2:n0-Q9daIxSc
Waiting 10 seconds...

This error got thrown on a lot of jobs. Sample URLs: https://www.youtube.com/watch?v=F-hLWk2C-nw https://www.youtube.com/watch?v=vr9vXIBSAt4 https://www.youtube.com/watch?v=z9olfr-UtIM https://www.youtube.com/watch?v=SZH6jeI-NyE

Issue occours on the docker grab container(newest version, 20240307.01).

Arkiver2 commented 1 month ago

The item from your example v2:n0-Q9daIxSc is working fine for me. Comments are found and archived.

What I think may be going on is that YouTube has various implementations in production that look the same in the browser, but have data stored differently. We currently support one version after YouTube seems to have largely switched to that a few years ago.

When the errors like you described happen, the video will be handed out to someone else later for who downloading may work. We retry 20 times currently before deciding a videos is not retrievable.

This error got thrown on a lot of jobs.

As in for the majority (over 50%)? Or, do you simply regularly see it.

Arkiver2 commented 1 month ago

My explanation is related to https://github.com/ArchiveTeam/youtube-grab/issues/8.