search

forensic-architecture / mtriage

framework to orchestrate the download and analysis of media
Other
98 stars 16 forks source link

Youtube-dl version needs upgrade #181

Closed batpad closed 3 years ago

batpad commented 3 years ago

Firstly, thank you for wonderful software!

I was getting errors when attempting to use the Youtube selector, something like:

ERROR: exxxxxxxiU: YouTube said: Unable to extract video data
[youtube] zxxxxxxxxtA: Downloading webpage
ERROR: Youtube: retrieve: DxxxxkEe30: Something went wrong downloading DxxxxxkEe30. It may have been deleted. - skipping element

I attempted to install youtube-dl locally and was able to download the same video. I was able to ssh into the mtriage docker container and get a similar error when trying to run youtube-dl from the command line:

[youtube] iQxxxxxxx4Q: Downloading webpage
ERROR: iQxxxxxxxx4Q: YouTube said: Unable to extract video data

Checking versions of youtube-dl showed me, version working on my laptop:

$ youtube-dl --version
2021.02.22

Inside the docker container:

# youtube-dl --version
2020.05.03

Running a pip install youtube-dl --upgrade inside the container fixes it, of course. I can look around a bit at where dependency versions are defined and try and submit a PR, but thought I'd outline the issue. Thank you again!

batpad commented 3 years ago

I see youtube-dl is defined without pinning to a version here: https://github.com/forensic-architecture/mtriage/blob/main/src/lib/selectors/Youtube/requirements.txt#L1

So it seems like the docker build should have pulled in the latest version, but some-how did not, and we were able to reproduce the same download failure on two environments. Will attempt a clean rebuild and see if that fixes.

batpad commented 3 years ago

Turns out I had a really old Docker image cached some-how. Apologies for the noise. Closing.

breezykermo commented 3 years ago

Hi @batpad yes this is probably our fault sorry. The Docker image on Dockerhub is very old, and so I recommend always running mtriage with --dev to ensure it's using the latest code on your local.

batpad commented 3 years ago

No worries at all - thanks for the tip about --dev. I some-how missed that flag. And now everything makes a lot more sense about the docker image versions :-) . Thanks for the response here.