search

webrecorder / browsertrix-crawler

Run a high-fidelity browser-based web archiving crawler in a single Docker container
https://crawler.docs.browsertrix.com
GNU Affero General Public License v3.0
657 stars 83 forks source link

Error: "Missing required argument: url" when using yaml config option #76

Closed QFD-Felix closed 3 years ago

QFD-Felix commented 3 years ago

Hello,

I am currently trying to record a collection of urls using the crawler. However, when I was doing this command: "docker run -v $PWD/crawl-1.yaml:/app/crawl-1.yaml -v $PWD/crawls:/crawls/ webrecorder/browsertrix-crawler crawl —-config /app/crawl-1.yaml" The program will show "Missing required argument: url". The craw-1.yaml is downloaded from the fixtures folder. Did I do something wrong?

Thank you so much!

ikreymer commented 3 years ago

It's sort of hard to see but the —-config has an invalid dash, it should be --config. This was also the case for some examples in the README, now fixed.

The command should be:

docker run -v $PWD/crawl-1.yaml:/app/crawl-1.yaml -v $PWD/crawls:/crawls/ webrecorder/browsertrix-crawler crawl --config /app/crawl-1.yaml

With the invalid dash, it should just exit (as no config is specified). The missing required argument is from previous versions, so It looks like you may also have an older version of the image, so would recommend either running docker-compose build to build the latest, or pulling it again via docker pull webrecorder/browsertrix-crawler