search

jsvine / waybackpack

Download the entire Wayback Machine archive for a given URL.
MIT License
2.8k stars 189 forks source link

URLs with operators and hyphens #44

Closed gus5298 closed 3 years ago

gus5298 commented 3 years ago

Hi, I noticed that when I try to print the URLs of all archived snapshots for the SecureWorks RSS feed link I get an error. The link is causing waybackpack to think I am passing wrong arguments. I guess this happens because the link includes operators such as '&':

waybackpack https://www.secureworks.com/rss?feed=research&category=threat-analysis --list >> ./wbp/secureworks.txt

and the error message:

waybackpack: error: one of the arguments -d/--dir --list is required
'category' is not recognized as an internal or external command,
operable program or batch file.

Is there a way to avoid this error and get results for an URL like that?

jsvine commented 3 years ago

Hi, and thanks for your interest in this tool. The problem you've encountered isn't particular to waybackpack, but more generally a command-line constraint — if a command-line parameter includes certain special characters (e.g., ? in your case), you need to quote it:

waybackpack 'https://www.secureworks.com/rss?feed=research&category=threat-analysis' --list >> ./wbp/secureworks.txt

gus5298 commented 3 years ago

Hello,

Okay, I see. Thanks for the explanation!

gus5298 commented 3 years ago

Just wanted to add that I copied your quoted version to test it and I got the same error as before; The solution is to use double quotation marks:

waybackpack "https://www.secureworks.com/rss?feed=research&category=threat-analysis" --list >> ./wbp/secureworks.txt

jsvine commented 3 years ago

Huh, strange. Both single and double-quoting works on my machine. Perhaps it's a platform-dependent convention. Regardless, thanks for noting!