Pipe it with Waybackurls?

[danzee1]

Hi @Zarcolio

Cool tool as I've just run it. I was thinking to pipe the output of waybackurls. Is it possible?

Sincerely,

[Zarcolio]

Hi there. Thanks for the compliment!

What's your thought on piping the output through waybackurls? I'm trying to think of a way, but waybackurls uses a domain for input and gives urls as output. The best way, I think, to use word lists is with fuzzers / brute forcers. For example:

wget -nd -r example.com -q -E -R woff,jpg,gif,eot,ttf,svg,png,otf,pdf,exe,zip,rar,tgz,docx,ico,jpeg cat .|wwwordlist --nh 4 --cl --co --max 10 -full|ffuf -recursion -w - -u https://sub.example.com/FUZZ -r

Or are you trying to get the content from URLs from waybackurls? Try using: cat domains.txt|waybackurls| xargs -n1 wget -qO - |wwwordlist -full I've added this example to the readme, thanks for the tip ;)

In the mean time I've added an option to do an analysis of the full text file with -full.

[danzee1]

Hi @Zarcolio

Nice.. I was thinking of the workflow as below..

Subdomain Scan by subfinder or any other tool == > "httprobe" on them ==> "waybackurls" or "gau" on all hosts ==>> Remove Duplicates along with meta data filetypes == > Now pipe that output to wwwordlist...

Thanks for the response !!

[danzee1]

Tried "cat domains.txt|waybackurls| xargs -n1 wget -qO - |wwwordlist -full"

But as expected "wget" is expecting "URL" & giving Error.

[Zarcolio]

That's strange... What's the platform you're running? And what's the exact error message you're getting?

I'm running an updated Kali Rolling and this exact command works like a charm.

[Zarcolio]

Also, added another example to the readme section for generating a wordlist if a target has code on Github. Clone the repo and fire this command inside the cloned folder: find . -type f -exec strings {} +|wwwordlist -full

Does this work for you?

[danzee1]

That's strange... What's the platform you're running? And what's the exact error message you're getting?

I'm running an updated Kali Rolling and this exact command works like a charm.

I am using Ubuntu Latest. and "wget" asks for "URL" thats the Error.

[Zarcolio]

Downloading Ubuntu 20.04 LTS now, but I'll be away in a little while. Will check it out when I'm back. I'm quite curious now, hopefully I can find a solution :)

[danzee1]

Hi @Zarcolio No need to download Ubuntu... I have again tested and now it showing that its working.... Though I've not seen output yet

[Zarcolio]

Good to hear it's working, depending on the quantity of input, it may take a while to complete. I'll try to optimize the code this week, hopefully it'll help.

[Zarcolio]

Did you manage to get some output from waybackurls wwwordlist?

[Zarcolio]

Closing the isssue, if you're problem has been resolved.

[Zarcolio]

Hi again, would mind testing it again? I've rewrote the code, so it's faster and has less bugs :)

[danzee1]

Hi @Zarcolio Oh.. SO sorry... I will now download, test and will report back.. Thanks

[danzee1]

Go this error:

xargs: unmatched double quote; by default quotes are special to xargs unless you use the -0 option

[Zarcolio]

This means that some URL from waybackurls contains a double quote. Please try it with urlcoding: cat domains.txt | waybackurls | urlcoding -e | xargs -n1 wget -T 2 -qO - | wwwordlist

Also, I use 2ulb a lot to create a symbolic link for a script.This saves you from typing python3, and the full path + the extension of the script every time ;)

[Zarcolio]

Or even better/faster, try it with parallel: cat domains.txt | waybackurls | urlcoding -e | parallel --pipe xargs -n1 wget -T 2 -qO - | wwwordlist