adamdehaven fetchurls issues - Githubissues

adamdehaven / fetchurls

A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

https://www.adamdehaven.com/blog/easily-crawl-a-website-and-fetch-all-urls-with-a-shell-script/

MIT License

127 stars 45 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

not working on kali linux

#18 THAnos0007 opened 2 years ago
0
I'm very leary about downloading a BASH script. Can you offer a MD5/SHA256 or other code to verify the download?

#17 Spazum closed 2 years ago
1
feat(regex): use wget reject-regex

#16 adamdehaven closed 2 years ago
0
Question - fetchUrlsForDomain

#15 usermor closed 2 years ago
2
[Feature Request] Add Option for User-Agent

#14 sommer-gei opened 2 years ago
4
Fix for executable

#13 Lamera closed 3 years ago
2
v3.2.3 Non Interactive Mode now allowing a few attributes

#12 PetroLogic closed 3 years ago
1
[Bug Report] does not run in Mac OS because of bash v3.2 - workaround

#11 CouldBeThis closed 3 years ago
1
[Feature Request] Add flag to allow bypassing robots.txt

#10 adamdehaven closed 3 years ago
1
Error: <domain> is unresponsive or is not a valid URL.

#9 Lilja closed 4 years ago
2
[Feature Request] username and password

#8 TonyStark closed 4 years ago
7
Adding flags, and enabling non-interactive mode

#7 adamdehaven closed 4 years ago
0
Add ability to pass required parameters as arguments to the script

#6 andrew-acpatt closed 4 years ago
1
Grep problem

#5 Flower7C3 closed 4 years ago
1
Permission issue to write file

#4 momnpophubadmin closed 5 years ago
1
Update fetchurls.sh

#3 viston03 closed 5 years ago
0
Fixes

#2 vendforce closed 5 years ago
2
Bash Error

#1 cyaris closed 5 years ago
1