Gospider produces too many files for `cat` to handle - resulting in error.

[frost19k]

Describe the bug It may happen that gospider produces way too many files for cat to handle

❯ ./reconftw.sh -d <Target> -s -v
...
...
 Running : Source code scraping subdomain search 
./reconftw.sh: line 519: /usr/bin/cat: Argument list too long

❯ sed -n 519p reconftw.sh | tr -d '\t'
[[ $NUMFILES -gt 0 ]] && cat .tmp/gospider/* | sed '/^.\{2048\}./d' | anew -q .tmp/gospider.txt

❯ ls -A /root/Tools/reconftw/Recon/<Target>/.tmp/gospider | wc -l
888970

To Reproduce Run reconFTW.sh in any mode that calls sub_scraping(). Error might or might not show, depending on the target.

Expected behavior

❯ ./reconftw.sh -d <Target> -s -v
...
...
 Running : Source code scraping subdomain search 

 XXX new subs (code scraping)

Desktop (please complete the following information):

• Architecture: x86_64
  • OS: Manjaro
  • Version: Rolling (Currently 21.2.0)

Additional context Testing a solution that employs find to feed files one at a time. Will make a PR if it works.

[six2dez]

Fixed on dev branch https://github.com/six2dez/reconftw/commit/026956a46976bfd25959ececf61895aa28019be5

[frost19k]

Hey, thanks for taking care of this. I'm sorry I didn't get around to making a PR. Turns out my CEH voucher is expiring next month & so I've had to abruptly drop almost everything :-P