How to setup API Keys? There are many errors in uro and others

[hackersthan]

I already provide information to setup api keys with links and methods. Please check README file.

And for uro, pip3 install uro that added in setup_linux.sh Line:90

If you encounter with any types of errors, please add screenshot or paste the errors. I'm closing this issue here, if you want to specify your problem just comment here.

[Nixon-H]

Please check this

https://ibb.co/x16M0CH https://ibb.co/frJD8t8 https://ibb.co/qCqJFSn https://ibb.co/F5ZHKzf https://ibb.co/W2d6vpr https://ibb.co/MGZP8XV https://ibb.co/wBk1Gnk https://ibb.co/2qbQR9g https://ibb.co/VWFBtYm https://ibb.co/G32TXg8 https://ibb.co/Qkm58nP https://ibb.co/SJj2qfZ https://ibb.co/2FFJxDK

I am experiencing 'No Module' errors. I have tried using pip install, but it doesn't work. The other tools are also not finding anything. I couldn't upload here, so I'm using these links

[Nixon-H]

https://ibb.co/3s2yx3d https://ibb.co/TTbwqKW

Check this too

[hackersthan]

First you did not run the setup_linux.sh file, thats why you are facing no tool errors, This script downloads all the tool that anveshan needs. Second you have to add --break-system-packages to the end of your pip3 command like this

pip3 install PACKAGE_NAME --break-system-packages

also let me update the setup_linux.sh file, so it will works fine

[Nixon-H]

I have executed setup_linux.sh but the errors still persist

[hackersthan]

I just updates the scripts, remove old anveshan repo, download the latest repo and try again. This time it will works.

If you still got errors. comment here.

[Nixon-H]

https://ibb.co/m5Knjnk

Watch this. It's still not working. For now, no module error is solved . Rest problems are still existing

[Nixon-H]

https://ibb.co/23g6YPk https://ibb.co/8r4MvtF https://ibb.co/PrdfT27 https://ibb.co/qgG02N0

Many problems

[hackersthan]

Your setup command did not run successfully, thats why it did not install anew, bbot and other tools. manually install anew using go to save the output.

go install -v github.com/tomnomnom/anew@latest

You still miss the bbot tool and configuration files. I suggest you to run setup command again with sudo

sudo bash setup_linux.sh

it will install bbot tool and these configuration files.

#\\ downloading amass config //#
mkdir -p $HOME/.config/amass
wget -O ~/anveshan/.config/amass/datasources.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/datasources.yaml
wget -O ~/anveshan/.config/amass/config.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/config.yaml
wget -O ~/anveshan/.config/waymore/config.yml https://raw.githubusercontent.com/xnl-h4ck3r/waymore/refs/heads/main/config.yml

If you still encounter with errors, please let me know here. Thank you.

[Nixon-H]

amass error is still existing. The rest of the errors are solved.

[Nixon-H]

https://ibb.co/XsWb7Xf https://ibb.co/y6dxvvm

I have added API Keys. Followed the steps you gave me, and also re-ran the command of setup, but amass error is still existing

[Nixon-H]

BBot API setup is done, but still it is saying no API set for censys and others

[Nixon-H]

https://ibb.co/NtMDyPH https://ibb.co/1JLYCy7 https://ibb.co/D1CxpPs

[Nixon-H]

[+] running amass ... Failed to load the configuration file: error mapping configuration settings to internal values: yaml: line 114: mapping values are not allowed in this context Failed to load the configuration file: failed to get absolute path for wordlist file: file does not exist: stat /home/nixon/anveshan/.config/amass/wordlists/subdomains-top1mil-110000.txt: no such file or directory cat: amassP.txt: No such file or directory cat: amassA.txt: No such file or directory rm: cannot remove 'amassP.txt': No such file or directory rm: cannot remove 'amassA.txt': No such file or directory

[hackersthan]

Hello @Nixon-H

For some reasons some command did not run perfectly, please manually install getJS tool and configuration files.

1. First can you check your amass version, and make sure its in latest version [4.2.0], If not update using this command

   go install -v github.com/owasp-amass/amass/v4/...@master

2. Second install getJS tool properly using this golang command

   go install github.com/003random/getJS/v2@latest

3. Third for the api files, go to

   cd ~/anveshan/.config/amass/

   and make sure that config.yaml and datasources.yaml files are exist there. If not download using wget

   wget -O datasources.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/datasources.yaml
   wget -O config.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/config.yaml

And your problem solved. If you still encounter with errors please let me know.

Thank You, Hackersthan.

[Nixon-H]

https://ibb.co/9vkKLm2 https://ibb.co/KsmsbJQ https://ibb.co/v1QRhk2 https://ibb.co/PQyVRNr

Steps are followed. .config files exist, but amass is not running still

[Nixon-H]

Respond?

[hackersthan]

Hello @Nixon-H

All of your problems are gone expect amass config, This problem occurs because of /root/anveshan/amass/config.yaml and not /root/anveshan/.config/amass/config.yaml

1. You need to install amass config file here. Use these commands.

   wget -O /root/anveshan/.config/amass/datasources.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/datasources.yaml
   wget -O /root/anveshan/.config/amass/config.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/config.yaml

Now it should work with your root account.

Sorry for late reply Thank You, Sachin

[hackersthan]

If still not working

2. open anveshan.sh file

   cd anveshan [Go inside your downloaded anveshan folder from github]
   nano anveshan.sh

3. In Line number 93 and 94 change it to (change ~ into /root)

   amass enum -passive -d $domain -norecursive -nocolor -config /root/anveshan/.config/amass/datasources.yaml -o amassP.txt
   amass enum -active -d $domain -nocolor -config /root/anveshan/.config/amass/config.yaml -o amassA.txt

Now it should work with both of your local [nixon] and root user.

Thank You, Sachin

[Nixon-H]

https://ibb.co/N2sYd8B https://ibb.co/2k6HZKB

I still have this errors as well as amass error even after doing as you told me to. Sorry for late response. I had my exams

[hackersthan]

I updated this tool including both scripts, please reinstall and run linux_setup.sh file again. Then you can use bash anveshan.sh propely.

[Nixon-H]

The tool got installed without any issues on new linux but there is not .config files for any tool to add apis

[hackersthan]

I got it now,

There is a missing anveshan inside setup_linux.sh #L212. I fixed it.

You have no need to re run this script just run these command and everything will works

mkdir -p $HOME/anveshan/.config/amass
mkdir -p $HOME/anveshan/.config/waymore
wget -O $HOME/anveshan/.config/amass/datasources.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/datasources.yaml
wget -O $HOME/anveshan/.config/amass/config.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/config.yaml
wget -O $HOME/anveshan/.config/waymore/config.yml https://raw.githubusercontent.com/xnl-h4ck3r/waymore/refs/heads/main/config.yml

and then you can edit amass and waymore configuration files to add your api files.

open $HOME/anveshan/.config/amass/config.yaml
open $HOME/anveshan/.config/waymore/config.yml

[Nixon-H]

What about bbot, and subdominator? This tools can be used with API keys to get better results. Should API keys for them be added?

[Nixon-H]

Also, for amass it should be this for adding API keys. open /$HOME/anveshan/.config/amass/datasources.yaml

[hackersthan]

yeah this one

open /$HOME/anveshan/.config/amass/datasources.yaml 

and for the bbot and subdominator there configuration files are already in your system,

cd ~/.config

if not just run

subdominator

and

bbot

once they execute, they automatically creates config files inside ~/.config folder.

[Nixon-H]

Some tools, despite being properly installed, fail to run. Additionally, when upgrading tools like Subdominator, errors occur due to version mismatches between required package versions and the installed versions (either lower or higher).

[hackersthan]

Thanks for sharing these errors,

For amass error, i will fix it soon, I'm trying to find solution for it. For findomain, this error shows that you did not setup crtsh api key. I will suggest you to use crtsh api key on subdominator's config file, If you put that on findomain too, anveshan will send double requests and this will impact your api limitations.

Thank You, Sachin

[Nixon-H]

Thanks for sharing these errors,

Your thank you has been duly denied with utmost courtesy, brother.

For amass error, i will fix it soon, I'm trying to find solution for it. For findomain, this error shows that you did not setup crtsh api key. I will suggest you to use crtsh api key on subdominator's config file, If you put that on findomain too, anveshan will send double requests and this will impact your api limitations.

In which tools I must integrate the API, and where can they be located? Furthermore, what is CRTSH, how do I configure it, and what is the process for acquiring it?

[hackersthan]

Hello @Nixon-H

No problem, Here it is https://crt.sh is a searchable database of certificate transparency logs, It helps us to find subdomains. [No API Needed] Findomain and other tools automate fetch subdomains from crtsh

All of the API services with there URL, tool name to put, are available in README FILE.

I create a list that tells you, In which tool you should put api key

amass        : $HOME/anveshan/.config/amass/datasources.yaml
bbot         : $HOME/.config/bbot/secrets.yml
subdominator : $HOME/.config/Subdominator/provider-config.yaml

If you still have any kind of question just reply here. Thank You, Sachin

[Nixon-H]

Waymore has been executing since yesterday afternoon, and during my previous usage of this tool, Waymore extended beyond 72 hours to complete its run. I request you to either optimize the tool for efficiency or consider substituting it with a more expedient alternative. Moreover, it would be prudent to incorporate an option allowing the user to selectively enable or disable Waymore in the recon, as its current execution time significantly undermines the tool’s overall utility by consuming an excessive duration merely to retrieve links.

[hackersthan]

Some targets have much more urls, just like drive.google.com or mega or dropbox, they have common links to share documents, some targets have same type of products like amazon so they have lots of urls, waymore find all of them from multiple sources. It takes time, but not 72 hour.

It took only 06.59 minutes to fetch all of them for upwork.com

So anveshan or waymore have no issues, there might be issue with your internet connection. If you still getting this error, feel free to reply here.

Thank You, Sachin

[Nixon-H]

I understand your point, but the tool has now been running continuously for over 48 hours.
Attached is the screenshot displaying the current status of the Waymore which has been running for two days.

Additionally, here is the screenshot reflecting my internet speed.

[Nixon-H]

Please see the errors

[Nixon-H]

[hackersthan]

Wow lots of work to do here, Starting from pip

1. See have lots of old pip packages, anveshan can the function to update all of them, For now you can do this manually by running

   pip3 list --outdated
   pip3 list --outdated | grep "[0-9\.[0-9]" | cut -d " " -f1 | xargs pip3 install --upgrade --break-system-packages

2. Second, you have to reinstall urllib3 and six packages, that functionality is also available in anveshan, For now you can run,

   sudo apt remove -y python3-urllib3 python3-six
   sudo pip3 uninstall -y urllib3 six --break-system-packages
   sudo pip3 install urllib3 six --break-system-packages

and with these 2 steps most of the problem are gone.

3. Third, I also fix amass config, you can do manually by running,

   mkdir -p $HOME/anveshan/.config/amass
   mkdir -p $HOME/anveshan/.config/waymore
   wget -O $HOME/anveshan/.config/amass/datasources.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/datasources.yaml
   wget -O $HOME/anveshan/.config/amass/config.yaml https://raw.githubusercontent.com/owasp-amass/amass/refs/heads/master/examples/config.yaml
   wget -O $HOME/anveshan/.config/waymore/config.yml https://raw.githubusercontent.com/xnl-h4ck3r/waymore/refs/heads/main/config.yml

4. Forth, Here is the corrected grep command, I also update it in anveshan [DONE] Please change grep in #L244

   cat urls.txt | grep -Ei ".+\.js(?:on|p|x)?$" | $HOME/go/bin/httpx -mc 200 | anew jsurls.txt

5. getJS does not support -nocolors options now, You can fix it by removing -nocolors options at #L214 I will also fix this in anveshan [DONE]

   getJS --input ../webdomains.txt --output getjs.txt --complete

Everything will works perfectly now.

run linux_setup,sh and anveshan.sh both on same user, For you its either Nixon or root So the files will be downloads in same user. [You aready installed everything in your local account nixon so please run anveshan without sudo]

bash anveshan.sh

If you still facing any kind of issue while running anveshan, Feel free to reply here.

Thank You, Sachin