lanmaster53 / recon-ng

Open Source Intelligence gathering tool aimed at reducing the time spent harvesting information from open sources.
GNU General Public License v3.0
4.05k stars 641 forks source link

Is it possible to execute a task through through the web-server #51

Closed makstakeda closed 4 years ago

makstakeda commented 4 years ago

Hello!

I run the recon-ng web server by ./recon-web and create a new task by passing path through POST /tasks and see my tasks in a list ...

Screenshot from 2019-11-20 22-37-45

So. If it is in the list with status queued how to execute it? I could not find any mention about it on API endpoints.

I am wondering cause it looks like API has all the features about how to prepare module and task to run but I could not get hot to execute it...

lanmaster53 commented 4 years ago

If you have the redis server and worker running in the background, the tasks execute automatically as they are added in the queue. Are you running the full environment with docker-compose?

makstakeda commented 4 years ago

Strange. My tasks are queued for few days and stuck there :cry:

I am running from a source. I did not try run through a docker. My logs do not say anything suspicious when i try to hit POST /tasks/{path} (just disabled modules and missed API keys)

~/recon/recon-ng$ ./recon-web 
*************************************************************************
 * Welcome to Recon-web, the analytics and reporting engine for Recon-ng!
 * This is a web-based user interface. Open the URL below in your browser to begin.
 * Recon-web includes the Recon-API, which can be accessed via the `/api/` URL.
*************************************************************************
[*] Marketplace disabled.
[!] 'github_api' key not set. github_miner module will likely fail at runtime. See 'keys add'.
[!] 'shodan_api' key not set. shodan_org module will likely fail at runtime. See 'keys add'.
[!] Module 'recon/domains-contacts/metacrawler' disabled. Dependency required: ''PyPDF3''.
[!] 'github_api' key not set. github_users module will likely fail at runtime. See 'keys add'.
[!] 'bing_api' key not set. bing_linkedin_contacts module will likely fail at runtime. See 'keys add'.
[!] 'namechk_api' key not set. namechk module will likely fail at runtime. See 'keys add'.
[!] 'twitter_api' key not set. twitter_mentions module will likely fail at runtime. See 'keys add'.
[!] 'twitter_secret' key not set. twitter_mentions module will likely fail at runtime. See 'keys add'.
[!] 'twitter_api' key not set. twitter_mentioned module will likely fail at runtime. See 'keys add'.
[!] 'twitter_secret' key not set. twitter_mentioned module will likely fail at runtime. See 'keys add'.
[!] 'censysio_id' key not set. censysio module will likely fail at runtime. See 'keys add'.
[!] 'censysio_secret' key not set. censysio module will likely fail at runtime. See 'keys add'.
[!] 'github_api' key not set. github_dorks module will likely fail at runtime. See 'keys add'.
[!] 'fullcontact_api' key not set. fullcontact module will likely fail at runtime. See 'keys add'.
[!] 'bing_api' key not set. bing_linkedin_cache module will likely fail at runtime. See 'keys add'.
[!] 'hashes_api' key not set. hashes_org module will likely fail at runtime. See 'keys add'.
[!] 'hibp_api' key not set. hibp_breach module will likely fail at runtime. See 'keys add'.
[!] 'hibp_api' key not set. hibp_paste module will likely fail at runtime. See 'keys add'.
[!] 'github_api' key not set. github_commits module will likely fail at runtime. See 'keys add'.
[!] 'pwnedlist_api' key not set. leaks_dump module will likely fail at runtime. See 'keys add'.
[!] 'pwnedlist_secret' key not set. leaks_dump module will likely fail at runtime. See 'keys add'.
[!] 'pwnedlist_api' key not set. api_usage module will likely fail at runtime. See 'keys add'.
[!] 'pwnedlist_secret' key not set. api_usage module will likely fail at runtime. See 'keys add'.
[!] Module 'recon/domains-credentials/pwnedlist/account_creds' disabled. Dependency required: ''pyaes''.
[!] Module 'recon/domains-credentials/pwnedlist/domain_creds' disabled. Dependency required: ''pyaes''.
[!] 'pwnedlist_api' key not set. domain_ispwned module will likely fail at runtime. See 'keys add'.
[!] 'pwnedlist_secret' key not set. domain_ispwned module will likely fail at runtime. See 'keys add'.
[!] 'google_api' key not set. geocode module will likely fail at runtime. See 'keys add'.
[!] 'google_api' key not set. reverse_geocode module will likely fail at runtime. See 'keys add'.
[!] 'github_api' key not set. github_repos module will likely fail at runtime. See 'keys add'.
[!] 'binaryedge_api' key not set. binaryedge module will likely fail at runtime. See 'keys add'.
[!] 'builtwith_api' key not set. builtwith module will likely fail at runtime. See 'keys add'.
[!] 'shodan_api' key not set. shodan_hostname module will likely fail at runtime. See 'keys add'.
[!] 'bing_api' key not set. bing_domain_api module will likely fail at runtime. See 'keys add'.
[!] 'binaryedge_api' key not set. binaryedge module will likely fail at runtime. See 'keys add'.
[!] 'shodan_api' key not set. shodan_ip module will likely fail at runtime. See 'keys add'.
[!] 'twitter_api' key not set. twitter module will likely fail at runtime. See 'keys add'.
[!] 'twitter_secret' key not set. twitter module will likely fail at runtime. See 'keys add'.
[!] 'flickr_api' key not set. flickr module will likely fail at runtime. See 'keys add'.
[!] 'shodan_api' key not set. shodan module will likely fail at runtime. See 'keys add'.
[!] 'google_api' key not set. youtube module will likely fail at runtime. See 'keys add'.
[!] 'shodan_api' key not set. shodan_net module will likely fail at runtime. See 'keys add'.
[!] 'virustotal_api' key not set. virustotal module will likely fail at runtime. See 'keys add'.
[!] 'ipstack_api' key not set. ipstack module will likely fail at runtime. See 'keys add'.
[!] 'bing_api' key not set. bing_ip module will likely fail at runtime. See 'keys add'.
[!] 'ipinfodb_api' key not set. ipinfodb module will likely fail at runtime. See 'keys add'.
[!] 'virustotal_api' key not set. virustotal module will likely fail at runtime. See 'keys add'.
[!] 'google_api' key not set. pushpin module will likely fail at runtime. See 'keys add'.
[*] Version check disabled.
 * Workspace initialized: default
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [21/Nov/2019 08:59:52] "GET /api/modules HTTP/1.1" 308 -
127.0.0.1 - - [21/Nov/2019 08:59:52] "GET /api/modules/ HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 08:59:52] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [21/Nov/2019 09:00:10] "GET /api/tasks HTTP/1.1" 308 -
127.0.0.1 - - [21/Nov/2019 09:00:10] "GET /api/tasks/ HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:12] "GET /api/modules/ HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /api HTTP/1.1" 308 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /api/ HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /flasgger_static/swagger-ui.css HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /flasgger_static/swagger-ui-bundle.js HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /flasgger_static/swagger-ui-standalone-preset.js HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /flasgger_static/lib/jquery.min.js HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:19] "GET /apispec_1.json HTTP/1.1" 200 -
127.0.0.1 - - [21/Nov/2019 09:00:39] "POST /api/tasks/ HTTP/1.1" 201 -
lanmaster53 commented 4 years ago

Do you have Redis and a worker instance running? If not, then it's not going to work do anything.

I recommend using Docker. That's the best way to run this as the configuration is complicated. I won't be supporting custom configurations and manual setups to get all of that running. That's the reason I built the Docker stuff. But if you really want to, the docker-compose file should have enough information for you to cobble your way through it.

makstakeda commented 4 years ago

Yeah. Thanks. I set up the docker environment. I am trying with docker-compose at the moment but getting no data

http://127.0.0.1:5000/api/modules/ -> {"modules": []}

It looks like it talks with some another environment. If I do docker run - works fine and I see my data there.

Do I miss something? How to set it up?

makstakeda commented 4 years ago

ahh. got it. need to set it up through docker-compose run web /bin/sh.

Screenshot from 2019-11-24 13-51-42

Thank you for your help!

lanmaster53 commented 4 years ago

docker-compose up should have worked as well. It uses the local ~/.recon-ng folder on your host system if it exists, otherwise it creates a new one. Then you need to install modules, etc. What are you doing with it? Just testing, or building something around it?

makstakeda commented 4 years ago

At the moment just playing and looking how it works. When hope to use it as a data provider by hitting API from the app. It looks like really good all-in-one solution. Just do not need to store the response in tables. Need just return it :-)