13o-bbr-bbq / machine_learning_security

Source code about machine learning and security.
1.96k stars 645 forks source link

No open port report in Deepexploit and scan show all ports open #42

Open matcon opened 5 years ago

matcon commented 5 years ago

everything was wonderful these days training with metasploitable, since before yesterday I scan but can not find open ports despite creating the report as an example: nmap_result_192.168.56.101 Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-19 20:40 -04 Nmap scan report for 192.168.56.101 Host is up (0.0015s latency). Not shown: 65506 closed ports PORT STATE SERVICE VERSION 21/tcp open ftp vsftpd 2.3.4 22/tcp open ssh OpenSSH 4.7p1 Debian 8ubuntu1 (protocol 2.0) 23/tcp open telnet Linux telnetd 25/tcp open smtp Postfix smtpd 53/tcp open domain ISC BIND 9.4.2 80/tcp open http Apache httpd 2.2.8 ((Ubuntu) DAV/2) 111/tcp open rpcbind 2 (RPC #100000) 139/tcp open netbios-ssn Samba smbd 3.X - 4.X (workgroup: WORKGROUP) 445/tcp open netbios-ssn Samba smbd 3.X - 4.X (workgroup: WORKGROUP) 512/tcp open exec netkit-rsh rexecd 513/tcp open login 514/tcp open shell Netkit rshd 1099/tcp open rmiregistry GNU Classpath grmiregistry 1524/tcp open bindshell Metasploitable root shell 2049/tcp open nfs 2-4 (RPC #100003) 2121/tcp open ftp ProFTPD 1.3.1 3306/tcp open mysql MySQL 5.0.51a-3ubuntu5 3632/tcp open distccd distccd v1 ((GNU) 4.2.4 (Ubuntu 4.2.4-1ubuntu4)) 5432/tcp open postgresql PostgreSQL DB 8.3.0 - 8.3.7 5900/tcp open vnc VNC (protocol 3.3) 6000/tcp open X11 (access denied) 6667/tcp open irc UnrealIRCd 6697/tcp open irc UnrealIRCd 8009/tcp open ajp13 Apache Jserv (Protocol v1.3) 8180/tcp open http Apache Tomcat/Coyote JSP engine 1.1 8787/tcp open drb Ruby DRb RMI (Ruby 1.8; path /usr/lib/ruby/1.8/drb) 35544/tcp open mountd 1-3 (RPC #100005) 41383/tcp open nlockmgr 1-4 (RPC #100021) 46105/tcp open status 1 (RPC #100024) 57343/tcp open rmiregistry GNU Classpath grmiregistry Service Info: Hosts: metasploitable.localdomain, localhost, irc.Metasploitable.LAN; OSs: Unix, Linux; CPE: cpe:/o:linux:linux_kernel

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 125.48 seconds

I will copy this report from the msfconsole, after executing the script in training mode, and the script returns me: [+] Execute Nmap against 192.168.56.101 [*] nmap -p0-65535 -T4 -Pn -sV -sT --min-rate 1000 -oX nmap_result_192.168.56.101.xml 192.168.56.101

[] Start time: 2019/07/19 20:40:24 [] Port scanning: 192.168.56.101 [Elapsed time: 0 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 5 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 10 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 15 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 20 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 25 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 30 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 35 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 40 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 45 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 50 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 55 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 60 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 65 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 70 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 75 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 80 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 85 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 90 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 95 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 100 s] [] Executing keep_alive.. [] Port scanning: 192.168.56.101 [Elapsed time: 105 s] [] Executing keep_alive.. [] End time : 2019/07/19 20:42:30 [+] Get port list from nmap_result_192.168.56.101.xml. [!] No open port. [!] Shutdown Deep Exploit...

reinstall the requirements of pip and remains the same, i used metasploit from another server using msgrpc and get the same result, some idea of how to fix it, I do not believe the json, there must be some dependency that will have been updated in my system that is not working correctly. Screenshot_20190719_210135

cstayyab commented 5 years ago

Facing exact same issue... I even tried to allow all the connections using Windows Firewall on Metasploitable 3. Nmap Scan shows all port open but when nmap is run using proxychains it shows all ports are closed. @matcon can you run nmap using proxychains? Please provide the output of nmap when run with proxychains.

13o-bbr-bbq commented 5 years ago

@matcon Please show me the all content of the nmap_result_192.168.56.101.xml. . If XML format of Nmap result has changed, DeepExploit can't extract open ports information.

hamidb commented 5 years ago

I had similar issue. You can modify the code to read nmap output directly from xml file.

matcon commented 5 years ago

this is xml i change to txt for upload here. nmap_result_192.168.56.101.xml.txt

cstayyab commented 5 years ago

I had similar issue. You can modify the code to read nmap output directly from xml file.

@hamidb can you please send the modified code or tell where exactly to modify?

qiwihui commented 4 years ago

@cstayyab in get_port_list method, just replace nmap_result by nmap_result = open(nmap_result_file, 'rb').read() before using BeautifulSoup

qiwihui commented 4 years ago

@matcon This issue occurred because Msgrpc client failed to read whole nmap result. The cat command show the whole result while just first line was returned from the rpc client. Maybe it is due to there is '\n\n' between first line and the rest of lines.

https://github.com/13o-bbr-bbq/machine_learning_security/blob/2fc25589b3194953d21f68fe8a47a0076fc1915e/DeepExploit/DeepExploit.py#L888-L895

the ret should return:

{b'data': b'[*] exec: cat nmap_result_192.168.51.2.xml\n\n(and nmap result blabla...)', b'prompt': b'\x01\x02msf5\x01\x02 \x01\x02> ', b'busy': False}

it actually returned:

{b'data': b'[*] exec: cat nmap_result_192.168.51.2.xml\n\n', b'prompt': b'\x01\x02msf5\x01\x02 \x01\x02> ', b'busy': False}
researchlab17 commented 4 years ago

Can the fixed portion of the code be uploaded into the main repo?

cstayyab commented 4 years ago

This error is because the Output of any bash command is not returned with the output. There should be a way to do that.

The line [*] exec: cat nmap_result_192.168.51.2.xml\n\n is output from MSFConsole and all the other output of nmap is printed in the bash child process that Msfconsole has open. The output of that child process (bash) is not being included in 'console.read'

cstayyab commented 4 years ago

@cstayyab in get_port_list method, just replace nmap_result by nmap_result = open(nmap_result_file, 'rb').read() before using BeautifulSoup

@qiwihui This solution does not work if Metasploit RPC is on another System in the network and the DeepExploit is running on some other System. Because the path to nmap file will be local but the file would actually exist on the Other System (which has Metasploit and MsgRPC)

capce commented 3 years ago

As mentioned in #49 there were two things to change to get it work for me.

First thing is here: Replace Line 2226 with nmap_result = os.getcwd() + '/nmap_result_' + env.rhost + '.xml' https://github.com/13o-bbr-bbq/machine_learning_security/blob/76a283d1df59dd99e25d102490075d43c97f5a8d/DeepExploit/DeepExploit.py#L2226-L2229

Second thing is: Insert between line 914 and 915 nmap_result = open(nmap_result_file, 'rb').read() as @qiwihui suggested. https://github.com/13o-bbr-bbq/machine_learning_security/blob/2fc25589b3194953d21f68fe8a47a0076fc1915e/DeepExploit/DeepExploit.py#L914-L915

1939552724 commented 2 years ago

[!] 302/2006 linux/pop3/cyrus_pop3d_popsubfolders module is danger (rank: normal). Can't load. [] 303/2006 Loaded exploit: linux/postgres/postgres_payload [] 304/2006 Loaded exploit: linux/pptp/poptop_negative_read [] 305/2006 Loaded exploit: linux/proxy/squid_ntlm_authenticate [] 306/2006 Loaded exploit: linux/redis/redis_replication_cmd_exec [] 307/2006 Loaded exploit: linux/samba/chain_reply [] 308/2006 Loaded exploit: linux/samba/is_known_pipename [] 309/2006 Loaded exploit: linux/samba/lsa_transnames_heap [!] 310/2006 linux/samba/setinfopolicy_heap module is danger (rank: normal). Can't load. [] 311/2006 Loaded exploit: linux/samba/trans2open [!] 312/2006 linux/smtp/apache_james_exec module is danger (rank: normal). Can't load. [] 313/2006 Loaded exploit: linux/smtp/exim4_dovecot_exec [] 314/2006 Loaded exploit: linux/smtp/exim_gethostbyname_bof [!] type:<class 'KeyError'> [!] args:(b'rank',) [!] b'rank' [!] Failed: module.info

Hi, how can I solve the above problem?