search

SECFORCE / sparta

Network Infrastructure Penetration Testing Tool
GNU General Public License v3.0
1.61k stars 445 forks source link

'NoneType' object has no attribute 'id' #122

Closed ghost-ng closed 4 years ago

ghost-ng commented 4 years ago

This seems to only happen when I do the staged scans. I have the dev branch.

[+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200520170132-nmapstage4.xml [+] The process is done! [-] Something went wrong when parsing the nmap file.. [-] Unexpected error: <class 'AttributeError'> 'NoneType' object has no attribute 'id'

[+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200520171143-nmapstage4.xml [+] The process is done! [-] Something went wrong when parsing the nmap file.. [-] Unexpected error: <class 'AttributeError'> 'NoneType' object has no attribute 'id' nmap-xml.zip

ghost-ng commented 4 years ago

I stand corrected, just got one for a full tcp scan [+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200520173556-nmapfulltcp-10.11.1.251.xml [+] The process is done! [-] Something went wrong when parsing the nmap file.. [-] Unexpected error: <class 'AttributeError'> 'NoneType' object has no attribute 'id'

20200520173556-nmapfulltcp-10.11.1.251.xml.zip

ghost-ng commented 4 years ago

After doing some troubleshooting this appears to be the offending line:

LINE 601 of login.py: db_port = nmap_port(p.portId, p.protocol, p.state, db_host.id, db_service.id)

    [-] Something went wrong when parsing the nmap file..
    [-] Unexpected error: <class 'AttributeError'>

/usr/share/sparta/app/logic.py <class 'AttributeError'> 601

LINE 573: db_host = self.db.session().query(nmap_host).filter_by(ip=h.ip).first() # fetch the host This is returning NoneType

st3r30byt3 commented 4 years ago

Hi! We are looking into it and will update this thread.

st3r30byt3 commented 4 years ago

Should be fixed now. Thanks for the bug report. Please let me know if you face other issues.

ghost-ng commented 4 years ago

Thanks for the quick response,

Seemed to fix it but then got the same error after a few scans: 20200521071831-nmapstage4.xml.zip

I was actually able to import about 20 xml file but the one attached kept erroring out...

On another note, would be nice to have an autosave feature and the ability to bulk import / select multiple files to import. Also maybe error catch the nonetype before any actions are taken on it and display an error message to skip the import. This way the whole program doesnt crash and all the new scans are lost....I had to restart my work many many times. If i need to submit enhancement requests I can.

Thanks for the responsiveness, cheers!

[+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200521071831-nmapstage4.xml [-] Something went wrong when parsing the nmap file.. [-] Error: (<class 'AttributeError'>, AttributeError("'NoneType' object has no attribute 'id'"), <traceback object at 0x7fd9be17c040>) 'NoneType' object has no attribute 'id' [-] Error is on line 601 [+] The process is done! 20200521071831-nmapstage4.xml.zip

st3r30byt3 commented 4 years ago

Are you sure you fetched the latest version?

That file imports without problems for me.

Autosave actually already happens except for notes where you have to press save. Everything else is automatically added to the database. :)

ghost-ng commented 4 years ago

Just tried to import again, this time received a much more verbose error.

[-] Something went wrong copying the imported XML to the project folder. [-] Unexpected error: <class 'shutil.SameFileError'> [+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200521071831-nmapstage4.xml [-] Something went wrong when parsing the nmap file.. [-] Error: (<class 'AttributeError'>, AttributeError("'NoneType' object has no attribute 'id'"), <traceback object at 0x7f268a069e00>) 'NoneType' object has no attribute 'id' [-] Error is on line 601 qt.qpa.xcb: QXcbConnection: XCB error: 3 (BadWindow), sequence: 16310, resource id: 14327469, major code: 40 (TranslateCoords), minor code: 0 Exception during reset or similar Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 664, in _finalize_fairy fairy._reset(pool) File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 835, in _reset pool._dialect.do_rollback(self) File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 459, in do_rollback dbapi_connection.rollback() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139803499177728 and this is thread id 139804310357824. Exception closing connection <sqlite3.Connection object at 0x7f2690078a80> Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 664, in _finalize_fairy fairy._reset(pool) File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 835, in _reset pool._dialect.do_rollback(self) File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 459, in do_rollback dbapi_connection.rollback() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139803499177728 and this is thread id 139804310357824.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/pool/base.py", line 269, in _close_connection self._dialect.do_close(connection) File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 465, in do_close dbapi_connection.close() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139803499177728 and this is thread id 139804310357824.

To rule out the sqldb error (too many writes???), I re-opened the project and the first thing I did was import that file.

[+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200521071831-nmapstage4.xml [-] Something went wrong copying the imported XML to the project folder. [-] Unexpected error: <class 'shutil.SameFileError'> [-] Something went wrong when parsing the nmap file.. [-] Error: (<class 'AttributeError'>, AttributeError("'NoneType' object has no attribute 'id'"), <traceback object at 0x7feca6003a40>) 'NoneType' object has no attribute 'id' [-] Error is on line 601 qt.qpa.xcb: QXcbConnection: XCB error: 3 (BadWindow), sequence: 1751, resource id: 14357532, major code: 40 (TranslateCoords), minor code: 0

ghost-ng commented 4 years ago

[+] Parsing nmap xml file: /home/kali/Documents/Labs/pwk-lab-tool-output/nmap/20200521092610-nmapstage4.xml [-] Something went wrong when parsing the nmap file.. [-] Error: (<class 'AttributeError'>, AttributeError("'NoneType' object has no attribute 'id'"), <traceback object at 0x7fa88ab789c0>) 'NoneType' object has no attribute 'id' [-] Error is on line 601 [+] The process is done!

20200521092610-nmapstage4.xml.zip

ghost-ng commented 4 years ago

I should note i have the latest pull form the dev branch (with your updates included)

st3r30byt3 commented 4 years ago

It seems that in one of your errors, you were importing a file that you have already imported as the output indicates: [-] Unexpected error: <class 'shutil.SameFileError'>

We should handle that case better and we will, of course, but it's a corner case that shouldn't happen much.

Line 601 should now be: if db_service:

So, how exactly are you printing "Error is on line 601"?

Another question, was the project you are opening originally created with the dev version?

st3r30byt3 commented 4 years ago

Ok, I managed to replicate the sqlite error. We're looking into it.

ghost-ng commented 4 years ago

It seems that in one of your errors, you were importing a file that you have already imported as the output indicates: [-] Unexpected error: <class 'shutil.SameFileError'>

We should handle that case better and we will, of course, but it's a corner case that shouldn't happen much.

Line 601 should now be: if db_service:

So, how exactly are you printing "Error is on line 601"?

Another question, was the project you are opening originally created with the dev version?

Yes, I started with the dev branch.

to get the line number:

app/logic.py
app/settings.py

add - import sys (or just import the function)

             print("\t[-] Error:\n" + str(sys.exc_info()))
             print("\t[-] Error is on line {}".format(sys.exc_info()[-1].tb_lineno))

Line numbers are suppppper helpful.

st3r30byt3 commented 4 years ago

For sure. Our exception handling wasn't the best to start with :) This will change.

In the meantime, I could replicate the error so I'm looking into it. Thanks

ghost-ng commented 4 years ago

ok great! Thank you so much. By the way, everything is smooth right now, but I am not scanning multiple targets; I am trying to go in series as much as possible and it appears to avoid any import/sql related issues.

st3r30byt3 commented 4 years ago

Could you try now, please? :)

ghost-ng commented 4 years ago

so far everything is good! I've initiated quite a few staged scans and how not received any errors. Once my jobs end I'll open a new project and scale it with an entire /24.

So far so good, thanks for the support.

st3r30byt3 commented 4 years ago

Glad to hear. No prob.

Closing for now. Feel free to reopen if needed.