NeffIsBack
commented
1 month ago These are Exception thrown by sqlalchemy, not sure if we can do much about it except updating the package
Open dazzgt opened 1 month ago
NeffIsBack
commented
1 month ago These are Exception thrown by sqlalchemy, not sure if we can do much about it except updating the package
NeffIsBack
commented
1 month ago Maybe this is relatedd: https://github.com/Pennyw0rth/NetExec/issues/209#issuecomment-2278660141
dazzgt
commented
1 month ago @NeffIsBack documentation of SQLAlchemy says that this is driver error. I think it can be some kind of misuse of concurrention with SQLite. In my project with async tasks i just used Lock on my side because SQLite support is only one read write operation at the same time anyway. I tried to look for problem by myself but didn't localize it. Here is description of error from documenatation.
InterfaceError
Exception raised for errors that are related to the database interface rather than the database itself.
This error is a [DBAPI Error](https://docs.sqlalchemy.org/en/20/errors.html#error-dbapi) and originates from the database driver (DBAPI), not SQLAlchemy itself.
The InterfaceError is sometimes raised by drivers in the context of the database connection being dropped, or not being able to connect to the database. For tips on how to deal with this, see the section [Dealing with Disconnects](https://docs.sqlalchemy.org/en/20/core/pooling.html#pool-disconnects).Most confusing part is that it write in DB normally but, because it have read problems, there a lot more entities in DB in the end.
NeffIsBack
commented
4 weeks ago From my understanding sqlalchemy should take care of concurrency issues with sqlite. Of course we are calling the nxcdb from several different (protocol) instances, but that should (afaik) not cause an issue and didn't to me until now. At least when the db file is on my own system drive. I once had an issue with sqlite when it was on an smb mounted share, but that is probably an edge case, which we can't solve anyway.
Can you reproduce it consistently somehow? If this a normal setup, or some kind of weird folder structure stuff that could confuse sqlalchemy?
AkechiShiro
commented
4 weeks ago I believe I can reproduce the issue, I'll try to run the command a 100 times and check if the output is always strictly the same or not. I'll report back my findings, on my side it was with nxc smb, I haven't tried the ssh module yet.
NeffIsBack
commented
4 weeks ago Just a setup would be nice so i can configure it and reproduce it on my side. Or indicators what the issue could be
AkechiShiro
commented
4 weeks ago I deployed Game Of Active Directory just as a lab for AD training (using Ludus on Proxmox), I think it would be possible to reproduce the issue with the default env but a more minimal example would help for sure maybe, two Windows VMs
dazzgt
commented
4 weeks ago Can you reproduce it consistently somehow? If this a normal setup, or some kind of weird folder structure stuff that could confuse sqlalchemy?
UPD: While i wrote this i actually though i can try to lock self.session.execute and check locally if it will resolve issue. I will check it now and tell you about result a bit later today
@NeffIsBack just normal setup through pipx on ubuntu. Absolutly basic user with my nickname in sudoers, but nxc installed and running without sudo. Everything in their base location like ~/.local and ~/.nxc. I think important part that i run it against subnet where a lot of hosts with SSH, like 30-50. In that case bug is reproduced in 100% cases but only in the begining when a lot of select query is executed. Then sometimes i see index error. If i run ssh module against just a few it work normal. Because of that i think root of problem in concurency.
I didin't work with DB isolation level much but how i know SQLAlchemy don't care about concurency at all. It just store data in memory based on different scope, dump this data on commit and use lazy load for read. But i work with DB in other direction. I create model and then make migrations with alembic and then create tables, but you create tables and then reflect on them. So i believe this may work differently. I using SQLite and SQLAlchemy in my project on work with async tasks, so i catched a lot of errors when there 20+ tasks working with DB. Because i need only simple select, upsert query so i just locked it with context manager like that.
class BaseRepository:
lock = Lock()
_conn: AsyncConnection
async def get():....
async def del():....
async def etc():....
async def __aenter__(self):
await self.lock.acquire()
self._conn = await engine.connect()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
try:
await self._conn.commit()
await self._conn.close()
finally:
self.lock.release()@NeffIsBack lock on session execute completely eradicate of InterfaceError
just replaced self.sess.execute with self.db_execute
from threading import Lock
class database:
def __init__(self, db_engine):
self.lock = Lock()
def db_execute(self, *args):
self.lock.acquire()
res = self.sess.execute(*args)
self.lock.release()
return resUPD: I will prepare PR in next few day. I think about create base class for database of all protocols. I beleave it's common issue for networks where a lot of hosts with available proto. If you think my solution is acceptable
dazzgt
commented
4 weeks ago Ok. I found that IndexError origin is the same. RowBase use some calls to DB. Most strange behavior for me is that calling __repr__ of host_result call the same _mapping as _asdict but don't raise this execption. I think this is because dict() call some other methods, that raise it. Also if __repr__ called before _asdict then there is no IndexError, probably because data is preloaded after __repr__ call.
AkechiShiro
commented
3 weeks ago Thanks a lot for your digging on this issue @dazzgt
Once you find some time to start a PR make sure to tag me along, I will try and test it on my side
NeffIsBack
commented
2 weeks ago Thanks a lot from me too! Probably @Marshall-Hallenbeck will take a look at the PR as soon as he has the time (he worked on the database the most and has much more knowledge about it than me)
Describe the bug There is a few exception when i run ssh. Both are connected to db.
To Reproduce Command:
netexec ssh 172.x.x.x/24 -u realuser -p realpassDon't think that this is important but there is quite a few machines with this creds in subnet. Alsowinrmmodule gave similar errors but where fixed after i ranpipx upgrade-allso i believe that there were fixes for similar issues in previous two weeks or so. Unfortunatelly i didn't look into error much so it possible that it was some kind of other error.Resulted in: 1) less commmon index error:
2) And more common exception: sqlite3.InterfaceError
NetExec info