Closed kokanin closed 11 months ago
jmbesnard
commented
1 year ago Thanks for reporting this. It looks like an issue we had a while back but got solved. Could you please provide the exact command line because the one provided does not include setting the 36 cores. Another thing you could try (and maybe already tried is to rerun) because that was already computed will be retrieved from cache.
Tanguy-Boisset
commented
1 year ago It looks like you have a loooot of objects on the previous request :
[101/119] [+]Requesting : RBCD attacks ... [-]Done in 17.49 s - 565395 objects
As the 102nd request builds the graph to these objects, the number of paths can grow exponentially and thus taking a very long time. In your case, I think it's advisable to disable this request altogether (the graph would not have been readable anyway).
To do this, edit the sources/modules/config.json file and set graph_rbcd to false.
jmbesnard
commented
1 year ago Ok now I get it. To avoid a long story, can you try with (for short, there are 2 algorithms that handles multi-processing).
ad-miner -u neo4j -p password -c -cf my_report -b bolt://localhost:7687 --cluster localhost:7687:36
kokanin
commented
1 year ago 10:25, ad-miner -u neo4j -p password -c -cf my_report -b bolt://localhost:7687 --cluster localhost:7687:36 12:54, stuck at same location with no cpu usage.
Setting graph_rbcd to false allows it to finish.
jmbesnard
commented
1 year ago thanks for the update. Good at least that you pull a report. We will need to look into what could go wrong with the rbcd control. Is it a rather large database (would help if you could indicate how many users,groups, computers, etc you have in it)
kokanin
commented
1 year ago Users | 19720 Groups | 15654 Computers | 12273 OUS | 1518 GPOs | 628 Domains | 4
jmbesnard
commented
1 year ago OK so not a tiny data set. My guess is that there is something like a deadlock but of course that is quite difficult to troubleshoot without access to the same data to reproduce the issue. Could you try the following ?
Also, could you share neo4j logs (because there may be a stack trace showing information about the issue which might be happening on neo4j side).
thanks
kokanin
commented
1 year ago setting cores to 10 or 1 has the same result, idling once it hits the section. I stopped the neo4j service, deleted the logs and executed ad-miner -u neo4j -p secret -c -cf my_report -b bolt://localhost:7687 --cluster localhost:7687:1 - these are the logs from that, and as you can see they don't really contain anything.
jmbesnard
commented
1 year ago nothing worrying in the logs. We will try to look into the issue but again this is particularly hard when not having the same data to reproduce.
jmbesnard
commented
1 year ago Would you maybe have some time tomorrow to join us on our Discord server and troubleshoot the issue ?
kokanin
commented
1 year ago travelling, but we can take a look in November if you want.
On Thu, 12 Oct 2023 at 17:44, Jean-Michel Besnard @.***> wrote:
Would you maybe have some time tomorrow to join us on our Discord server and troubleshoot the issue ?
— Reply to this email directly, view it on GitHub https://github.com/Mazars-Tech/AD_Miner/issues/39#issuecomment-1759881346, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATOZX6HIFOUY4MS2TSI3OTX7AF5VANCNFSM6AAAAAA52R5JVM . You are receiving this because you authored the thread.Message ID: @.***>
jmbesnard
commented
11 months ago We are already in November so... Closing this issue. Will reopen if needed.
Describe the bug ad-miner -c -cf bla -u neo4j -p secret never finishes despite running for a few days now
Terminal Output [+]Connected to database [1/119] [+]Requesting : Delete orphan objects that have no labels [-]Done in 0.61 s - 0 objects [2/119] [+]Requesting : Clean AD Miner custom attributes [-]Done in 0.23 s - 0 objects [3/119] [+]Requesting : Delete objects for which SID could not resolved [-]Done in 0.13 s - 0 objects [4/119] [+]Requesting : Set domain names to upper case when not the case [-]Done in 0.07 s - 0 objects [5/119] [+]Requesting : Clean AD Miner custom relations [-]Done in 0.02 s - 0 objects [6/119] [+]Requesting : Set is_server=TRUE to computers for which operatingsystem contains Server) [-]Done in 0.07 s - 0 objects [7/119] [+]Requesting : Set is_server=FALSE to other computers ) [-]Done in 0.10 s - 0 objects [8/119] [+]Requesting : Set dc=TRUE to computers that are domain controllers) [-]Done in 0.04 s - 0 objects [9/119] [+]Requesting : Set dc=FALSE to computers that are not domain controllers) [-]Done in 0.11 s - 0 objects [10/119] [+]Requesting : ADD CanExtractDCSecrets relation from BACKUP OPERATORS OR SERVER OPERATORS groups to DCs of same domain [-]Done in 0.07 s - 0 objects [11/119] [+]Requesting : ADD UnconstrainedDelegations relation from objects with KUD to the corresponding domain [-]Done in 0.09 s - 0 objects [12/119] [+]Requesting : Set is_adminsdholder to Container with AdminSDHOLDER in name [-]Done in 0.05 s - 0 objects [13/119] [+]Requesting : Set is_dnsadmin to Group with DNSAdmins in name [-]Done in 0.03 s - 0 objects [14/119] [+]Requesting : ADD CanLoadCode relation from PRINT OPERATORS groups to DCs of same domain [-]Done in 0.06 s - 0 objects [15/119] [+]Requesting : ADD CanLogOnLocallyOnDC relation from ACCOUNT OPERATORS groups to DCs of same domain [-]Done in 0.07 s - 0 objects [16/119] [+]Requesting : Set da=TRUE to users that are domain admins or administrators or enterprise admin [-]Done in 0.05 s - 0 objects [17/119] [+]Requesting : Set the da type (domain, enterprise, key or builtin) [-]Done in 0.12 s - 0 objects [18/119] [+]Requesting : Set da=TRUE to groups that are domain admins or administrators or enterprise admin [-]Done in 0.05 s - 0 objects [19/119] [20/119] [+]Requesting : Set da=TRUE to groups that are domain admins or administrators or enterprise admin [-]Done in 0.04 s - 0 objects [21/119] [22/119] [+]Requesting : Set dag=TRUE to the exact domain admin group (end with 512) [-]Done in 0.03 s - 0 objects [23/119] [+]Requesting : Set is_da=FALSE to all objects that do not have is_da=TRUE [-]Done in 0.42 s - 0 objects [24/119] [+]Requesting : Set is_dag=FALSE to all objects that do not have is_da=TRUE [-]Done in 0.39 s - 0 objects [25/119] [+]Requesting : Delete AdminTo edges from non-DA to DC [-]Done in 0.02 s - 0 objects [26/119] [+]Requesting : Set is_group_operator to Operator Groups (cf: ACCOUNT OPERATORS, SERVER OPERATORS, BACKUP OPERATORS, PRINT OPERATORS) [-]Done in 0.06 s - 0 objects [27/119] [+]Requesting : Set is_operator_member to objects member of Operator Groups (cf: ACCOUNT OPERATORS, SERVER OPERATORS, BACKUP OPERATORS, PRINT OPERATORS) [-]Done in 0.09 s - 0 objects [28/119] [+]Requesting : Set dcsync=TRUE to nodes that can DCSync (GetChanges/GetChangesAll) [-]Done in 2.30 s - 5 objects [29/119] [+]Requesting : Set dcsync=TRUE to nodes that can DCSync (GenericAll/AllExtendedRights) [-]Done in 2.40 s - 837 objects [30/119] [+]Requesting : Get list of objects that can DCsync (and should probably not be to) [-]Done in 0.07 s - 262 objects [31/119] [+]Requesting : Set path_candidate=TRUE to candidates eligible to shortestPath to DA [-]Done in 0.49 s - 0 objects [32/119] [+]Requesting : Set ou_candidate=TRUE to candidates eligible to shortestou to DA [-]Done in 0.40 s - 0 objects [33/119] [+]Requesting : Set contains_da_dc=TRUE to all objects that contains a domain administrator [-]Done in 0.05 s - 0 objects [34/119] [+]Requesting : Set contains_da_dc=TRUE to all objects that contains a domain controller [-]Done in 0.06 s - 0 objects [35/119] [+]Requesting : Set is_da_dc=TRUE to all objects that are domain controller or domain admins [-]Done in 0.10 s - 0 objects [36/119] [+]Requesting : Set members_count to groups (recursivity = 5) scope size : 15688 | nb chunks : 36 | nb cores : 36 100%|██████████████████████████████████████████████████████████████████████████████████| 36/36 [34:24<00:00, 57.34s/it] [-]Done in 34.41 m - 0 objects [37/119] [+]Requesting : Set has_member=True to groups with member, else false [-]Done in 0.16 s - 0 objects [38/119] [+]Requesting : Set the number of machines where Computers, Users, or Groups are admin (if too long, set recursivity to 3 into the query) [-]Done in 12.65 s - 0 objects [39/119] [+]Requesting : Set the count of links/object where the GPO is applied [-]Done in 0.06 s - 0 objects [40/119] [+]Requesting : Set has_links=True to GPOs with links, else false [-]Done in 0.03 s - 0 objects [41/119] [+]Requesting : Set is_adcs to ADCS servers [-]Done in 0.04 s - 1 objects [42/119] [+]Requesting : Set groups which are direct admins of computers [-]Done in 0.12 s - 133 objects [43/119] [+]Requesting : 1 - Set groups which are indirect admins of computers, ie. admins of admin groups (see precedent request) [-]Done in 0.07 s - 47 objects [44/119] [+]Requesting : 2 - Set groups which are indirect admins of computers, ie. admins of admin groups (see precedent request) [-]Done in 0.03 s - 0 objects [45/119] [+]Requesting : 3 - Set groups which are indirect admins of computers, ie. admins of admin groups (see precedent request) [-]Done in 0.02 s - 0 objects [46/119] [+]Requesting : 4 - Set groups which are indirect admins of computers, ie. admins of admin groups (see precedent request) [-]Done in 0.02 s - 0 objects [47/119] [+]Requesting : Count number of domains collected [-]Done in 0.02 s - 1 objects [48/119] [+]Requesting : Count number of users in group [-]Done in 0.10 s - 0 objects [49/119] [+]Requesting : Returns all users member of an admin group [-]Done in 5.28 s - 16067 objects [50/119] [+]Requesting : Returns all groups member of an admin group [-]Done in 0.10 s - 170 objects [51/119] [+]Requesting : Returns all computers administrated by an admin group [-]Done in 8.73 s - 27799 objects [52/119] [+]Requesting : Return direct admin users [-]Done in 0.24 s - 291 objects [53/119] [+]Requesting : Set ghost_computer=TRUE to computers that did not login for more than 90 days [-]Done in 0.12 s - 0 objects [54/119] [+]Requesting : List of domains [-]Done in 0.02 s - 1 objects [55/119] [+]Requesting : Number of domain controllers [-]Done in 0.03 s - 13 objects [56/119] [+]Requesting : Domain Organisational Units [-]Done in 1.43 s - 48397 objects [57/119] [+]Requesting : Non privileged users that can impersonate privileged users scope size : 14680 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:10<00:00, 3.56it/s] [-]Done in 10.42 s - 1872 objects [58/119] [+]Requesting : Non privileged users that can be impersonated by non privileged users scope size : 15678 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [19:31<00:00, 32.54s/it] [-]Done in 19.57 m - 352944 objects [59/119] [+]Requesting : Number of domain accounts [-]Done in 0.56 s - 19608 objects [60/119] [+]Requesting : Number of domain accounts enabled [-]Done in 0.58 s - 14692 objects [61/119] [+]Requesting : Number of domain accounts disabled [-]Done in 0.15 s - 4915 objects [62/119] [+]Requesting : Number of groups [-]Done in 0.65 s - 15688 objects [63/119] [+]Requesting : Number of computers [-]Done in 0.51 s - 12011 objects [64/119] [+]Requesting : Computers not connected since [-]Done in 0.46 s - 12011 objects [65/119] [+]Requesting : Number of domain admin accounts [-]Done in 0.11 s - 12 objects [66/119] [+]Requesting : Number of OS [-]Done in 0.49 s - 10630 objects [67/119] [+]Requesting : Kerberos password last change in days [-]Done in 0.03 s - 1 objects [68/119] [+]Requesting : Number of Kerberoastable accounts [-]Done in 0.07 s - 111 objects [69/119] [+]Requesting : Number of AS-REP Roastable accounts [-]Done in 0.04 s - 0 objects [70/119] [+]Requesting : Number of machines with unconstrained delegations [-]Done in 0.05 s - 4 objects [71/119] [+]Requesting : Number of users with unconstrained delegations [-]Done in 0.05 s - 7 objects [72/119] [+]Requesting : Number of users with constrained delegations [-]Done in 0.03 s - 0 objects [73/119] [+]Requesting : Number of enabled and never used accounts [-]Done in 0.16 s - 3685 objects [74/119] [+]Requesting : Dormant accounts [-]Done in 0.30 s - 6004 objects [75/119] [+]Requesting : Password last change in days [-]Done in 0.65 s - 14692 objects [76/119] [+]Requesting : Number of accounts where password cleartext password is populated [-]Done in 0.04 s - 0 objects [77/119] [+]Requesting : Number of accounts where password is not required [-]Done in 0.08 s - 1 objects [78/119] [+]Requesting : Number of sleeping accounts per domain [-]Done in 0.04 s - 1 objects [79/119] [+]Requesting : N objects have AdminSDHolder [-]Done in 0.11 s - 283 objects [80/119] [+]Requesting : Last logon in days [-]Done in 0.57 s - 14692 objects [81/119] [+]Requesting : Password never expired [-]Done in 0.13 s - 1592 objects [82/119] [+]Requesting : Domain accounts breakdown [-]Done in 0.08 s - 1 objects [83/119] [+]Requesting : Domain computers breakdown [-]Done in 0.07 s - 1 objects [84/119] [+]Requesting : High privilege group computer member [-]Done in 0.35 s - 0 objects [85/119] [+]Requesting : Objects with path to DA scope size : 44579 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:41<00:00, 1.14s/it] [-]Done in 41.55 s - 16163 objects [86/119] [+]Requesting : Objects with path to ADCS servers scope size : 44579 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:01<00:00, 19.81it/s] [-]Done in 2.11 s - 526 objects [87/119] [+]Requesting : Users admin on machines [-]Done in 12.32 s - 335624 objects [88/119] [+]Requesting : Users admin on servers n°1 scope size : 14680 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:07<00:00, 4.73it/s] [-]Done in 7.91 s - 14680 objects [89/119] [+]Requesting : Users admin on servers n°2 scope size : 14680 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:01<00:00, 25.77it/s] [-]Done in 1.62 s - 14680 objects [90/119] [+]Requesting : Number of computers admin of computers [-]Done in 0.34 s - 464 objects [91/119] [+]Requesting : Domain map trust [-]Done in 0.03 s - 0 objects [92/119] [+]Requesting : Object with path to non-DC computers with unconstrained delegations scope size : 44579 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:46<00:00, 1.30s/it] [-]Done in 47.84 s - 64654 objects [93/119] [+]Requesting : Objects with paths to users that have unconstrained delegations scope size : 44579 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [01:18<00:00, 2.19s/it] [-]Done in 1.32 m - 1841 objects [94/119] [+]Requesting : Number of computers with laps [-]Done in 0.45 s - 11094 objects [95/119] [+]Requesting : Objects allowed to read LAPS [-]Done in 19.36 s - 462 objects [96/119] [+]Requesting : Objects to dcsync scope size : 44343 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [22:01<00:00, 36.70s/it] [-]Done in 22.97 m - 4172874 objects [97/119] [+]Requesting : Domain admin with session on non DC computers [-]Done in 0.05 s - 3 objects [98/119] [+]Requesting : Unprivileged users with path to DNSAdmins scope size : 14680 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:09<00:00, 3.65it/s] [-]Done in 10.08 s - 1 objects [99/119] [+]Requesting : Users with RDP-access to Computers [-]Done in 20.46 s - 622860 objects [100/119] [+]Requesting : Non-domain admins that can directly or indirectly impersonate a Domain Controller [-]Done in 2.95 s - 13 objects [101/119] [+]Requesting : RBCD attacks scope size : 14680 | nb chunks : 36 | nb cores : 36 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [00:16<00:00, 2.24it/s] [-]Done in 17.49 s - 565395 objects [102/119] [+]Requesting : Builds RBCD attack path graph and sets is_rbcd_target attribute
System information
Additional context it happily ate all 36 cores in the early analysis, but now nothing has happened for quite a while, in the order of 2 days. The ad-miner.exe process uses 0 cpu and 500k memory.