DirSync

[ethhack]

Is there any documentation or decent example on using the DirSync extension for ldap3?

[cannatag]

Hi this is how dir_sync works. Keep in mind that I don't use it and has been used only by a few users, so I'm not sure if it works in edge cases.

I've followed the official Microsoft docs, To be compliant with the docs I added the LDAP_SERVER_DIRSYNC_OID control, the LDAP_SERVER_EXTENDED_DN_OID control and the LDAP_SERVER_SHOW_DELETED_OID control.

I think the usage is quite simple:

to start a sync: sync = connection.extend.microsoft.dir_sync(root_partition)

to get the next batch of modifications: mods = [] while sync.more_data: mods += sync.loop()

You can set all the options available in Microsoft docs for the DIrSync operation:

• sync_base: must be a root partition
• sync_filter : defaults to '(objectclass=*)'
• attributes: defaujlts to ALL_ATTRIBUTES
• cookie: you can leave this None or set a cookie from a previous loop.
• object_security: if True you get only object available to the user account, default to False
• ancestors_first: if True you get all the containers objects before their children, defaults to True
• public_data_only: if True you get data that don't need authorization, default to False
• incremental_values: after the first loop you get only the modified attributes, defaults to True
• max_length: the max number of bytes that could be returned, defauilt to 2147483647 (the max possible value). The actual number of bytes returned is constrained by the server too
• hex_guid: If True returns GUID in hex form. defaults to False.

As per Microsoft specifications the DN of each object has also the GUID (and SID if available) information.

You can aso add the LDAP_SERVER_EXTENDED_DN_OID and LDAP_SERVER_SHOW_DELETED_OID to your other searches.

For some usage example look at \ldap3\test\testMicrosoftAD.py

Please give me feedback and suggestion on how this function can be improved.

Bye, Giovanni

2017-03-30 23:53 GMT+02:00 ethhack notifications@github.com:

Is there any documentation or decent example on using the DirSync extension for ldap3?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cannatag/ldap3/issues/329, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQhpVSfR-I0JorAIkuN9nTekrTLLXz2ks5rrCR3gaJpZM4Mu-Hb .

[ethhack]

Much obliged.

For some reason, I was just having fits trying to sort out how to actually use it.

I'll give this a read over, then take a shot and see if it helps, and let you know.

Thanks, Giovanni.

[ethhack]

Hi Giovanni.

Never did get this worked out. Trying to run a sync per the info you provided, previously, the mods += sync.loop() returns:

... mods += sync.loop() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ldap3/extend/microsoft/dirSync.py", line 90, in loop raise LDAPExtensionError('error %r in DirSync' % result) ldap3.core.exceptions.LDAPExtensionError: error {'result': 12, 'description': 'unavailableCriticalExtension', 'dn': '', 'message': '00000057: LdapErr: DSID-0C0907C5, comment: Error processing control, data 0, v1db1\x00', 'referrals': None, 'type': 'searchResDone'} in DirSync

I'm assuming this is due to the persistent search controls which I've used elsewhere for standard ldap searches, but I never had any luck with this just trying per your previous response.

Any thoughts appreciated. Thanks.

[ethhack]

Looks like my imports might be a problem... I'll see if I can sort that out, before you take the time to think on this anymore. (Sorry)

Thanks

[tuftedocelot]

For what it's worth, @ethhack, I get the same error trying with a brand new test AD environment. Trying to turn up the logging yields:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3.6/site-packages/ldap3/extend/microsoft/dirSync.py", line 75, in loop
    show_deleted_control(criticality=False)]
  File "/usr/lib/python3.6/site-packages/ldap3/core/connection.py", line 775, in search
    response = self.post_send_search(self.send('searchRequest', request, controls))
  File "/usr/lib/python3.6/site-packages/ldap3/strategy/base.py", line 305, in send
    self.sending(ldap_message)
  File "/usr/lib/python3.6/site-packages/ldap3/strategy/base.py", line 829, in sending
    log(EXTENDED, 'ldap message sent via <%s>:%s', self.connection, format_ldap_message(ldap_message, '>>'))
  File "/usr/lib/python3.6/site-packages/ldap3/utils/log.py", line 184, in format_ldap_message
    for line in (message.prettyPrint().split('\n') if isinstance(message, LDAPMessage) else pformat(message).split('\n')):  # uses pyasn1 LDAP message prettyPrint() method
  File "/usr/lib/python3.6/site-packages/pyasn1/type/univ.py", line 2467, in prettyPrint
    representation, componentValue.prettyPrint(scope)
  File "/usr/lib/python3.6/site-packages/pyasn1/type/univ.py", line 1854, in prettyPrint
    representation += componentValue.prettyPrint(scope)
  File "/usr/lib/python3.6/site-packages/pyasn1/type/univ.py", line 2467, in prettyPrint
    representation, componentValue.prettyPrint(scope)
  File "/usr/lib/python3.6/site-packages/pyasn1/type/univ.py", line 926, in prettyPrint
    numbers = self.asNumbers()
  File "/usr/lib/python3.6/site-packages/pyasn1/type/univ.py", line 896, in asNumbers
    return tuple(self._value)
  File "/usr/lib/python3.6/site-packages/pyasn1/type/base.py", line 199, in plug
    raise error.PyAsn1Error('Attempted "%s" operation on ASN.1 schema object' % name)
pyasn1.error.PyAsn1Error: Attempted "__iter__" operation on ASN.1 schema object

@cannatag Attached log shows two attempts at doing a dirsync, once over plaintext, once over TLS. That was me thinking the operation could only be performed over a secure connection. But that appears to not be the case. I'm not sure if the show_deleted_control(criticality=False)] has any impact on the dirSync failing, but at any rate, it would appear that there is at least an issue with the logging.

Let me know if you need more info and/or if this should be spun out to its own issue

dirsync.log

[cannatag]

Hi, the exception is from the logging module itself. This really should not happen... :(

Are you using the latest ldap3 and pyasn1 packages? There is a test case for DirSync in the testMicrosoftAD.py test file, and it worked for me. Now I don't have easy access to an Active Directory server, so I didn't tested it with the latest updates. Can you have a look at it?

[tuftedocelot]

How curious. The tests pass successfully when I ran them against my environment so I can only assume it's something I'm doing wrong...More digging to do

ldap3.log

[cannatag]

Ok, let me know if you need more help!

Bye, Giovanni

[tuftedocelot]

Circling back around to this, I still get the error when attempting just the dir_sync. Diving into the test cases though, I don't see what I'm expecting...The test case indicates that a new user account gets created, but I don't see evidence of that in the Event Viewer when auditing user events or in the logs. Creating a user manually does yield the expected 4720 event code.

ldap3.log

eventlog.csv

[tuftedocelot]

The dir_sync test appears to be disabled?: https://github.com/cannatag/ldap3/blob/master/test/testMicrosoftAD.py#L91

Commenting out line 91 and uncommenting line 92 yields the same error I've been getting

[cannatag]

It’s disabled because I don’t have access to an Active Directory for testing anymore. It worked when I run it against a Windows 2012R2 server.

Il giorno 24 mar 2018, alle ore 20:07, tuftedocelot notifications@github.com ha scritto:

The dir_sync test appears to be disabled?: https://github.com/cannatag/ldap3/blob/master/test/testMicrosoftAD.py#L91

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[tuftedocelot]

Interesting...I've been running on Server 2016, but the forest/domain is in Win2012 mode

[tuftedocelot]

The issue seems to be with how the DirSyncControlValue is built. I can presupply a value generated from ldp.exe and DirSync works without issue. However, using the value calculated by ldap3 causes the unavailableCriticalExtension failure.

https://github.com/cannatag/ldap3/blob/master/ldap3/protocol/microsoft.py

Control:
 controlType=1.2.840.113556.1.4.841
 criticality=True
 controlValue=

Working value using ldp.exe: 0x300a02010002031000000400 Value from ldap3: 0x300e0205008000080002031000000400

My hacked up version of that method:

def build_control(oid, criticality, value, encode_control_value=True):
    control = Control()
    control.setComponentByName('controlType', LDAPOID(oid))
    control.setComponentByName('criticality', Criticality(criticality))
    if value is not None:
        if encode_control_value:
            #if oid == '1.2.840.113556.1.4.529':
            #    value = b'0\x84\x00\x00\x00\x03\x02\x01\x00'
            if oid == '1.2.840.113556.1.4.841':
                value = b'0\n\x02\x01\x00\x02\x03\x10\x00\x00\x04\x00'
            else:
                value = encode(value)
            control.setComponentByName('controlValue', value)
            #control.setComponentByName('controlValue', encode(value))
        else:
            control.setComponentByName('controlValue', value)
    print(control)
    return control

[tuftedocelot]

Further update, it would appear that setting incremental_values=True causes the error

[tuftedocelot]

I get the same behavior on a Win2012 install. @cannatag, I have a Windows 2016 and 2012R2 install in AWS set up. If you have some time, I can send you the details if you want to take a look?

[cannatag]

Sure, send me the details for accessing and I will try to understand what is going wrong.

Giovanni

Il giorno 01 apr 2018, alle ore 17:58, tuftedocelot notifications@github.com ha scritto:

I get the same behavior on a Win2012 install. @cannatag, I have a Windows 2016 and 2012R2 install in AWS set up. If you have some time, I can send you the details if you want to take a look?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cannatag]

Thanks, I've tested with your server and found that when adding or removing members from group a specific syntax ("member:range=1-1" or "member:range=0-0") was used. This was handled by the auto_range feature of ldap3. Now these ranges are excluded from being automatically processed.

Can you test the code in dev?

[cannatag]

Fixed in v2.5.