Open mathieudevos opened 5 years ago
@mathieudevos not sure I fully understand what you are asking here. Do you already have a running instance of osixia/openldap for which you want to change configuration and add .schema files ? Or do you simply want to add custom .schema files to a non-existing osixia/openldap instance ? Thanks for clarifying
Hey,
Thanks for the quick response. I have a bunch of .schema files from here: https://github.com/dkoudela/active-directory-to-openldap/tree/master/schema
And they need to be in the following order for slaptest to compile:
microsoftattributetype
microsoftattributetypestd
core (modified core from that url)
... everything else
microsoftobjectclass
To do that I will need to most likely modify the boostrap script. However I'm at a loss here with doing that. Currently I rebuilt the container with a hard copy of that folder, manually edited the schema-to-ldif.sh file to do the following:
echo "include /container/service/slapd/schemas/microsoftattributetype.schema" >> convert.dat
echo "include /container/service/slapd/schemas/microsoftattributetypestd.schema" >> convert.dat
echo "include /container/service/slapd/schemas/core.schema" >> convert.dat
echo "include /container/service/slapd/schemas/cosine.schema" >> convert.dat
echo "include /container/service/slapd/schemas/inetorgperson.schema" >> convert.dat
if [ -e "/etc/ldap/schema/rfc2307bis.schema" ]; then
echo "include /etc/ldap/schema/rfc2307bis.schema" >> convert.dat
else
echo "include /etc/ldap/schema/nis.schema" >> convert.dat
fi
for schema in ${SCHEMAS} ; do
echo "include ${schema}" >> convert.dat
done
echo "include /container/service/slapd/schemas/microsoftobjectclass.schema" >> convert.dat
Which passes the slaptest - however the server remains on "slapd starting" and never progresses beyond that. Going into the container and going to the /etc/ldap/slapd.d/cn=schema folder shows that there's just 5 files:
cn={0}core.ldif
cn={1}cosine.ldif
cn={2}nis.ldif
cn={3}inetorgperson.ldif
cn={4}ppolicy.ldif
Which has the microsoft ones missing from it.
Other ways around this? Or how would I just start with something like this? More than happy to contribute to this so that we can have a flag for windows openldap settings if we figure this out.
Fast edit: to answer your question, I will happily start from scratch on this, so a non-existing one would most likely be the valid format. I have a bunch of groups & users from a windows AD, and would like to duplicate them in an openldap container so I can test-run my program to connect to it :-).
I'm trying to do the same thing. The repository that @mathieudevos mentions above contains a customised schema that more closely matches that of Active Directory, along with a slapd.conf that contains, amongst one or two other things, some rewrite rules that influence the search behaviour. Any guidance on how we can apply this would be appreciated.
@obourdon To clarify, my intention here is to bootstrap a fresh container with the modified schema and configuration in a new image derived from osixia/openldap.
And they need to be in the following order for slaptest to compile:
have you tried naming your files like 01microsoftattributetype.schema, 02microsoftattributetypestd.schema ... ?
There are two different things at play here.
".schema" files are traditionally only for slapd.conf ".ldif" files are for use with cn=config
Second, I would note that OpenLDAP master now has an AD schema (msuser.schema/msuser.ldif).
Generally one should not be in the business of converting schema, since OpenLDAP already ships with .ldif format schema files for use with cn=config.
I'm struggling here to do some similar thing. Add custom schemas to this openldap setup
I'm struggling as well. Trying to add AD-LDS schema to OpenLdap to no success.
Hey, I would like to duplicate data from an existing Ldap server (OpenDJ) to an osixia/openLap container. I've exported data as an ldif file and also schema as another ldif file with LDap Admin tool. Default osixia schema is different from the one I've exported, so I would like to load at container creation the schema I would like. I've tried to do like the "Seed ldap database with ldif" paragraph without success. Does someone get succeed in defining its own schema ? Thanks for any help.
I'm wondering the same thing. How do get this to use active directory schema?
https://github.com/dkoudela/active-directory-to-openldap/tree/master/schema
@mathieudevos any luck?
There are two different things at play here.
".schema" files are traditionally only for slapd.conf ".ldif" files are for use with cn=config
Second, I would note that OpenLDAP master now has an AD schema (msuser.schema/msuser.ldif).
Generally one should not be in the business of converting schema, since OpenLDAP already ships with .ldif format schema files for use with cn=config.
How would I used msuser
? I cannot find it in the docker image.
@quanah
I also had this issue, and tried editing schema-to-ldif.sh with no luck. I was further able to add it by producing a new Docker image, copying my new schema (via Dockerfile) to "/container/service/slapd/assets/config/bootstrap/schema/" - see custom schema
I am also trying to figure out a solution to adding a new schema to an existing container.
I am trying to add postfix-book.schema
I managed to convert it to an ldif by creating a conf like so
include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/inetorgperson.schema
include /ldif/postfix-book.schema
but I am unsure what to do with the generated files and directory, truthfully I was expecting it to generate one file.
can I just add the schema file to /container/service/slapd/assets/config/bootstrap/ldif/custom/
Has anyone made progress on this topic? I'm also in the midst of trying to incorporate an ActiveDirectory lite schema into osixia and having no end of troubles.
You might find this useful. It's a general guide on how to convert .schema files to .ldif files for use with cn=config.
Thanks. I was writing a reply about the dependency hell I couldn't resolve, but I actually managed to convert the schema files included in https://github.com/dkoudela/active-directory-to-openldap to ldif.
Here's the config I used. The order of the schema files is key:
include microsoftattributetypestd.schema
include microsoftattributetype.schema
include core.schema
include cosine.schema
include inetorgperson.schema
include nis.schema
include microsoftobjectclass.schema
include collective.schema
include dyngroup.schema
include misc.schema
include corba.schema
include java.schema
include openldap.schema
include pmi.schema
include ppolicy.schema
include duaconf.schema
I'm at the same point as Karmis. What do I do with the resulting directory ldif files? Do I just mount the direct as a volume in osixia? Thanks.
Also as quanah suggested, I grabbed the msusers.ldif and tried messing around it.
Simply adding it as a custom ldif didn't work. There are complaints about a duplicate attribute. BTW, the encoded characters are different everytime I start the osixia container. I make sure there's no persistent storage of the ldap data.
Add custom bootstrap ldif...
Processing file /container/run/service/slapd/assets/config/bootstrap/ldif/custom/msuser.ldif
5ef26d9a conn=1024 fd=12 ACCEPT from PATH=/var/run/slapd/ldapi (PATH=/var/run/slapd/ldapi)
5ef26d9a conn=1024 op=0 BIND dn="" method=163
5ef26d9a conn=1024 op=0 BIND authcid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" authzid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
5ef26d9a conn=1024 op=0 BIND dn="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" mech=EXTERNAL sasl_ssf=0 ssf=71
5ef26d9a conn=1024 op=0 RESULT tag=97 err=0 text=
5ef26d9a conn=1024 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef26d9a conn=1024 op=1 RESULT tag=105 err=80 text=olcAttributeTypes: Duplicate attributeType: "�~4v"
5ef26d9a conn=1024 op=2 UNBIND
5ef26d9a conn=1024 fd=12 closed
5ef26d9a conn=1025 fd=12 ACCEPT from IP=127.0.0.1:58360 (IP=0.0.0.0:389)
5ef26d9a conn=1025 op=0 BIND dn="cn=admin,dc=csdpac,dc=demo" method=128
5ef26d9a conn=1025 op=0 BIND dn="cn=admin,dc=csdpac,dc=demo" mech=SIMPLE ssf=0
5ef26d9a conn=1025 op=0 RESULT tag=97 err=0 text=
5ef26d9a conn=1025 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef26d9a conn=1025 op=1 RESULT tag=105 err=50 text=
5ef26d9a conn=1025 op=2 UNBIND
5ef26d9a conn=1025 fd=12 closed
ldap_add: Other (e.g., implementation specific) error (80)
additional info: olcAttributeTypes: Duplicate attributeType: "�~4v"
adding new entry "cn=msuser,cn=schema,cn=config"
ldap_add: Insufficient access (50)
adding new entry "cn=msuser,cn=schema,cn=config"
*** /container/run/startup/slapd failed with status 50
The admin account doesn't have access to cn=config. I can't view the schema of any running osixia container. If I embed msuser.lidf as /container/service/slapd/assets/config/bootstrap/ldif/06-msuser.ldif in a Docker image that's FROM the osixia image, I still get:
adding new entry "cn=msuser,cn=schema,cn=config"
ldap_add: Insufficient access (50)
Maybe I'm going crazy, but I thought seeding the osixia container would allow for schema level seeding. What am I missing? I'm on the verge of throwing the whole thing out.
There seems to be something seriously wrong with the msuser LDIF file you're adding, given that it prints out those characters. I would note that OpenLDAP already has this file converted for use so you shouldn't need to convert it yourself.
Hi quanah.
That's the ldif I'm using.
curl https://git.openldap.org/openldap/openldap/-/raw/master/servers/slapd/schema/msuser.ldif -o ./ldif/msuser.ldif
Here's my Dockerfile:
FROM osixia/openldap
COPY ${PWD}/ldif/msuser.ldif /container/service/slapd/assets/config/bootstrap/ldif/06-msuser.ldif
I build it tagged as "test/test-ldap", and run with compose:
ldap:
image: test/test-ldap:latest
container_name: ldap
domainname: "test.local"
hostname: "ldap"
command: --copy-service --loglevel debug
ports:
- "389:389"
env_file:
- ${PWD}/ldap.env
My ldap.env just has this in it:
LDAP_LOG_LEVEL=256
LDAP_ORGANISATION=Test Inc.
LDAP_DOMAIN=test.local
LDAP_BASE_DN=dc=test,dc=local
LDAP_ADMIN_PASSWORD=admin
LDAP_TLS=false
Docker logs for the container indicates the same problem, no matter how I approach it:
...
5ef29060 conn=1024 fd=12 ACCEPT from PATH=/var/run/slapd/ldapi (PATH=/var/run/slapd/ldapi)
5ef29060 conn=1024 op=0 BIND dn="" method=163
5ef29060 conn=1024 op=0 BIND authcid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" authzid="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
5ef29060 conn=1024 op=0 BIND dn="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" mech=EXTERNAL sasl_ssf=0 ssf=71
5ef29060 conn=1024 op=0 RESULT tag=97 err=0 text=
5ef29060 conn=1024 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef29060 conn=1024 op=1 RESULT tag=105 err=80 text=olcAttributeTypes: Duplicate attributeType: "�~4�"
5ef29060 conn=1024 op=2 UNBIND
5ef29060 conn=1024 fd=12 closed
5ef29060 conn=1025 fd=12 ACCEPT from IP=127.0.0.1:34038 (IP=0.0.0.0:389)
5ef29060 conn=1025 op=0 BIND dn="cn=admin,dc=test,dc=local" method=128
5ef29060 conn=1025 op=0 BIND dn="cn=admin,dc=test,dc=local" mech=SIMPLE ssf=0
5ef29060 conn=1025 op=0 RESULT tag=97 err=0 text=
5ef29060 conn=1025 op=1 ADD dn="cn=msuser,cn=schema,cn=config"
5ef29060 conn=1025 op=1 RESULT tag=105 err=50 text=
5ef29060 conn=1025 op=2 UNBIND
5ef29060 conn=1025 fd=12 closed
ldap_add: Other (e.g., implementation specific) error (80)
additional info: olcAttributeTypes: Duplicate attributeType: "�~4�"
adding new entry "cn=msuser,cn=schema,cn=config"
ldap_add: Insufficient access (50)
adding new entry "cn=msuser,cn=schema,cn=config"
*** /container/run/startup/slapd failed with status 50
*** Run commands before finish...
*** Killing all processes...
5ef29060 daemon: shutdown requested and initiated.
5ef29060 slapd shutdown: waiting for 0 operations/tasks to finish
5ef29060 slapd stopped.
I've tried making is a custom volume mount, embedding it in a different location and setting LDAP_SEED_INTERNAL_LDIF_PATH, and as you can see above, I embedded it in my own Docker image in the same ldif directory that the osixia container will run last.
The default ldif files are run, including setting the admin password. It's when the msuser.ldif is processed that things fall apart. It looks like cn=admin doesn't have access to cn=config.
I don't doubt there could be something wrong with the ldif, or my apporach, but out-of-the-box I can access/view the installed default schema. The admin account doesn't seem to have the ACLs needed to update things.
I have no clue where those attributeType characters are coming from.
I'm not familiar enough with the ldif ACL syntax to say if the included one would impact the msuser.ldif:
dn: olcDatabase={1}{{ LDAP_BACKEND }},cn=config
changetype: modify
delete: olcAccess
-
add: olcAccess
olcAccess: to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break
olcAccess: to attrs=userPassword,shadowLastChange by self write by dn="cn=admin,{{ LDAP_BASE_DN }}" write by anonymous auth by * none
olcAccess: to * by self read by dn="cn=admin,{{ LDAP_BASE_DN }}" write by * none
There was a commit to it in April: https://github.com/osixia/docker-openldap/issues/402
I'm not familiar enough with the ldif ACL syntax to say if the included one would impact the msuser.ldif:
dn: olcDatabase={1}{{ LDAP_BACKEND }},cn=config
The Osixia container already provides full write access to the cn=config db for the root user over ldapi. There is nothing you need to change in regards to the ACLs. The above change also is completely invalid since it's modifying the olcDatabase={1}mdb,cn=config database access rules and not the access rules for the cn=config DB.
This particular error is bizarre, as it seems to indicate that somehow you have gotten non-ascii text in your LDIF file:
Duplicate attributeType: "�~�4�"
I would note there is a typo in your curl command, you have msuser.lidf instead of msuser.ldif as the destination object. Is it possible your install is picking up something other than the actual msuser.ldif from the OpenLDAP repo?
Thanks quanah. I won't pretend to understand openldap enough to diagnose this. That is a typo in my post, not in how I downloaded the file. Sorry about that.
I don't know where the (maybe utf-8) characters are coming from. It's literally the 1st non-comment entry in the msuser.ldif. I grepped through all the olcAttributeTypes and they aren't complicated. I downloaded the file directly to my CentOS8 dev env, then copy it into the image from my Dockerfile.
I also blew away all my osixia images and rebuilt. Then I inspected the msuser.ldif file inside the container and it looks the same.
Hello, what's the correct way to add a custom .schema file based on the above discussion?
/service/slapd/assets/schema-to-ldif.sh
converts .schema files into .ldif, so you don't need to convert .schema files manually:
@752319951 Only you have to do is
/container/service/slapd/assets/config/bootstrap/schema/
--copy-service
(See: https://github.com/osixia/docker-openldap/issues/301#issuecomment-724669148, https://github.com/osixia/docker-openldap#seed-ldap-database-with-ldif )/etc/ldap/slapd.d
, /var/lib/ldap
here is my docker-compose.yml
version: '3'
services:
ldap-service:
image: osixia/openldap:1.1.8
container_name: ldap-service
environment:
LDAP_ORGANISATION: example
LDAP_DOMAIN: example.com
LDAP_ADMIN_PASSWORD: admin
ports:
- "389:389"
- "636:636"
command: --copy-service
volumes:
- ldapdb:/var/lib/ldap
- ldapconf:/etc/ldap/slapd.d
- ./foobar.schema:/container/service/slapd/assets/config/bootstrap/schema/foobar.schema
@Kumassy Thank you very much, this method works!
by the way... I export whole schema file from old ldap and try to use above method but failed.. so i delete irrelevant content (maybe contents in core.schema etc.) and keep newly added content only. This process is complex for me. is there any way to use whole old schema file?
startup.sh
converts .schema -> .ldif and ldapadd
them, only when is_new_schema
returns 1
.
Writing a shell script based on these lines may work.
Sorry to labor the point, but I have existing ldap container with database -- not starting from scratch. Is it possible to add a schema in this instance? The bootstrap method seems to be for empty containers. I'd like to add ldapns.schema schema.
I don't have a good workaround for this. But I use docker-phpLDAPadmin to export/import existing LDAP records.
Here is what I try:
Hey guys, did any of you made any progress about this? I'm stuck at the same point, unable to load my active directory schema.
@752319951 Only you have to do is
* Mount .schema files to `/container/service/slapd/assets/config/bootstrap/schema/` * Start container with `--copy-service` (See: [Error volume mounting custom schema #301 (comment)](https://github.com/osixia/docker-openldap/issues/301#issuecomment-724669148), https://github.com/osixia/docker-openldap#seed-ldap-database-with-ldif ) * Persist `/etc/ldap/slapd.d`, `/var/lib/ldap`
here is my docker-compose.yml
version: '3' services: ldap-service: image: osixia/openldap:1.1.8 container_name: ldap-service environment: LDAP_ORGANISATION: example LDAP_DOMAIN: example.com LDAP_ADMIN_PASSWORD: admin ports: - "389:389" - "636:636" command: --copy-service volumes: - ldapdb:/var/lib/ldap - ldapconf:/etc/ldap/slapd.d - ./foobar.schema:/container/service/slapd/assets/config/bootstrap/schema/foobar.schema
this has not worked for me. I have worked with both ldap (mounted to /container/service/slapd/assets/config/6 bootstrap/ldif/custom/foo.ldif) and schema (mounted to container/service/slapd/assets/config/bootstrap/schema/foo.schema) files.
When using the ldif mount approach the changed schema is visible during the first startup of the container only but not on consecutive starts. I am using --copy-service and these mounts:
What can I do to investigate?
Hi :)
Did anyone managed to add new .schema after the db is created ? @kevdogg did you found a workaround may be ?
@daufinsyd Yes it's definitely possible to add schema after db is created since I added at least 2-3 new schemas. Only problem is -- I don't remember exactly how I did it since I've been working with the db for about 1-2 years now since I've done it. The bootstrap method as @christf spoke about above isn't going to work since the db is already created and the bootstrap is only valid when first starting the container.
I believe the following will work - however I'd create a backup of the database and config before trying, since I don't exactly remember how completed the process. Looking back at some of my notes (which unfortunately weren't as thorough as they should have been), here's what I came up with and I believe it should give you a start on the process:
In order to add any schema, you either need to convert the .schema file to its corresponding .ldif equivalent, or the schema might be distributed in a ldif equivalent form. If you have the ldif equivalent (for example here is the sudo schema ldif as an example - https://github.com/Lullabot/openldap-schema/blob/master/sudo.ldif) you would add the module like (I'm running this command within the actual container itself) -
ldapadd -Y EXTERNAL -H ldapi:/// -f <name of .ldif file>
If you need to convert the schema file to an ldif file, the process is a little convoluted but doable -- process is documented here: https://www.zytrax.com/books/ldap/ch6/slapd-config.html#use-schemas. Another link I found useful was: https://www.cyrill-gremaud.ch/how-to-add-new-schema-to-openldap-2-4/. And yet another video -- which kind of runs through one of the scenarios explained in the documentation (if you like to learn via videos): https://www.youtube.com/watch?v=qAedVMMunk8.
Please note I'm doing a bind mount for my volumes:
volumes:
...
- /data/ldap/config:/etc/ldap/slapd.d
When messing around with config I really find ApacheDirectoryStudio (https://directory.apache.org/studio/) helpful since it gives you a graphical representation of what is going on. You don't specifically need this tool, however its in some cases a lot quicker than querying the database directory.
Other useful commands (While in the container):
List all installed schemas:
ldapsearch -Q -Y EXTERNAL -H ldapi:/// -b cn=schema,cn=config cn
Verify config as valid (While in the container):
slaptest -F /etc/ldap/slapd.d -v
I don't know where the (maybe utf-8) characters are coming from. It's literally the 1st non-comment entry in the msuser.ldif. I grepped through all the olcAttributeTypes and they aren't complicated. I downloaded the file directly to my CentOS8 dev env, then copy it into the image from my Dockerfile.
After some experimenting, it appears the file is simply too large… :scream:
I downloaded https://git.openldap.org/openldap/openldap/-/raw/master/servers/slapd/schema/msuser.ldif and split it into 5 chunks named 00-msuser.ldif
through 04-msuser.ldif
:
$ wc -l *
996 00-msuser.ldif
1003 01-msuser.ldif
997 02-msuser.ldif
1004 03-msuser.ldif
299 04-msuser.ldif
4299 total
Then I ran:
$ docker run --volume `pwd`:/container/service/slapd/assets/config/bootstrap/schema/custom osixia/openldap:latest --copy-service -l debug
and all was well. So the bug is that apparently the container can't read files above a certain size.
Hey guys, I'm trying to figure out how to make this work as well. I have a firewall that has a directory connector for pulling down users and groups. I'd like to use this for authenticating vpn users... but when I place the msuser.ldif file in /container/service/slapd/assets/config/bootstrap/schema/custom nothing seems to change, our connector fails to be able to connect unless I use the default format for openldap... but even when I do that, users and groups do not enumerate.
It is very "easy" to add a new schema:
volumes:
- ./sudo-ldap.schema:/tmp/sudo.schema
- ./certificate:/container/service/slapd/assets/certs
- ./database:/var/lib/ldap
- ./config:/etc/ldap/slapd.d
docker exec -it ldap bash
root@ldap:/# /container/service/slapd/assets/schema-to-ldif.sh /tmp/sudo.schema
config file testing succeeded
root@ldap:/# ls /tmp/
sudo.ldif sudo.schema
root@ldap:~# ldapadd -Y EXTERNAL -H ldapi:/// -f /tmp/sudo.ldif
root@ldap:~# slaptest -F /etc/ldap/slapd.d -v
config file testing succeeded
root@ldap:~# ldapsearch -Q -Y EXTERNAL -H ldapi:/// -b cn=schema,cn=config cn
Hey,
How can you rerun the base ldap.conf so that new .schema files can be added to the setup?
I have .schema files to emulate a windows AD, but I cannot reconfigure the whole thing since all the base files are deleted.
Any pointers on how to get this running would be great.