Open vvirehead opened 5 years ago
@michalgardela not sure if this will fix your issue but you have 2 occurrences of entry command
in the openldap section of your docker compose file (--copy-service
and --loglevel debug
). The --copy-service
which is mandatory in your case as you mount volumes might therefore be uneffective/not taken into account and therefore the failure
Please also note that besides you have LDAP_TLS_VERIFY_CLIENT: "never"
set in your docker-compose.yml file the traces show LDAP_TLS_VERIFY_CLIENT = demand
which is also weird.
Also the fact that LDAP_READONLY_USER_PASSWORD
seems also present in the trace but not in docker-compose.yml.
Some other traces you gave which can be worth looking at:
*** failed to link /container/service/:ssl-tools/startup.sh to /container/run/startup/:ssl-tools: [Errno 17] File exists
*** link /container/service/slapd/startup.sh to /container/run/startup/slapd
*** failed to link /container/service/slapd/startup.sh to /container/run/startup/slapd: [Errno 17] File exists
*** link /container/service/slapd/process.sh to /container/run/process/slapd/run
*** directory /container/run/process/slapd already exists
*** failed to link /container/service/slapd/process.sh to /container/run/process/slapd/run : [Errno 17] File exists
Please note also that in 1.2.5 some strict checking has been introduced for environment variable values and you might end up with issues like:
Error: domain mycompany.org derived from LDAP_BASE_DN dc=mycompany,dc=org does not match LDAP_DOMAIN example.org
. You might want to add LDAP_DOMAIN to match your other values in your docker-compose.yml file.
Reading this link give me questions on why you also use CONTAINER_SERVICE_DIR
in your docker-compose.yml file but may be that can be forgotten for now.
On my side, I do not manage to get the trace starting at openldap | Check previous TLS certificates...
but again I might not be in the exact same situation.
HTH
Hi,
I do have the same issues as @michalgardela , the docker-compose file is similar too , mounted cert from outside, but during the startup, it always complains "No certificate file and certificate key provided
" then trying to generating a new one. Finally it says "chmod: cannot access '/container/run/service/slapd/assets/certs/dhparam.pem': No such file or directory
" then failed.
I could not understand quite well what @obourdon suggest to resolve ...
@ajeecai Can you also post the contents of your docker-compose.yml file please ?
By experience, similar
is not something we can rely on to do proper debugging and help you out. Debug logs or any additional information you might think useful is also greatly appreciated so that we do not waste anybody's time looking in bad directions.
In my previous post, I was just suggesting to pay more attention to the contents of docker-compose.yml file because of duplicates entry command
which might overwrite one another. Additionally, some other weird "options" like CONTAINER_SERVICE_DIR
might not be useful or might have some hidden effect.
Some fixes in 1.2.5 release like #341 might also have impacts on setups which worked with previous version (to be checked of course, might not be the case for all existing environments)
Hope this makes sense
Hi,
In order not to copy paste too long lines here, I have attached the compose file and log here, with some sensitive information replaced with "myexample". I have two commands, just following some snippet code from internet, not sure what copy-service is and if working to simply combine them into one
command: ["--copy-service"]
command: --loglevel debug
I have tried to removed one of them, still get the error and fail to start the docker.
Hi @obourdon Do you have any suggestion?
Thanks
I encountered the same issue with versions 1.2.3 and above. Reverting to 1.2.2 and below makes the problem go away which is similar to the findings in the OP of #283.
docker-compose.yml:
version: '2.3'
services:
openldap:
cpus: 1
mem_limit: 1024m
restart: always
image: osixia/openldap:1.3.0
command: "--loglevel debug"
environment:
LDAP_TLS_CRT_FILENAME: "cert.pem"
LDAP_TLS_KEY_FILENAME: "privkey.pem"
LDAP_TLS_CA_CRT_FILENAME: "chain.pem"
LDAP_TLS_VERIFY_CLIENT: "allow"
volumes:
- ./data/conf:/etc/ldap/slapd.d
- ./data/db:/var/lib/ldap
- ./data/certs:/container/service/slapd/assets/certs
ports:
- "389:389"
- "636:636"
My debug logs are the same as @ajeecai.
Same for me, I always get
openldap | /!\ WARNING: LDAP_TLS=false but the container was previously started with LDAP_TLS=true
openldap | TLS can't be disabled once added. Ignoring LDAP_TLS=false.
with the latest version. Using 1.2.2 works.
same problem
docker info
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 4
Server Version: 17.05.0-ce
Storage Driver: overlay
Backing Filesystem: extfs
Supports d_type: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-229.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.127GiB
Name: iast-centos
ID: CN3K:PLL4:YHKE:LNQH:AMX5:FHEH:D64U:XXYP:MBYK:GVZH:LK5F:2A53
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://100.104.67.9:3128
Https Proxy: http://100.104.67.9:3128
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
the openldap logs
*** CONTAINER_LOG_LEVEL = 3 (info)
*** Copy /container/service to /container/run/service
*** Search service in CONTAINER_SERVICE_DIR = /container/run/service :
*** link /container/run/service/:ssl-tools/startup.sh to /container/run/startup/:ssl-tools
*** link /container/run/service/slapd/startup.sh to /container/run/startup/slapd
*** link /container/run/service/slapd/process.sh to /container/run/process/slapd/run
*** Set environment for startup files
*** Environment files will be proccessed in this order :
Caution: previously defined variables will not be overriden.
/container/environment/99-default/default.startup.yaml
/container/environment/99-default/default.yaml
To see how this files are processed and environment variables values,
run this container with '--loglevel debug'
*** Running /container/run/startup/:ssl-tools...
*** Running /container/run/startup/slapd...
Database and config directory are empty...
Init new ldap server...
Backing up /etc/ldap/slapd.d in /var/backups/slapd-2.4.48+dfsg-1~bpo10+1... done.
Creating initial configuration... done.
Creating LDAP directory... done.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of restart.
Start OpenLDAP...
Waiting for OpenLDAP to start...
Add bootstrap schemas...
config file testing succeeded
mv: cannot move './cn=config/cn=schema/cn={4}dhcp.ldif' to './dhcp.ldif': Operation not permitted
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
sed: can't read dhcp.ldif: No such file or directory
mv: cannot stat 'dhcp.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={5}dnszone.ldif' to './dnszone.ldif': Operation not permitted
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
sed: can't read dnszone.ldif: No such file or directory
mv: cannot stat 'dnszone.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={6}mail.ldif' to './mail.ldif': Operation not permitted
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
sed: can't read mail.ldif: No such file or directory
mv: cannot stat 'mail.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={7}mmc.ldif' to './mmc.ldif': Operation not permitted
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
sed: can't read mmc.ldif: No such file or directory
mv: cannot stat 'mmc.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={8}openssh-lpk.ldif' to './openssh-lpk.ldif': Operation not permitted
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
sed: can't read openssh-lpk.ldif: No such file or directory
mv: cannot stat 'openssh-lpk.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={9}quota.ldif' to './quota.ldif': Operation not permitted
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
sed: can't read quota.ldif: No such file or directory
mv: cannot stat 'quota.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={10}radius.ldif' to './radius.ldif': Operation not permitted
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
sed: can't read radius.ldif: No such file or directory
mv: cannot stat 'radius.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={11}samba.ldif' to './samba.ldif': Operation not permitted
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
sed: can't read samba.ldif: No such file or directory
mv: cannot stat 'samba.ldif': No such file or directory
mv: cannot move './cn=config/cn=schema/cn={12}zarafa.ldif' to './zarafa.ldif': Operation not permitted
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
sed: can't read zarafa.ldif: No such file or directory
mv: cannot stat 'zarafa.ldif': No such file or directory
Add image bootstrap ldif...
Add custom bootstrap ldif...
Add TLS config...
No certificate file and certificate key provided, generate:
/container/run/service/slapd/assets/certs/ldap.crt and /container/run/service/slapd/assets/certs/ldap.key
2020/02/18 08:41:58 [INFO] generate received request
2020/02/18 08:41:58 [INFO] received CSR
2020/02/18 08:41:58 [INFO] generating key: ecdsa-384
2020/02/18 08:41:58 [INFO] encoded CSR
2020/02/18 08:41:58 [INFO] signed certificate with serial number 667732584405916187938769008644749019727189510820
mv: cannot move '/tmp/cert.pem' to '/container/run/service/slapd/assets/certs/ldap.crt': Operation not permitted
mv: cannot move '/tmp/cert-key.pem' to '/container/run/service/slapd/assets/certs/ldap.key': Operation not permitted
Link /container/run/service/:ssl-tools/assets/default-ca/default-ca.pem to /container/run/service/slapd/assets/certs/ca.crt
*** /container/run/startup/slapd failed with status 80
It might be related with this piece of code. Quoting:
if the config was bootstraped with TLS to avoid error (#6) (#36) and (#44) we create fake emporary certificates if they do not exists
It relies on checking the existence of a so-called "file flag": /etc/ldap/slapd.d/docker-openldap-was-started-with-tls (+ executing its content, more on this below).
The thing is: that file might exist from a previous run where TLS had been enabled, and remain in a volume mapped to /etc/ldap/slap.d (which is very likely if you've abode by the README instructions).
That file has executable content: it exports env var which end up "messing" up with the container starting.
For instance, in a different scenario than yours but resulting in the same "bug", I simply renamed my certificates files (cert and private key) from .crt to .pem-but the file flag triggers, and it exports unwanted values:
sudo cat /var/lib/docker/volumes/stack-ldap-server_config/_data/docker-openldap-was-started-with-tls
export PREVIOUS_LDAP_TLS_CA_CRT_PATH=/container/service/slapd/assets/certs/certificate.crt
export PREVIOUS_LDAP_TLS_CRT_PATH=/container/service/slapd/assets/certs/certificate.crt
export PREVIOUS_LDAP_TLS_KEY_PATH=/container/service/slapd/assets/certs/privatekey.key
export PREVIOUS_LDAP_TLS_DH_PARAM_PATH=/container/service/slapd/assets/certs/dhparam.pem
Deleting this file in the docker volume fixes the issue (container starts without generating certificate).
Deleting this file in the docker volume fixes the issue (container starts without generating certificate). i tried this and the docker does not "crash" anymore but instead gives me this messages:
updating file uid/gid ownership
Start OpenLDAP...
Waiting for OpenLDAP to start...
5f6b67cf @(#) $OpenLDAP: slapd 2.4.50+dfsg-1~bpo10+1 (May 4 2020 05:25:06) $
Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>
TLS: could not use CA certificate file `/container/service/slapd/assets/certs/ca.crt': Error while reading file. (-64)
5f6b67cf main: TLS init def ctx failed: -1
5f6b67cf slapd stopped.
5f6b67cf connections_destroy: nothing to destroy.
For me the issue comes up when testing my backup of openldap. i basicly just copied over all the compose files and volumes and am trying to get them startet. (btw the ip address is different for my test server, thats why i might get ca issues)
@cgsecret do you know how to solve the error:
TLS: could not use CA certificate file `/container/service/slapd/assets/certs/ca.crt': Error while reading file. (-64)
?
I opened an issue for it: #548.
I don't fully understand what you mean by the ip address is different for my test server, thats why i might get ca issues
. Do you mean that the certificate is created for a different hostname than the LDAP server in the container?
try this it has worked for me with several different types of certbots. first make sure whatever files you give it there not "softlinks" it seems to just copies instead of following the softlinks so the path will typically fail. so when it trys to verify the cert it will fail generating new certs.
I had no luck mounting my certs in the asset folder do to ownership issues clashing (but it works great if you build your container instead of pulling. Then you don't need the seeding vars but the filenames need to match)
mount your certs somewhere inside the container and use both env vars for "internal seeding" and "filenames"
ldap3:
image: osixia/openldap:1.5.0
environment:
LDAP_TLS: "true"
### Copy seed files from internal path if specified NOTE THIS IS NOT THE "SOFTLINKED" LIVE DIR
LDAP_SEED_INTERNAL_LDAP_TLS_CRT_FILE: /etc/letsencrypt/archive/example.com/cert1.pem
LDAP_SEED_INTERNAL_LDAP_TLS_KEY_FILE: /etc/letsencrypt/archive/example.com/privkey1.pem
LDAP_SEED_INTERNAL_LDAP_TLS_CA_CRT_FILE: /etc/letsencrypt/archive/example.com/fullchain1.pem
LDAP_SEED_INTERNAL_LDAP_TLS_DH_PARAM_FILE: /etc/nginx/dhparams.pem
###cert names i dont think the names matter as long as there not default ldap.*etc
LDAP_TLS_CRT_FILENAME: "cert.pem"
LDAP_TLS_KEY_FILENAME: "privkey.pem"
LDAP_TLS_DH_PARAM_FILENAME: "dhparam.pem"
LDAP_TLS_CA_CRT_FILENAME: "fullchain.pem"
LDAP_TLS_ENFORCE: "true"
LDAP_TLS_CIPHER_SUITE: SECURE256:+SECURE128:-VERS-TLS-ALL:+VERS-TLS1.2:-RSA:-DHE-DSS:-CAMELLIA-128-CBC:-CAMELLIA-256-CBC
command: "--copy-service --loglevel debug"
# For replication to work correctly, domainname and hostname must be
# set correctly so that "hostname"."domainname" equates to the
# fully-qualified domain name for the host.
domainname: "example.com"
hostname: "ldap"
volumes:
- ${PWD}/etc/letsencrypt/:/etc/letsencrypt/:ro
- ${PWD}/swag/nginx/dhparams.pem:/etc/nginx/dhparams.pem:ro
Deleting this file in the docker volume fixes the issue (container starts without generating certificate). i tried this and the docker does not "crash" anymore but instead gives me this messages:
updating file uid/gid ownership Start OpenLDAP... Waiting for OpenLDAP to start... 5f6b67cf @(#) $OpenLDAP: slapd 2.4.50+dfsg-1~bpo10+1 (May 4 2020 05:25:06) $ Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org> TLS: could not use CA certificate file `/container/service/slapd/assets/certs/ca.crt': Error while reading file. (-64) 5f6b67cf main: TLS init def ctx failed: -1 5f6b67cf slapd stopped. 5f6b67cf connections_destroy: nothing to destroy.
For me the issue comes up when testing my backup of openldap. i basicly just copied over all the compose files and volumes and am trying to get them startet. (btw the ip address is different for my test server, thats why i might get ca issues)
May check apparmor already add permission for cert files.
Original cert files path is setting to /etc/ssl/private/
,
you can add new condition in /etc/apparmor.d/usr.sbin.slapd
to work with your own path.
/path/to/dir/ r,
/path/to/dir/* r,
Then restart the service systemctl restart apparmor
.
Hi,
I'm stuck with this problem - it looks like OpenLDAP is trying to create certs (for TLS?), but somehow cannot access /container/run/service directory (doesn't exists):
Funny thing TLS is disabled...
This happened just today - I'm not sure if the image was updated or something else changed in the server (Watchtower is being used for container updates, also automatic system updates are enabled).
A year ago we tried to enable TLS, but for some reasons we dropped it.
So I manually created missing directories with docker exec, and now I get:
I've tried 1.2.5 and latest - nothing works.
Here's my docker-compose.yml:
Edit:
Running
slapd -d 4
from inside container gives this error:Kindly please - help required.
Thank you in advance!