Open jstaf opened 1 month ago
I got this to work with a custom solr image with a custom version of solr.in.sh
and zkcli.sh
. Still needs to be fixed in solr-operator though.
solr.in.sh (this change has solr zk
auto-trust the zk tls certificate and obey settings from $SOLR_OPTS
set by solrZkOpts
)
#!/bin/bash
set -eo pipefail
# import zk certificate into default system truststore if not present
if [ ! -f /tmp/cacerts.jks ]; then
# this step is extremely race-y for some reason
sleep $((RANDOM % 10))
echo |\
openssl s_client -connect $(echo $ZK_SERVER | sed 's/,.*//g') -showcerts |\
sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/ca.crt
# we create a copy of the system truststore in /tmp since the root
# filesystem is not writeable >:(
cp /opt/java/openjdk/lib/security/cacerts /tmp/cacerts.jks
keytool -import -noprompt -trustcacerts -alias solr-ca -file /tmp/ca.crt -keystore /tmp/cacerts.jks -storepass changeit
fi
# have "solr zk" and zkcli.sh use $SOLR_OPTS
SOLR_TOOL_OPTS="$SOLR_OPTS -Dzookeeper.ssl.trustStore.location=/tmp/cacerts.jks -Dzookeeper.ssl.trustStore.password=changeit"
ZKCLI_JVM_FLAGS="$SOLR_TOOL_OPTS"
zkcli.sh (make this script actually obey $SOLR_OPTS
by sourcing our custom solr.in.sh
)
#!/usr/bin/env bash
# You can override pass the following parameters to this script:
#
JVM="java"
# Find location of this script
sdir="`dirname \"$0\"`"
log4j_config="file:$sdir/../../resources/log4j2-console.xml"
solr_home="$sdir/../../solr"
# Settings for ZK ACL
#SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.DigestZkACLProvider \
# -DzkCredentialsProvider=org.apache.solr.common.cloud.DigestZkCredentialsProvider \
# -DzkCredentialsInjector=org.apache.solr.common.cloud.VMParamsZkCredentialsInjector \
# -DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD \
# -DzkDigestReadonlyUsername=readonly-user -DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD"
# optionally, you can use using a a Java properties file 'zkDigestCredentialsFile'
#...
# -DzkDigestCredentialsFile=/path/to/zkDigestCredentialsFile.properties
#...
source /etc/default/solr.in.sh # this is the only change
PATH=$JAVA_HOME/bin:$PATH $JVM $SOLR_ZK_CREDS_AND_ACLS $ZKCLI_JVM_FLAGS -Dlog4j.configurationFile=$log4j_config -Dsolr.home=$solr_home \
-classpath "$sdir/../../solr-webapp/webapp/WEB-INF/lib/*:$sdir/../../lib/ext/*:$sdir/../../lib/*" org.apache.solr.cloud.ZkCLI ${1+"$@"}
Tried setting up solr-operator with TLS between Solr and Zookeeper. (Zookeeper is setup with TLS, and Solr must connect to Zookeeper's TLS port.) It doesn't work or appear to have ever been tested.
Summary of problems I found before giving up:
alert: protocol_version
until you downgrade).spec.solrZkOpts
completely... spec.solrZkOpts sets SOLR_OPTS, but thesolr zk
andzkCli.sh
commands ignore SOLR_OPTS.solr zk
usesSOLR_TOOL_OPTS
notSOLR_OPTS
(and this is specific to Solr 9... Solr 8 uses a different env var) https://github.com/apache/solr/blob/main/solr/bin/solr#L601-L610zkcli.sh
usesZKCLI_JVM_FLAGS
notSOLR_OPTS
solr zk
andzkcli.sh
in the initContainers, you can't add the certificates to setup TLS to Zookeeper anyways.To reproduce the issue:
Setup Zookeeper with TLS using the Bitnami Helm chart
helm install zookeeper bitnami/zookeeper -f zookeeper.yml
. Examplezookeeper.yml
(uses TLS for encryption, but not for authentication):Then install solr-operator, and create a SolrCloud CRD. In this case I've generated some Certificate resources via cert-manger that the SolrCloud is using...