Open angrosz opened 5 years ago
Contributions to this driver aiming at supporting cluster topology events are welcome. So would be improving refresh()
to support calls at runtime (see https://github.com/Yelp/casper/pull/47), which would be a nice first step.
Same issue. News ?
@nbisson No news on this front. Contributions still welcome.
We are suffering from this problem too.
As a workaround we had to come up with a very ugly solution:
Each Kong runs with custom nginx server block:
server {
server_name kong-reload;
listen 0.0.0.0:8333;
location /kong-reload {
content_by_lua_block {
os.execute("/usr/local/bin/kong reload")
}
}
access_log /dev/stdout;
error_log /dev/stderr error;
}
That allows to call endpoint "/kong-reload" on the Kong node and that executes command /usr/local/bin/kong reload
which under the hood causes sending SIGHUP to Kong nginx process. As a result Kong reloads its configuration and re-discovers Cassandra cluster members.
nodetool status
, parces output and keeps track of IPs of your current Cassandra cluster memebers. Once it realises that the list of Cassandra IPs becomes different - it calls http://kong-node:8333/kong-reload .Here is example of my script.
#!/bin/bash
# This script is to be used as an entrypoint for Cassandra-watcher.
# It is running infinite loop with periodically checking if
# the list of Cassandra cluster IPs has changed.
# If the list changed, then it calls Kong reload HTTP endpoint.
# If the list isn't changed it does nothing and waits for the next check.
# Cassandra container with this script as antrypoint should be running as
# a side-car container in the POD that runs Kong.
# Obtains list of Cassandra cluster IPs and:
# - stores list in provided file location
# - returns list as resul
# Params:
# 1. File to save the list.
get_cassandra_ips() {
IPS_FILE=$1
nodetool -h cassandra status | grep UN | awk '{print $2}' | tee $IPS_FILE
}
# Compare two files.
# Params:
# 1. file to compare
# 2. file to compare
# Returns usual return code of diff tool which is:
# - 0 if files are identical
# - 1 if files are different
if_file_lists_same() {
file_list1=$1
file_list2=$2
# Use standard diff tool to check if lists are different.
# Soft both files before diff comparision.
diff -q <(sort $file_list1) <(sort $file_list2) > /dev/null 2>&1
}
get_time() {
date "+%Y-%m-%d %H:%M:%S"
}
# In the beginning we initialise PREV_IPS with list of current IPs
PREV_IPS=/tmp/prev_cluster_ips
get_cassandra_ips $PREV_IPS
CURRENT_IPS=/tmp/current_cluster_ips
# Expose ability to configure Cassandra discovery period in seconds
# via K8s container env variable. If such env var is defined and not
# empty via K8s container env variable then use that value.
# Defaults to 60 seconds.
CASSANDRA_DISCOVERY_PERIOD=${CASSANDRA_DISCOVERY_PERIOD:-60}
while [ true ]
do
echo "$(get_time): Waiting $CASSANDRA_DISCOVERY_PERIOD seconds to check Cassandra IPs again."
sleep $CASSANDRA_DISCOVERY_PERIOD
echo "$(get_time): Checking current Cassandra cluster ips:"
get_cassandra_ips $CURRENT_IPS
# Special EGDE case!
# If the current list of Cassandra IPs is empty - do not reload Kong!
if [ ! -s $CURRENT_IPS ]
then
echo "$(get_time): Unable to detect any Cassandra cluster IPs. Not going to reload Kong."
continue
# If current list of IPs is different from previous list
# then at least one Cassandra nodes restarted and we should:
# - SIGHUP Kong
# - Save current list of IPs as previous
elif ! if_file_lists_same $CURRENT_IPS $PREV_IPS
then
echo "$(get_time): Detected Cassandra IPs change!"
echo "$(get_time): Reloading Kong"
curl http://127.0.0.1:8333/kong-reload
mv $CURRENT_IPS $PREV_IPS
else
echo "$(get_time): Cassandra IPs are the same."
fi
done
The solution has a lot of drawbacks and requires maintenance.
Ideally Kong should be able to rediscover Cassandra cluster members via this cassandra db driver.
Hi, I'm working with Kong 0.13.1 that uses lua-cassandra 1.2.3 and OpenResty 1.11.2.5. Kong talks to Cassandra 3.11.2. My problem is that Kong doesn't keep track of changing Cassandra cluster nodes. Details are following:
My question is: do you plan to enhance lua-cassandra driver to keep track of Cassandra cluster nodes changes (as I described above)?
Best Regards