Esmond cleanup process does not succeed on freeing space

[igarny]

Hello Andy,

We are observing continuous growth of the DB despite our efforts to reduce it. Our deployment is:

• CentOS 6.9
• esmond-2.1.1-1.el6.x86_64

Attached:

• /etc/perfsonar/toolkit/clean_esmond_db.conf clean_esmond_db.zip

• extract of cassandra logs cass_log.tar.gz

From the logs of cassandra there are statements that around 16GB were freed every day for about a week (October 13-20). Meaning 1/10 of the DB was freed each day. Nonetheless this cannot be observed on our monitoring. http://perfsonar-smallnodes.geant.org/grapherrd/grapherrd.php?page=perfsonar.cfg&target=ps-sn-central&graph=yearly&style=extralong

We are using the default script

ps-sn-central ~]$ cat /usr/lib/perfsonar/scripts/clean_esmond_db.sh
#!/bin/bash
export ESMOND_ROOT=/usr/lib/esmond
export ESMOND_CONF=/etc/esmond/esmond.conf
export DJANGO_SETTINGS_MODULE=esmond.settings
cd $ESMOND_ROOT
if [ -e /opt/rh/python27/enable ]; then
    source /opt/rh/python27/enable
fi
. bin/activate
python ./util/ps_remove_data.py -c /etc/perfsonar/toolkit/clean_esmond_db.conf

We are observing similar behavior on our deployment of pS Toolkit with SmallNodes

Regards, Ivan