http request benchmark, no difference between with or without cachalot

[Sispheor]

Hello guys,

I've setup cachalot on my project. I use then this script to compare the perf:

SQUEST_HOST=127.0.0.1
SQUEST_PORT=8000
LOGIN_URL="http://${SQUEST_HOST}:${SQUEST_PORT}/accounts/login/"
INSTANCE_LIST_URL="http://${SQUEST_HOST}:${SQUEST_PORT}/ui/service_catalog/instance/"
YOUR_USER='admin'
YOUR_PASS='admin'
COOKIES=/home/nico/Documents/squest/cookies.txt
CURL_BIN="curl -c $COOKIES -b $COOKIES -e $LOGIN_URL"

echo -n "Get middleware token ..."
$CURL_BIN $LOGIN_URL > /dev/null
DJANGO_MIDDLEWARE_TOKEN="csrfmiddlewaretoken=$(grep csrftoken $COOKIES | sed 's/^.*csrftoken\s*//')"

echo "${DJANGO_MIDDLEWARE_TOKEN}"

echo -n "Perform login ..."
$CURL_BIN \
    -d "${DJANGO_MIDDLEWARE_TOKEN}&username=${YOUR_USER}&password=${YOUR_PASS}" \
    -X POST $LOGIN_URL

DJANGO_TOKEN="$(grep csrftoken $COOKIES | sed 's/^.*csrftoken\s*//')"
echo "${DJANGO_TOKEN}"
SESSION_ID="$(grep sessionid $COOKIES | sed 's/^.*sessionid\s*//')"
echo "${SESSION_ID}"
COOKIE_STRING="Cookie: csrftoken=${DJANGO_TOKEN};sessionid=${SESSION_ID}"
#echo -n "Get list instance via curl"
#$CURL_BIN \
#    -d "csrftoken=$DJANGO_TOKEN" \
#    -X GET $INSTANCE_LIST_URL

httperf \
--server ${SQUEST_HOST} \
--port ${SQUEST_PORT} \
--uri /ui/service_catalog/instance/ \
--num-conns 100 \
--rate 10 \
--timeout 5 \
--add-header "$COOKIE_STRING\n"

loadtest -n 100 -k \
-H "$COOKIE_STRING" \
http://${SQUEST_HOST}:${SQUEST_PORT}/ui/service_catalog/instance/

The script uses httperf and loadtest .

I was expecting some perf enhancement when I enable cachalot but I have the same number of request per second.

I've checked with a script that redis was actually "moving".

import redis
r = redis.Redis(host='localhost', port=6379, db=0, password="redis_secret_password")
print(len(r.keys())

I can see that the number of keys changes when I browse some pages of my app.

Is that relevant to benchmark from an outside tool like I did? How can I check that this lib actually improved my GET pages?

Many thanks !

My conf

INSTALLED_APPS = [
'.....',
'cachalot',
'.....',
]

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": f"redis://{REDIS_CACHE_HOST}:{REDIS_CACHE_PORT}/0",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "PASSWORD": REDIS_CACHE_PASSWORD
        }
    }
}

[Andrew-Chen-Wang]

If you're just looking to see how cachalot improved db queries, then you can create a timer object around where you're calling your queries. There's a chance most of the time is coming from latency, not the actual queries. This is especially try if you don't have enough data or your queries are not complicated.

The changing of keys might be a concern. It depends on what your query is and if you're doing any INSERTs/UPDATEs (which I don't think you are since this is a GET request).

[Sispheor]

Thanks for your answer. Yes the executed benchmark only perform a get operation.

Actually I tried at first with the default "site based cache" and the script help me to identify that I passed from 9 request to 80 request per seconds. But... after updating a model the same page was still sent 😅. And so I discovered cachalot which invalidate in case of INSERT/UPDATE/DELETE.

But, after adding it I was expecting to retrieve the same result as the full site cache based. But no, with or without the cachalot app enabled I have around 9 request per seconds...

[Andrew-Chen-Wang]

Actually I tried at first with the default "site based cache" and the script help me to identify that I passed from 9 request to 80 request per seconds. But... after updating a model the same page was still sent 😅.

I believe that's supposed so happen. Site based cache uses HTTP headers to denote that a request should re-utilize a client cache of the website previously given (I think through Cache-Control).