Be able to synchronize the TTL of cache-control header to expire in a…

[dgaubert]

… coherent way

[dgaubert]

Tests in staging

After deploying this branch in staging, I've created a "non-cartodbfied" table and I made some requests through CDN in order to check the behavior of the new TTL to have synced invalidation for free.

For the following query select cartodb_id, random() from test_ttl limit 1; where each call should give a unique result. For instance:

=> select cartodb_id, random() from test_ttl limit 1;
 cartodb_id |      random
------------+------------------
          1 | 0.45452244579792
(1 row)

=> select cartodb_id, random() from test_ttl limit 1;
 cartodb_id |      random
------------+-------------------
          1 | 0.774238015525043
(1 row)

I've sent the following requests (omitted non-relevant headers):

Where:

• Cache-Control: max-age is the max time (in seconds) in which the response should be cached.
• Last-Modified as the table is not registered it tells the time when the request was served.

• Age: Time in seconds where the response remains stored in the cache.

  1. Non cached, 00:02:21 to 15:45:00:

     $ curl -v 'https://carto-staging-1.global.ssl.fastly.net/dgaubert/api/v2/sql?q=select%20cartodb_id%2C%20random()%20from%20test_ttl%20limit%201&api_key=5310****e2d5'
     < Cache-Control: no-cache,max-age=141,must-revalidate,public
     < Last-Modified: Tue, 15 Oct 2019 15:42:38 GMT
     < Age: 0
     < X-Cache: MISS, MISS
     < X-Cache-Hits: 0
     {"rows":[{"cartodb_id":1,"random":0.762340249028057}],"time":0.002,"fields":{"cartodb_id":{"type":"number","pgtype":"int4"},"random":{"type":"number","pgtype":"float8"}},"total_rows":1}

  2. Cached, 135 seconds living in cache (6 remaining to expire the TTL):

     $ curl -v 'https://carto-staging-1.global.ssl.fastly.net/dgaubert/api/v2/sql?q=select%20cartodb_id%2C%20random()%20from%20test_ttl%20limit%201&api_key=5310****e2d5'
     < Cache-Control: no-cache,max-age=141,must-revalidate,public
     < Last-Modified: Tue, 15 Oct 2019 15:42:38 GMT
     < Age: 135
     < X-Cache: MISS, HIT
     < X-Cache-Hits: 1
     {"rows":[{"cartodb_id":1,"random":0.762340249028057}],"time":0.002,"fields":{"cartodb_id":{"type":"number","pgtype":"int4"},"random":{"type":"number","pgtype":"float8"}},"total_rows":1}

  3. Cached, 140 seconds living in cache (1 remaining to expire the TTL):

     $ curl -v 'https://carto-staging-1.global.ssl.fastly.net/dgaubert/api/v2/sql?q=select%20cartodb_id%2C%20random()%20from%20test_ttl%20limit%201&api_key=5310****e2d5'
     < Cache-Control: no-cache,max-age=141,must-revalidate,public
     < Last-Modified: Tue, 15 Oct 2019 15:42:38 GMT
     < Age: 140
     < X-Cache: MISS, HIT
     < X-Cache-Hits: 1
     {"rows":[{"cartodb_id":1,"random":0.762340249028057}],"time":0.002,"fields":{"cartodb_id":{"type":"number","pgtype":"int4"},"random":{"type":"number","pgtype":"float8"}},"total_rows":1}

  4. Non cached; see the response, the random column should provide a different number:

     $ curl -v 'https://carto-staging-1.global.ssl.fastly.net/dgaubert/api/v2/sql?q=select%20cartodb_id%2C%20random()%20from%20test_ttl%20limit%201&api_key=5310****e2d5'
     < Cache-Control: no-cache,max-age=296,must-revalidate,public
     < Last-Modified: Tue, 15 Oct 2019 15:45:03 GMT
     < Age: 0
     < X-Cache: MISS, MISS
     < X-Cache-Hits: 0
     {"rows":[{"cartodb_id":1,"random":0.111140322405845}],"time":0.005,"fields":{"cartodb_id":{"type":"number","pgtype":"int4"},"random":{"type":"number","pgtype":"float8"}},"total_rows":1}

Note: See the new Last-Modified and the max-age should give an idea when the next TTL is going to expire (15:50)