Track API requests with Piwik

[MattiSG]

On each API call, send a “pageview” to a Piwik instance. This will allow for usage analytics. This tracking must be nonblocking and have only a minimal impact on performance. A library such as grequests should be used on Piwik HTTP API, or a specific wrapper such as piwikapi.

The API must still work if the tracker is not installed.

• [x] Create a openfisca/tracker package.
• [x] Send a “pageview” on each API call, with the URL being the called endpoint URL.
• [x] Measure performance impact: benchmark the time to handle 100 computations, 100 parameters requests, before and after the change, in 3 different runs.
• [x] Document that behaviour, explain why this is done, explain why there are no CNIL & EU privacy requirements in the README of that package.
• [x] Add a target URL setup to opt-in to tracking (TRACKER=https://stats.openfisca.org).
• [x] Add an information line in the doc.
• [x] Output a reassuring information message on API start if tracking is set up.

[fpagnoux]

We set up a trial innocraft to test.

Example of a request to track an event:

http://openfisca.innocraft.cloud/piwik.php?idsite=1&rec=1&url=http%3A%2F%2Fgoogle.fr

[fpagnoux]

Performances

According to the script provided, we get the following results

/variables route without tracking
100 calls take 0.112409 s
/calculate route without tracking
100 calls take 0.582618 s
/variables route with tracking
100 calls take 0.382652 s
/calculate route with tracking
100 calls take 1.221149 s

This seems to indicate a factor of 2 to 4 when tracking, which seems a lot. The absolute overhead varies from 3 to 6 ms/request.

However, this script doesn't use a real app, but a flask test client, which is really fast. If I instanciate a real app, I get the following results:

Without tracking, for route /variables

1.598 
1.611 
1.618

=> 1.609

With tracking, for route /variables

1.797 
1.896 
1.807

=> 1.833
=> + 14 %
=> + 2 ms/req

Without tracking, for route /calculate

11.986
11.173
11.929

=> 11.696

with tracking, for route /calculate
12.316
14.087
12.870

=> 13.091
=> + 12 %
=> + 14 ms/req

Commands used:

time (for i in {1..100}; do; curl  -o /dev/null -s "localhost:5000/variables"; done;) 
time -f "" (for i in {1..100}; do; cat test.json | lwp-request -m POST -c application/json http://localhost:5000/calculate > /dev/null; done;)

The overhead seems more reasonable, but still significant.

Note that this second set of tests have been done with a local api. The impact should be relatively smaller on the real API.

Today, getting https://api-test.openfisca.fr/variables takes 800ms. The measured overhead by request when using the tracker is about 14ms.

If we assume this overhead doesn't increase much when we switch to a distant API, the expected overhead for the user is about 2%.

[MattiSG]

Anything more than 2% seems way too much to me.

[fpagnoux]

After introducing buffering (We send the tracking data every 30 requests, and every hour), we significantly reduce the overhead:

On average:

For 200 /calculate requests:

• 22.895s without tracking
• 23.08s with tracking
• +0.8%

For 200 /variables requests:

• 3.197s without tracking
• 3.215s with tracking
• +0.6%

The performance impact is thus acceptable.