Create test files - Githubissues

kiwix / mirrors-qa

Q/A tools for Kiwix Download Mirrors

GNU General Public License v3.0

1 stars 0 forks source link

Create test files #3

Closed rgaudin closed 2 months ago

rgaudin commented 4 months ago

All the mirrors must serve the same file for use with the speed test. Location and name needs to accommodate all servers. Although it should not require any mirror intervention, an email must be sent to admins informing them about this new file and its purpose.

File should be random binary data and weight 50MB. We may also host a 1MB file for simulation/devel for use where speed accuracy is not important

rgaudin commented 4 months ago

Constraints:

[X] - .zim named so it's synced by mirrors without action on their part
[X] - an actual ZIM file so it's not deceptive (explains it's not actually content)
[ ] - must be mirrored by all mirrors (or most of them) ; in case we cant have them all, we need a ticket to exclude from #12
[ ] - ideally a single file but can be duplicated in several folders: wikipedia/ folder is synced on all mirrors but IL one (which only syncs a subset of it)
[X] - must not be included in Kiwix Catalog
[x] - Must not be included in opentracker
[x] - Must not be included in matomo stats

Currently testing with wikipedia/speedtest_en_blob_2024-05.zim

rgaudin commented 4 months ago

Opentracker is not a concern as it is based on Catalog

rgaudin commented 4 months ago

As discussed with @kelson42 we will exclude those downloads from the stats via a User-Agent exclusion rule.

I've added /speedtest/ to the global matomo configuration and we know that the log-import script has a list of UA patterns to exclude.

For this, I suggest we use the speedtester/robot string (maybe with a version number) as User-Agent as to fall into the script exclusion rule and not upload those log entries at all.

rgaudin commented 2 months ago

We now have:

Both are available on 13 mirrors (all but IL). Code uses speedtester/robot as U-A.