We need to add the lint-lists script, that is run prior to merging any PR to citizenlab/test-lists, checks to ensure we don't end up with inconsistent category codes in the lists.
The checks we need to implement are the following:
The shortest representation of a URL is in the global list. If it's inside of the country list, we should prompt the user to migrate it there
In the global list we don't have multiple variations (i.e. www.example.com and example.com) with different category codes of the same domain name
We need to add the lint-lists script, that is run prior to merging any PR to citizenlab/test-lists, checks to ensure we don't end up with inconsistent category codes in the lists.
The checks we need to implement are the following:
Some research into the prevalence of this problem is documented here: https://gist.github.com/hellais/fab319ae20b0ccca7b548a060ed66e14.
The steps to do this are 2:
This came up as part of: https://github.com/ooni/api/pull/300