360: Accessing the list of Additional Checks

[stevieflow]

A number of users I have spoken to value the Additional Checks, but also want to see a list of these, independently of their implementation on a particular test.

Would we envisage a page similar to https://dataquality.threesixtygiving.org/common_errors - where we can list and descriibe the checks?

[robredpath]

We can...but what will users do with that information? Some of the tests are easy to replicate in Excel or give good pointers for things to look for by hand (eg missing fields), but some are harder (eg checking GB-CHC org IDs match UK charity number formatting)

Under the surface, is this pointing towards some sort of implementation guidance? Maybe 20 top ways you can improve your data, rather than a listing of what the software does?

[robredpath]

Or, is this about knowing what the software checks for, and therefore what they're getting right by implication?

[stevieflow]

Or, is this about knowing what the software checks for, and therefore what they're getting right by implication?

Yes, I think it's about transparency of the tests. There's wider questions, I agree, in terms of content management and such, but it seems important to make these public. This may also give us the opportunity to use headlines when we use the check in the test, and link to more info (as we do with common errors)

As useful example is the tests that IATI run on their dashboard (cc/ @Bjwebb ): http://dashboard.iatistandard.org/forwardlooking.html

[caprenter]

I guess this is partly sparked or related to #575 as well.

I think the tests are coded up and reported on starting here: https://github.com/OpenDataServices/cove/blob/master/cove_360/lib/threesixtygiving.py#L158

If you're not a coder or able to poke around the repo guessing which file they might be in, I can see how they are not easy to find.

[stevieflow]

The ask here is a page - similar to https://dataquality.threesixtygiving.org/common_errors - but listing the Additional Checks we may take.

How best to construct that page @robredpath ?

[robredpath]

The reason I'm ever so slightly reticent about this is that unless we're very disciplined, it'll go out of sync the first time we update Additional Checks without remembering that the page exists, or the first time we think it's a good idea to add additional guidance to the page that doesn't represent an additional check - it's a thin line between a list of the additional checks, and a helpful guide to things you can check about your data outside of CoVE. The latter would be a very helpful resource, but a bit different from what we're talking about here.

I wonder if the answer is to build for that from the off, and provide a page of guidance that includes an autogenerated part that draws from the comments in the code? Maybe? @Bjwebb , this feels like something you'd have ideas about.

[Bjwebb]

I wonder if the answer is to build for that from the off, and provide a page of guidance that includes an autogenerated part that draws from the comments in the code? Maybe? @Bjwebb , this feels like something you'd have ideas about.

Autogenerated is good. I'm wondering to what extent we can reuse the heading and message text we've already got, to pull together a list.

[stevieflow]

Yes to autogeneration

[KDuerden]

I'm a user that would like to know what the full range of Additional Checks are - to help my understanding when guiding people on publishing and using CoVE. Whether this should or could become public at some point I'll leave to you.

[robredpath]

@Bjwebb I think we can reuse the headings and the message text, but we'll need to write a human-readable description of the algorithm. It would make sense for that to live with the code so it's obvious that it needs to be updated whenever the algorithm is.

Shall we get the comments written so that we can extract them for @KDuerden easily, and look at autogenerating for the public website in the future?

[robredpath]

And by 'we', I mean - I'm happy to do it when I get back from holiday, but if we want it sooner than then, someone else will need to do it!

[robredpath]

Just had a chat with @Bjwebb about this - I'd misunderstood what was being asked for.

The work in this issue is to create a page that displays the title and text of all the additional checks, possibly with some additional information. This will be autogenerated, which will require a bit of refactoring, but shouldn't be too hard. I've scheduled some time in the next couple of months to get to this and a couple of similar issues.

Separately, I'll pull all of the existing text copy out into a Google Doc and supply it to @KDuerden and @ekoner with the proviso that it won't necessarily get updated if/when we change the code.

[stevieflow]

Great, thanks @robredpath

[Bjwebb]

An autogenerated list of checks is now on the dev site: http://release-201709.cove-360-dev.default.threesixtygiving.uk0.bigv.io/additional_checks

[robredpath]

This is now live at https://dataquality.threesixtygiving.org/additional_checks

[KDuerden]

And this will update, when the additional checks text is updated in due course?

[robredpath]

@KDuerden yes, that page is generated from the inline documentation of the code itself, so when we change the code and update the comments in the code, that page will change.