Editing interface for Pantheon 'admin'

[DavidRoy]

A controlled interface to allow changes to Pantheon data, including the Conservation Status information.

Limited to a defined editor pool drawn from the community, with necessary permissions

[Davydaveth]

A few observations on this. If it were to include conservation status revisions, then it ideally ought to trap that and send off the change to the UKSI folk. Otherwise it promotes drift between the 2 databases. We ought to be able to alter the resources against which a taxon is coded- a recent challenge was that a spider's range included much more than the upland against which it was coded. We need a decision on the level in the resource tree that can be edited without breaking the database. I assume it runs an editing audit of who made changes and when, but might it be useful for it to capture the intended source of those revisions? Perhaps a standard pick list of "user comment", "published review", "technical update", "DNA-confirmation", "taxonomic revision", etc?

[johnvanbreda]

I've done the non-contentious part of this. You should find a new "Edit species data" item in the menu (only available to people given the species editor role so no one else can see this). This shows a grid with a list of all UKSI species (filterable to only Pantheon data by entering Y in the cell at the top of the "In Pantheon" column then pressing tab. The edit button gives you a page where you can edit all the attributes (traits) and associations, but not yet the conservation statuses.

[johnvanbreda]

I've now added the conservation statuses to this page. You can currently only select from existing statuses so we may need to consider a tool for controlling the list of status categories and status descriptions?

One more requirement on this page is that after editing the taxon the species index table should be updated.

[Davydaveth]

Yes, I think you are right. Whilst the IUCN and national rarity stuff will not change, I can see policy and legislative changes might come along, be that a legal revision of the S41 list, or another species policy listing or legal category, and Pantheon ought to be agile enough to capture that as soon as it comes along. Is this not just writing a new category to the main conservation status tables?

We need to prompt the user then if the index needs updating..something irritating and obvious :)

[Davydaveth]

OK, so I took the opportunity of correcting a clearly erroneous S41 ascription which has been commented on by a number of users, and the switch from one Bembidion beetle to the correct Bembidion beetle went ok, but it still shows the incorrect entries, i.e. has not updated. Does "save" then not equal "update"? If not, can we have an update button and some text to prompt the user? Sorry if I have jumped the gun here.....

[Davydaveth]

Following a useful discussion with Hannah and Martin at CEH, we have arrived at a workable editing interface, and suggest the current design be slightly modified. In consideration of the degree of change that the database fields will be subject to, the following design changes are required:

1. Deletion of an editing option from "rarity score" to "grazing marsh- salinity" inclusive.
2. Reduction of page space left to increase screen real estate.
3. Creation of a free text change wiki at the page bottom, of length of around 500 characters, to formally track the changes made to the fields left above. This to be supported by Pantheon page example to guide editors.

If possible, change "Leave site" message to "Leave page", since the former implies leaving Pantheon, not the currrent page. We did discuss whether a "resource" change could automatically align the family hierarchy above, or whether users need to start at the top and change down the list items- from "broad biotope down". We assume Indicia will log both the user and the time stamp of the change. A super user will half yearly gather up the change wikis and either approve or query, or potentially reject, the changes made. Being able to identify new change wikis and their creation date thus seems important. Comments on the feasibility of this would be welcome.

[johnvanbreda]

@Davydaveth some progress for you:

1. Deletion of controls from the edit form from rarity score to grazing marsh - salinity now done.
2. The page size reduced accordingly. I also made it use the monitor width more effectively if on a large monitor.
3. Added a change log input at the bottom.

I can't change the Leave Site message since this is browser generated - in fact on Safari it says "Leave Page" already.

Still to do:

1. Do we need some sort of report or viewer for the change log data? If so, where? I think this ties in to your last comment. The change log entries are tagged with a date/time and user so we can list the entries for review as you suggest.
2. You are correct in the comment above - when you save a trait change using this page, although the data are updated, there is a snapshot table containing the species index data which is specifically built for Pantheon and this table still needs to be updated. So I have a task to trigger an update when you save any changes.

I'm not sure about the best approach for resource changes aligning the hierarchical parents. It's obviously more complex in the code if we do this, but maybe it's easier for you? I guess whatever process gets fired to update the species index could also do this.

[Davydaveth]

Great stuff John. Our meeting last week ended up with the notion of a sort of super-user who would bulk audit all the changes made since the last update, assessing the completion wiki entries, and "signing them off". There was some notion of periodicity to this, possibly tied into updates to the species status updates. It sounds like tying the super-user to the snapshot table already sort of does this then, in that they could authorise/ execute the update of the snapshot table change block for that period. That super-user would need to have coraled all the change wikis in one place and be able to read through them, challenge each editor if need be, reject a change, or accept the block (most likely scenario we felt). If the change log wiki could tot up the entries it currently holds and display them somewhere (a sort of notification idea, if easy), then if lots were pending then it might be timely to go early rather than be strictly bound to only doing this exercise on, say, 1st June each year. This is all a bit design on the hoof, so please bare with us!

We can make editors start from the top of the hierarchy and edit down the line if it makes the world easier....

[Davydaveth]

Hi John. I attach Hannah's notes of that meeting, highlighting the editor interface discussions we had: • What are the audit trails for editing a species? The changes need a who, what, why and when. • Do we archive old values? The cost could become large over time for storing archived data. Have only one record, which can change when edited but a textbox is provided so the person editing the record can leave a note as to why the changes have been made. • Changes need to be checked by one super editor – this is only to check that a good reason has been given for changing the record. The super editor only checks once every set period of time (e.g. 6 months). A back-up is taken before the super editor approves changes. When the changes are approved a new version of Pantheon then exists. • Update on the main page of Pantheon to show the changes made on the latest version. • Editor permissions: how big is the pool of editors? Do all editors have the same permissions? As a start, the steering group members are editors and super editors, e.g. if the steering group meets twice a year, then the changes can be reviewed and approved at the meeting. Ad hoc editors can be added as needed, e.g. if someone with expertise within a particular taxon group who like to make multiple changes. Users can contact Pantheon asking to be an editor and/or the steering group could invite people who they think would be good to have as an editor. Ad hoc editors may only have edit permissions for a limited period of time. • Habitat indices should not be edited as these are already published indices. Errors are likely to be transcription errors and we can ask John vB to make the changes directly. If a new species is added to the indices or a species changes this should be a completely new record and considered a revised or new indices. • Are the dropdowns for the habitat and resources hierarchy filtered by the parent choice? Does selecting the resource from the dropdown automatically add the parent habitat and biotope? • Inflorescence-associated is contained with the plant-associated so is it needed separately? Each element within both seem to be duplicated. • Large scale changes, e.g. where research has been done on a whole load of traits, will need to be dealt with in a different way.

[johnvanbreda]

@Davydaveth I've implemented the following:

• From the Admin page, you can now access a Species Change log report
• From the Admin page, you can now access a page with a tool that allows you to perform a full species index update.

Presumably the task of performing the index update needs a different set of permissions so that only a "super editor" can do it?

Noting Hannah's question about archiving old values, note that the log is simply a trail of comments attached to each species, not a full audit of the data changes. We do have the option of enabling full auditing, but I'd be cautious due to data volumes.

[Davydaveth]

can the super-user just be defined by their name/ user id? You could then extend the super-user pool by adding folk into it. Can the species change-log have a free text section at its start? Rather than noting the change, the key seems to be the rationale behind it, so that needs simply explaining at the start. Martin also rightly questioned over-doing the audit process, which is the right way to go.

[johnvanbreda]

The super-editor can be defined by a Drupal role, so you can easily add and remove people in future.

The species change log is actually just a free text box that gets recorded with a date stamp and user ID. So when you post a change, you are free to enter the rationale for the change. I've added a note to the edit species page to explain that you want the editor to describe the rationale behind the change, not just the change.

[Davydaveth]

that sounds sensible. I cannot see there will be a high turnover or ever a large number of super-editors, so i'm not sure we need an in-Pantheon mechanism to assign them. It seems a bit over the top. Thanks for setting the audit trail free text system up. I will inquire of Martin and Hannah how to progress with the super-editor role and will get back with the list soon.

[Davydaveth]

Typically, they are both on leave! However, a question arises. If the editor pool make changes to the dataset, and record those changes via the audit sheet, explaining the rationale, how are all those changes gathered together for the super-editor to sign off. Can the super editor pick and choose from the list? For example, there may be 10 changes, with 9 being OK, but the 10th requiring more debate. Rather than delaying upload of the 9, can the 10th be held back? The executed changes need to be removed from the pool or ticked as completed or similar.

[johnvanbreda]

At the moment, it's an all or nothing process. Basically, changes to the attributes are made directly in the top copy of the dataset. There is a species index table used to speed up many of the reports and it is only the update to that table which needs to be triggered at some point. This is currently done by creating a completely new species index table then renaming the old one, renaming the new one and deleting the old one (so there is no downtime).

We are moving in the direction of having a mechanism for keeping changes unpublished, but doing that on the live data as we don't have a means of tracking unpublished vs accepted versions of each piece of data in the system. It's only the fact that the species index isn't automatically rebuilt for each change that gives any sort of "publishing control". This is an issue because some reports won't use the species index table, so they would be updated immediately anyway. In other words, we are re-purposing a part of the system for something it wasn't designed for so it won't work entirely as expected.

Given that this aspect of the requirements is evolving, I think it's worth pausing and working out exactly how we want it to work (bearing in mind that a system for moderating changes like this might be complex to develop). We can then look into whether the current approach is the right way to do it.

[Davydaveth]

Thanks John. It may then just be easier to pause the exploration of the 10th entry, sort that out, and then publish all 10 in one go, side-stepping that differential approach. I think the overview and final authorisation from the super-editors is the way to go, as it otherwise opens it up to all sort of rogue actions. But their publication of the submitted changes ought to be simple. But, as you note, lets pause and consider the benefits and dis-benefits of it.