search

EventideSystems / tool_for_systemic_change

GNU Affero General Public License v3.0
6 stars 0 forks source link

Status and Activity Reporting review #809

Closed emily-humphreys closed 1 year ago

emily-humphreys commented 2 years ago

Working through Status change and Activity Reporting scenarios here https://docs.google.com/spreadsheets/d/1uihhiervqGDYb6F66ufrLpxLwzxManRNDr_QUDKcO5o/edit?usp=sharing

ferrisoxide commented 2 years ago

DEV NOTE

@emily-humphreys @SishaMish

I'm looking at using this ticket to coordinate the changes we talked about last Wednesday. I've jotted down the direction I think we need to take, based on my notes and thinking since then.

NB This aims to addresses issues raised in #808 and #685 as well.

This isn't set in stone, but I just wanted to make sure we're all on the same page before going too far into development. If we need to change things, let me know.

Proposed changes:

Stretch targets:

emily-humphreys commented 2 years ago

These look correct, however, just want to clarify you've also seen comment below:

Change filters on card "grid" to match the four possible statuses ("Gap" and "No Comment" have no meaning when each item has at least "More information" set

Gap will still be relevant as this will be all those that are not "Actual + Planned"

ferrisoxide commented 2 years ago

NOTES TO QA

@emily-humphreys @SishaMish

An initial cut of the new data model is up on staging. Notable changes:

See DEV NOTES below for a list of outstanding work.

For each scorecard there is a new tab "Changes" that should correspond to the data in the original "Activity" list.

If someone can please eyeball these two lists and make sure the comment / status changes line up for a sample of scorecards. NB Not all events are recorded in the "Changes" list, specifically:

will not be present in the "Changes" list. Also note, if you make any further changes to a scorecard comment or status it will not show up in "Activity", though it will appear in the "Changes". From here on in, "Changes" will be the source of truth for reporting, though right now I haven't updated the reports to use this. I will once we verify that the Changes log is correct.

You can test the new dialogs for updating comments / status. There's not much change, other than the system uses popover warnings to tell the user what fields are required.

QA required:

DEV NOTES

Outstanding development work:

emily-humphreys commented 2 years ago

Hi team,

I'll put some time into testing this tomorrow.

Cheers Emily

On Sun, Sep 11, 2022 at 5:59 PM Tom Tuddenham @.***> wrote:

NOTES TO QA

@emily-humphreys https://github.com/emily-humphreys @SishaMish https://github.com/SishaMish

An initial cut of the new data model is up on staging. Notable changes:

  • Speed of rendering the grid appears to have improved
  • "Checked" status for checklist items has been replaced with "status" indicator
  • All checklist items without a status (i.e. not "planned", "actual", "suggestion", etc) are marked as having "no comment"

See DEV NOTES below for a list of outstanding work.

For each scorecard there is a new tab "Changes" that should correspond to the data in the original "Activity" list.

If someone can please eyeball these two lists and make sure the comment / status changes line up for a sample of scorecards. NB Not all events are recorded in the "Changes" list, specifically:

  • Transition Card Created
  • Updated Checked

will not be present in the "Changes" list. Also note, if you make any further changes to a scorecard comment or status it will not show up in "Activity", though it will appear in the "Changes". From here on in, "Changes" will be the source of truth for reporting, though right now I haven't updated the reports to use this. I will once we verify that the Changes log is correct.

You can test the new dialogs for updating comments / status. There's not much change, other than the system uses popover warnings to tell the user what fields are required.

QA required:

  • Verify that Activity and Changes lists line up, with respect to comment / status changes
  • Verify that new Edit Comments dialog adds appropriate entries in the Changes list for a card Recommend using a new TransitionCard or SDGsCard for this
  • Verify that status filter is working for Transition Cards.

NB Every item without a comment will be now presented using the "no comment" filter. Every item not 'planned' or 'actual' (including 'no comment' items will be greyed out when using the "gap in effort" filter There is a problem rendering SDGs filters (see DEV NOTES), so ignore these for now.

DEV NOTES

Outstanding development work:

  • Modify reports to use new data model
  • Re-enable filters on grid for "snapshot" view of grid at a point in time
  • Re-enable subsystem tags on grid.

— Reply to this email directly, view it on GitHub https://github.com/ferrisoxide/wicked_software/issues/809#issuecomment-1242916363, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFVZORCQTM4WJ5OF4FXLC3V5WJ7BANCNFSM57KIPLFA . You are receiving this because you were mentioned.Message ID: @.***>

ferrisoxide commented 2 years ago

Fix display of "no comment" cells on SDGs Card grid

Fix in on staging

ferrisoxide commented 2 years ago

Re-enable filters on grid for "snapshot" view of grid at a point in time

@emily-humphreys @SishaMish Fix is on staging

ferrisoxide commented 2 years ago

Re-enable subsystem tags on grid.

@emily-humphreys @SishaMish Fix is on staging

ferrisoxide commented 2 years ago

Modify reports to use new data model

The first cut of the new Activity Report is up on staging. Under the "Reports" tab there should be a report temporarily labelled "Activity Report (NEW)".

The report is similar to the existing "Activity Report" report, and the existing report can be used to roughly verify the new report, but the actual data is a combination of the specification in the linked Google docs above and the definition per the Reports page of the wiki.

Initiative addition / removal data has not changed been. The sources of characteristics data are as follows:

Column Data
Characteristics beginning of period The count, per charactertistic, of all check list items marked 'actual' before the beginning of the period.
Additions The count, per charactertistic, of all check list items marked 'addition' during the period (per the Google Docs spreadsheet).
Removals The concept of "removal" isn't fully specified yet. This will always be 0 for now.
Characteristics end of period Characteristics beginning of period + Additions (- Removals)
New Comments Saved assigned Actuals The count, per charactertistic, of all check list items marked 'New Comments Saved assigned Actuals' during the period (per the Google Docs spreadsheet).

@emily-humphreys Can you verify that I have the sources for the characteristics data correct?

@emily-humphreys @SishaMish The new Activity Report is up on staging and ready for QA.

SishaMish commented 2 years ago

@ferrisoxide @emily-humphreys Started QA

Case 1. Using 'Client Account' on staging Transition Card - 'Damage Control' Comparing events on Activity tab and Changes tab

Problems encountered:

  1. On the Changes tab, activity described as 'New comments saved assigned actuals' all appear to state that they have changed from 'Actual' to 'Actual', whereas on the Activity tab they state they are changing from 'NA' to 'Actual'. This appears to be caused by the check box being checked and unchecked historically. Example: Changes tab screen shot Screen Shot 2022-09-14 at 6 00 43 pm Activity tab screen shots showing the entry that is different, and the earlier incident where the status was changed Screen Shot 2022-09-14 at 5 59 19 pm Screen Shot 2022-09-14 at 6 00 19 pm

  2. There is a major gap in the data available on the new Changes tab. The data appears to go back to 15/2/2021 on the Changes tab where it stops showing any changes to comments, while on the Activity tab there is more data showing the dates 11/2/2021 and extending back to 27/10/2020. I have also checked these date ranges on another Transition Card (Greening Marion), and the same issue appears there.

ferrisoxide commented 2 years ago

Problems encountered:

@emily-humphreys @SishaMish

Fixes are up on staging and ready for QA

SishaMish commented 2 years ago

@ferrisoxide @emily-humphreys

QA'd the issue regarding loss of data. The data is now all present. So this has passed.

QA'd the Reports, comparing the original Activity Report and Comment Report with the new versions.

Steps taken: Using the Client Account Producing an Activity Report Transition Card: Damage Control Date Range: 01/01/2021 to 15/09/2022

The totals at the top of this report match on the old and new versions, but there are some differences in the columns. I've highlighted the differences in the following screen shots. Original Report: Screen Shot 2022-09-15 at 5 43 53 pm New Report: Screen Shot 2022-09-15 at 5 44 09 pm

Steps taken: Using the Client Account Producing a Comment Report Transition Card: Damage Control Date : 15/09/2022

  1. The New Comment Report has added SDGs Targets, which shouldn't be there.

  2. The totals almost match up between these reports, except for one. Highlighted difference shown in the following screenshots. Original report: Screen Shot 2022-09-15 at 5 49 13 pm New report: Screen Shot 2022-09-15 at 5 49 24 pm

  3. Also noticed in these screenshots that there is an incorrect date showing on the Original version of the Comment report. The correct date is on the New report.

ferrisoxide commented 2 years ago

@emily-humphreys @SishaMish

See notes below:

Producing an Activity Report

The differences between reports are a bit hard to compare, as there are some major changes in the way the data is now handled. Checking and unchecking checklist items threw the orignal report out.

In the new report, tallies for activities are only counted once per characteristic per initiative within the period. For instance, if a characteristic has a new comment added and changes to "actual" mutliple times within the period it will only be counted as "New Comments Saved assigned Actuals" once. I'm not sure if this is what we actually want.

I think the only way we can really check the reports is to manually verify the report against the data as presented in the Changes history, e.g. tally up the "actual" states of checklist items immediately before the reporting period and comparing it against the report, and then doing the same for changes that occur within the reporting period.

If it will make manually verifying the reports easier, I could make it possible to export the changes as .csv files

Producing a Comment Report

  1. The New Comment Report has added SDGs Targets, which shouldn't be there.

This should be fixed now

  1. The totals almost match up between these reports, except for one. Highlighted difference shown in the following screenshots.

There is a bit of "wobble" with dates. The original report used end of day in UTC time, whereas the new reports use Adelaide s time zone. Because of this the dates don't line up. For instance, running the report now - hours after Lisa's run - produces identical reports.

Going forward, reports will be using the user's time zone (defaults to Adelaide. but can be set via the user's Profile). NB the new comments report includes the time zone for each time stamped comment to remove ambiguity.

  1. Also noticed in these screenshots that there is an incorrect date showing on the Original version of the Comment report. The correct date is on the New report.

This should be fixed now

emily-humphreys commented 2 years ago

Let me know when first round testing is complete and its good for me to hop in.

Thanks for this

Cheers Emily

On Thu, Sep 15, 2022 at 10:22 PM Tom Tuddenham @.***> wrote:

Producing a Comment Report

  1. The New Comment Report has added SDGs Targets, which shouldn't be there.

This should be fixed now

  1. The totals almost match up between these reports, except for one. Highlighted difference shown in the following screenshots.

There is a bit of "wobble" with dates. The original report used end of day in UTC time, whereas the new reports use Adelaide s time zone. Because of this the dates don't line up. For instance, running the report now - hours after Lisa's run - produces identical reports.

Going forward, reports will be using the user's time zone (defaults to Adelaide. but can be set via the user's Profile). NB the new comments report includes the time zone for each time stamped comment to reduce ambuguity.

  1. Also noticed in these screenshots that there is an incorrect date showing on the Original version of the Comment report. The correct date is on the New report.

This should be fixed now

— Reply to this email directly, view it on GitHub https://github.com/ferrisoxide/wicked_software/issues/809#issuecomment-1248060066, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFVZOSFUY5D2DNSMIK35G3V6MLYBANCNFSM57KIPLFA . You are receiving this because you were mentioned.Message ID: @.***>

ferrisoxide commented 2 years ago

@SishaMish @emily-humphreys

Just to summarise my notes above, I'm reasonably confident we have the reports, etc in place.

The only thing I'm unsure about is how we aggregate events - e.g. if an initiative / characteristics (aka checklist item) has a new comment "saved actuals" occurring multiple times within the reporting period, it is only recorded once. I think this makes sense, otherwise people toggling the state (as they did with checked/unchecked) would skew reports.

Does that make sense?

SishaMish commented 2 years ago

@ferrisoxide @emily-humphreys

I've done some delving in to the Activity report and the totals. Using the Damage Control transition card, and comparing the totals from the New Activity report and the activity from the Activity tab between two reporting times (first 1/12/20-31/12/20, and the second report 1/1/21-31/3/21), all totals of Additions and New Comments Saved assigned Actuals on the New report are correct for that reporting period.

I'm happy to pass this. I'll hand it over to you @emily-humphreys to take a look.

ferrisoxide commented 2 years ago

@emily-humphreys This all brooks a question around importing comments. The check/uncheck mechanism won't work anymore obviously, but a naive import probably isn't much use either.

This is out of scope for the current work, but we'll have to come back to how you want to import comments going forward - e.g. how do we manage "state", are imports treated as "new comments", etc? Is it even a feature you want to maintain? Stuff to discuss.

ferrisoxide commented 2 years ago

@emily-humphreys @SishaMish Complete aside, but I've just made some changes that has improved the rendering speed of the grid (for both card types) by an order of magnitude. You might want to check it out. :)

emily-humphreys commented 2 years ago

Guys - I'm on testing this today. Apologies for delay - need to come to this with a clean plate. Re import - yes definitely use this and will think through what we might do next and write up a ticket for it

On Tue, Sep 20, 2022 at 7:11 PM Tom Tuddenham @.***> wrote:

@emily-humphreys https://github.com/emily-humphreys @SishaMish https://github.com/SishaMish Complete aside, but I've just made some changes that has improved the rendering speed of the grid (for both card types) by an order of magnitude. You might want to check it out. :)

— Reply to this email directly, view it on GitHub https://github.com/ferrisoxide/wicked_software/issues/809#issuecomment-1252101583, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFVZOSCTRJXRVOYXDHLFHTV7GBFXANCNFSM57KIPLFA . You are receiving this because you were mentioned.Message ID: @.***>

emily-humphreys commented 2 years ago

I've put some notes together re the history of all your comments above - it was looking super messy as I replied to each so I thought I could consolidate in one doc. Apologies for the big dump - I know I've let you down on being responsive on this. Testing notes 21.9.22.docx

ferrisoxide commented 2 years ago

NOTES TO QA

@emily-humphreys @SishaMish

Re "Testing notes", see comments inline below.

  • Turn off ability for users to check/uncheck checklist items

Now that I see this – I think we need to bring back a tick as the icon (rather than the dot). This is so it aligns to existing user guides and videos and doesn’t leave a gap

Can we have a greyed-out box, that when comment box opens up and status saved, a tick appears in the colour for Planned or Actual (like currently) and left blank in all other situations. Apologies I need to see this to understand

Is this to replace the status icon currently in play, or a new element on the page?

  • Display current comment status as an icon beside each checklist item (e.g. [P] for planned, [A] for actual, etc)
    • Icon will include tooltip with literal text "Planned", "Actual", etc.
    • Icon will change when user updates a comment status to a new value
    • Icon will carry relevant colour value per status (blue for "Actual", green for "Planned", etc) This doesn’t seem to be present

The only thing that isn't there is the [P], [A], etc character. The icon appears as a coloured rectangle (or no colour for "No Comment" items). The tool tip appears when you hover over the item.

image

I removed the single character because it made the UI look noisy, as every item had a letter next to it. It was particular problematic for "no comment" checklist items as this became the base state. Peppering the [N] icon - with no color scheme associated to it - looked odd and busy.

  • Insert "More information" comment status for each checklist item without a comment
    • for items with no comment, but have been checked, insert the "More information" status as occurring at the time of checking To clarify this is only for conversion of data – not a wholesale rule. Characteristics not commented on are status free

"status free" is itself a status - call it a "null status", "pending" or "not started", but it's an explicit state of the checklist item. We need to be clear that we define the state that each item can be in, in order to manage the transitions. I'd suggest for clarity's sake we give this state a name.

What happened here is that every checklist item that had been checked, but no comment associated, was given the status of "More Information". Every other checklist item that had not been checked was assigned the base status of "No Comment". You seem to be suggesting that there is another status "Status Free" that means the same thing as "No Comment".

  • All checklist items without a status (i.e. not "planned", "actual", "suggestion", etc) are marked as having "no comment"

A Characteristic without a comment is just characteristics that yet to be commented and are ‘status free’.

On a Transition Card filter they would show as ‘gap in effort’. It is only for Characteristics with a “ticked + no comment’ status that we are replacing with “more information” in the conversion over to the new data – it is not a wholesale rule. i.e your notes ‘for items with no comment, but have been checked, insert the "More information" status as occurring at the time of checking For the ‘No comment’ filter on a Transition Card – this is now obsolete – so we can remove it

I agree that the "No Comment" filter is probably redundant. Right now anything that is in the status of "No Comment", "More Information" or "Suggestion" should be marked as a dark grey box when you apply the "Gap In Effort" filter. I'm happy to remove the "No Comment" filter.

  • Change filters on card "grid" to match the four possible statuses ("Gap" and "No Comment" have no meaning when each item has at least "More information" set This is not correct. There are Characteristics that are not commented on, i,e status free.

Filters will be Actual, Planned and Gap in Effort (= - Actual – Planned)

I realised this after the fact and left the filters as they were. The filters are Actual, Planned, Gap in Effort and "No Comment", with the last to be removed per the comment above.

  • Update reports, ensure Activity Report matches the expectations set out in the attached spreadsheet Will wait to test these

k++

Others items

  1. Changes and Activity Tab Did you see Activity and Changes being kept or is this just for testing? I’d like to integrate Activity and Changes as one tab – This seems like duplication of information.

The "Activity" tab will go away - it was only there to verify that the new data matches the old. The "Changes" tab will stay and will be the source of truth for activities. I'm happy to change the title back to "Activity" once the old one disappears.

  1. The sources of characteristics data are as follows: These are now all correct

Make me think we remove ‘Removal’ column and add a Total column – out of scope here so will create new ticket The only thing I'm unsure about is how we aggregate events - e.g. if an initiative / characteristics (aka checklist item) has a new comment "saved actuals" occurring multiple times within the reporting period, it is only recorded once. I think this makes sense, otherwise people toggling the state (as they did with checked/unchecked) would skew reports. This would happen for existing comments but not for Save New Comments. All new Actuals would be counted.

So, if I understand things correctly, if a user changes an item's state without creating a new comment - i.e. "Updating Existing" from planned to actual, back to planned and back to actual again - within the same reporting period - this will count as two Additions?

This all brooks a question around importing comments. The check/uncheck mechanism won't work anymore obviously, but a naive import probably isn't much use either. All imports would be a forced ‘New Comment’ + Actual (first comment) or ‘Save New Comment’ + actual’ status if a comment exists

Understood. So every imported comment is assumed to be an "Actual" new comment, changing the state of the checklist item.

We can give this instruction on the Import screen

On that, the link to the help document doesn't appear to work any more.

lets create a ticket and do this before we release new data model

k++

ferrisoxide commented 2 years ago

@emily-humphreys @SishaMish

Per notes from meeting last week (21/9/22):

Everything except "Write up activity report structure / assumptions in the wiki" has been done and is now on staging and ready for QA.

ferrisoxide commented 2 years ago

NOTES TO QA

@emily-humphreys @SishaMish

The "import comments" feature has been updated to use the new activity model. It will import all comments as new comments with the status of "actual". It will apply the import activity as "New comments saved assigned actuals" in the case of a previous "actual" comment, and "addition" for all others.

I'm going to draw a line here vis-a-vis the new functionality. I think the system is now feature-complete, in terms of the new comment model. Obviously, we still to look at issues relating to importing SDGs comment and the like, but these can be done as separate tickets. I won't be adding any more functionality at this point, instead I'll focus solely on fixing any bugs that come up through QA.

SishaMish commented 2 years ago

@ferrisoxide @emily-humphreys

I've started QAing the Activity Report.

Using the account: Test for #681 Transition Card: Food Security in the City of Green Valley Initiative: Moving Feast Year 1 Report

The problem I am encountering is when I edit a characteristic and change the state from Actual to Planned, the Activity Report shows the same total in the 'Characteristics beginning of period' and 'Characteristics end of period'. I would expect to see the end of period column to drop down by one.

Screenshot 1: The changes made shown in the recent Activity log. The circled lines are the activity not showing up in the totals of the report. Screen Shot 2022-09-28 at 4 07 31 pm Screenshot 2: The Activity Report with the unchanged totals.

Screen Shot 2022-09-28 at 4 10 28 pm

Now that I have been QAing this, I'm not sure I understand why the Removals column needed removing. Isn't changing a characteristic from an Actual status to any other status essentially removing it from the total number of characteristics on the report? I apologise if I'm skipping backwards here - I am just trying to understand what information the user is getting from the report.

The rest of the activity seems to be showing up as expected.

ferrisoxide commented 2 years ago

@emily-humphreys @SishaMish

The problem I am encountering is when I edit a characteristic and change the state from Actual to Planned, the Activity Report shows the same total in the 'Characteristics beginning of period' and 'Characteristics end of period'. I would expect to see the end of period column to drop down by one.

I've fixed the tallies so they should now show the number of initiative/characteristics with the status of 'actual' at the start of the reporting period (Characteristics at beginning of period) and those with 'actual' at the end of the reporting period (Characteristics at end of period).

The "Additions" column may have more than the difference between these two values, as a user may add a new comment and change the status to 'actual' more than once for the same initiative/characteristic in the same period (e.g. planned->actual->back to planned->back to actual, with a new comment each time, records two 'additions').

SishaMish commented 2 years ago

@ferrisoxide @emily-humphreys

I've QA'd this and the totals are now correct. I'm happy to pass this.

ferrisoxide commented 2 years ago

@SishaMish Here's a file to test importing comments with. You'll need to change the name of the scorecard/initiatives as appropriate:

import_test.zip

All comments will be imported as "actual". If this changes the state of an initiative/characteristic it will record in the history as an addition. If the initiative/characteristic is already 'actual' it will mark the activity as 'new comment saved assigned actual'.

ferrisoxide commented 2 years ago

@emily-humphreys @SishaMish

Per the wiki:

Characteristics end of period The tally (Characteristics beginning of period + Additions - Removals) for each Initiative Characteristic.

Emily, do you want this number to the be the tally of additions added to the count of Initiative Characteristics marked 'actual' at the start of the period?

Or, the number of Initiative Characteristics marked 'actual' at the end of the period?

I've changed the report to present the latter, as this made more sense when Lisa and I were discussing it, but it's at odds with the original definition of the report. I can easily change it.

ferrisoxide commented 2 years ago

@SishaMish @emily-humphreys

Per discussions last week, the "Characteristics end of period" total has now been changed to count all additions in the reporting period and adding that to the count of Initiative Characteristics marked 'actual' at the start of the period?

As far as I understand, this is the last thing that needs to be completed.

emily-humphreys commented 2 years ago

@SishaMish Here's a file to test importing comments with. You'll need to change the name of the scorecard/initiatives as appropriate:

import_test.zip

All comments will be imported as "actual". If this changes the state of an initiative/characteristic it will record in history as an addition. If the initiative/characteristic is already 'actual' it will mark the activity as 'new comment saved assigned actual'.

@SishaMish have you tested the imports of comments against reports yet?

SishaMish commented 1 year ago

@ferrisoxide @emily-humphreys Issue running a smoke test on the app. Using a new account - Test for #809 Created a new Transition card Adding comments under Initiative Characteristics When adding a new comment and clicking the Save New Comment button, the app appears to attempt to save (greying out the pointer), but then leaves the comment as unsaved with the text box still open.

SishaMish commented 1 year ago

@ferrisoxide @emily-humphreys Testing the same functionality on SDGs Alignment cards, the same issue occurs with saving a new comment.

ferrisoxide commented 1 year ago

Testing the same functionality on SDGs Alignment cards, the same issue occurs with saving a new comment.

@emily-humphreys @SishaMish Issue has been fixed and deployed to production.

ferrisoxide commented 1 year ago

Deployed to production. Closing