MoreCast: Validate Development Team Addressed User Needs

bcgov / wps

Wildfire Predictive Services to support decision making in prevention, preparedness, response and recovery

Apache License 2.0

41 stars 9 forks source link

MoreCast: Validate Development Team Addressed User Needs #3409

Open rajpersram opened 9 months ago

rajpersram commented 9 months ago

User Story: As a Forecaster, I want to ensure that the development team has adequately addressed our identified needs So that our tools and systems effectively support our work in forecasting and preventing wildfires.

Acceptance Criteria:

[x] Forecaster feedback is gathered within two weeks of deployment to development server.
[x] Feedback addresses Forecaster's previously identified priorities from Journey Map.
[x] WPS team digests feedback within a week and proposes any necessary adjustments the following week.
[x] Journey Map is updated to reflect the prioritized items have been implemented to meet Forecaster's needs.

rajpersram commented 7 months ago

A System Usability Scale (SUS) survey has been sent to Matthew MacDonald and his team, with a deadline of April 30th to get the results in. We'll need at least five entries for the feedback to be statistically significant.

rajpersram commented 7 months ago

System Usability Scale survey results are in from Matt's team, and the report says... that the workflow and user needs are below average :( and only marginally acceptable by definition.

Now that we know that users are either confused about the UI or that it's not providing what they need, I'll reach out to Mat to learn more details. Since changes are needed one way or another to provide a better user experience, perhaps this can dovetail into the Simple Ensemble features and Skill Scoring to make MoreCast more practical.

rajpersram commented 6 months ago

Matt has been contacted to gather feedback from his forecasters on how we can improve the UX workflow and the UI.

rajpersram commented 6 months ago

Follow up from Matt (May 13, 2024) regarding the System Usability Scale survey:

"The main issue with MoreCast right now is that it doesn’t work. The forecasters are getting frustrated with missing models and their tokens expiring which kicks them out. I tried using it last night at 7pm and only the NAM and GFS models were available…

So if we were to improve anything, it would be the tool’s stability and functioning.

A high glance, quick way of assessing which model has been performing the best would also be good.

Happy to discuss more later this week."