Epic: Content validation messages are not always clear to operators

[fiver-watson]

Describe the bug

Sprint 2 Talk-aloud testing found a number of issues when performing usability tests related to packages that fail validation. Problems observed include:

• Users did not know where to find their package
• Users stopped reading after encountering long absolute paths
• Error messages were often too vague for users to understand what was required to resolve the issue

After the talk-aloud testing, DG prepared an initial proposal based on the feedback received on how to revise the content validation error messages to make them more useful to operators. Two different groups at SFA reviewed these proposals and provided additional feedback. DG has now incorporated some of this additional feedback into a revised proposal for improving the error messages.

Proposed solution

See the attached PDF. A copy of this can also be found in the internal team Drive, here

---

UPDATE 2024-10-02

We got some additional feedback during the SFA onsite workshops, specifically about the proposed "Validate Metadata" error messages. See the notes here. Quick summary:

• Providing a line number in the metadata.xml is not helpful
• They will be using PackageHandler to fix things, not opening the file directly
• As such, providing info on the actual XML element that was a problem is more useful
• Radda looked at the XSD output and we should be able to get the specific element
• If getting the element is not possible THEN we can fall back to the line number
  • In such a case, PackageHandler probably won't be able to parse either, so line number is helpful for manual inspection

Also: while the proposal mockups include hyperlinks to either the failed packages directly, or the related failed minIO bucket, we noted during the review that this was a nice-to-have, but not a requirement if too complex.

---

SFA SIP validation error message change proposals REVISED V2 (2024-08-14).pdf

[fiver-watson]

Biggest single advancement on this issue would be to add the ability to format the error message outputs, so we can add linebreaks, emphasis / bolding, bullets, and more to help with legibility.

Second advancement would be to use relative paths starting from the package when reporting errors, rather than absolute paths.

[fiver-watson]

Specific mention of improving the format validation failure message from SFA - Trello card here:

• https://trello.com/c/5v2ChUeT/183-error-format-validation

[aseles13]

@fiver-watson Have we received any feedback on these messages?

[fiver-watson]

@aseles13 this issue was created after I incorporated the 2 separate rounds of feedback about error msgs we received into 1 proposal doc, mentioned and linked in the issue ticket description. To avoid endless back and forth (since some of the initial 2 rounds of feeback was already contradictory), we decided to just go ahead and implement something to get feedback, rather than keep sending drafts back and forth. We have not yet implemented anything to show to SFA. So: the attached PDF and the original GDoc copy in the internal team Drive, here, still represent the most up to date proposals based on previous feedback from PoC#2.

[fiver-watson]

Update 2024-10-02:

We reviewed the proposed error message enhancements during the SFA onsite workshops (2024-09-21 to 27). In most cases they were happy with the proposals though there was some feedback specifically about the metadata validation messages. I've added a summary of that feedback directly to the issue description - see the full discussion notes here.

Additionally, note that providing a hyperlink to the related fail bucket (or the failed pkg directly) is a nice-to-have, but not necessary if difficult.

[sallain]

Just a note for whomever picks up this card - the work that @fiver-watson has outlined is pretty extensive, so I think we can break this down into smaller increments that we work on over the next several months, not just in October. Here are cards for smaller pieces of work that can be completed independently of each other, all contributing towards fulfilling this work.

• [ ] #59
• [x] https://github.com/artefactual-sdps/enduro/issues/1039