Port over tombstoning from SA

[straleyb]

Descriptive summary

Should we have tombstoning in OD2? Do we have a usecase?

[jsimic]

Metadeities would like tombstoning to work like this: A work can be removed from public view (by pushing back to review or deleting), but will remain accessible to admins and curators. Tombstoning will remove the work from search results and My Shelfs. If someone has it bookmarked or finds it in some other way (direct hits) there should be a message that indicates that the work has been removed from the collection.

[wickr]

Also have an option for admins and curators to 'undelete' and change status. Another page in the Dashboard, similar to oEmbeds, that will list deleted works and keep them separate from others in the system.

[carakey]

QA Fail - a work is still fully public after tombstoning.

QA process:

• From my admin account, I created and reviewed a test work
  • work: https://staging.oregondigital.org/concern/images/pz50gw130
  • ~fileset: https://staging.oregondigital.org/concern/parent/pz50gw130/file_sets/f0pz50gw130~
• From a personal "other affiliation" account in a private tab, I verified the work was public and findable in search results, and added it to My Collection titled "Cara Test"
• From my admin account, I used the Tombstone button on the work.
  • This gave the "Something has gone wrong" message
  • On reload I got the intended Tombstone message
• From my admin account, I used the direct URL expecting to see the tombstone, but the work loaded
• From my personal account in the same private browser tab as before, I navigated to the work in My Collection and it still loaded.
• Trying to prove this wasn't a cookie issue, I opened the work using a different browser, also private browsing, not logged in; I used the direct URL, and the work loaded. I was able to pull it up from search as well.

FWIW after the steps above, I tried to edit the fileset and got the ACK message, then I deleted the fileset and tried to add a new file but it would not stick.

In addition to these issues, some suggested improvements around error messaging: Give a more obvious "go back" link to return to the OD home page from the Tombstone. I realize the "Oregon Digital" text will get the user back home, but it would be clearer if included in the error message text, e.g. adding a link to the "using the search box" text, or after "...or contact us" add "You can also return to the home page."

[straleyb]

@carakey This might need another QA. I didn't run the update to the workflows on the staging server. Ill run that and then ping you and we can see how it effects the QA. Sorry about that.

[straleyb]

@carakey Ok it ran successfully and i restarted the web server so its good for another test.

[carakey]

QA Round 2: QA Fail

• [ ] From admin account, I attempted to add a file to the test work, which still did not stick. Something is broken
• [x] From a private tab, not logged in, I verified the work can still be viewed (as expected).
• [x] From admin account, I clicked the tombstone button on the work. I was routed to the "Tombstone Message Editor" page (new behavior since yesterday). I edited the default message and clicked the save changes button. I got the ACK message. Something is broken
• [x] From a private tab, not logged in, I tested finding the tombstoned work:
  • [x] I used the direct URL for the work and got the tombstone page. The tombstone message had not successfully updated and just said "Default" above the list of citations. Sort of working but needs improvement:
    • [x] Needs an actual default message instead of the word "Default," which suggests to admin users that a real, useful default message will be used when it isn't.
  • [x] I searched for the item by its title, and it appeared in search results. QA criteria not met - tombstoned works should not appear in search results
    • [x] I went to the search result and got the tombstone page
• [ ] Logging in to my personal/unaffiliated account, I accessed My Collections and the tombstoned work is still there QA criteria not met -- tombstoned works should be removed from My Collections
  • [ ] Again, going to the work from My Collection returns the tombstone page
• [ ] From my admin account I tested management of the tombstoned work.
  • [ ] I can access it from the link or from my dashboard >> works, but can only change the message, edit, delete. QA criteria not met: per Ryan W comment (1) I should be able to restore the work, but that is not available; (2) Expecting another dashboard page to manage tombstoned works, which is not there.

I have not yet tested what happens on a direct link to the fileset, since I can't get a fileset added to the test work.

[straleyb]

Thanks for going through all that @carakey. Can we make a list of features and the way we want it to work here and what is done/not done so we can track this a little better? Im gonna swap this over to an epic ticket and i think we should spawn out other tickets to cover the features/needs for tombstones.

[carakey]

Sure - I was almost entirely referencing the above comments about functionality.

[straleyb]

Totally. I was just thinking that I was unsure about what people's expectations are about how this should work. I just yanked it over from Scholars Archive and want to make sure that this feature has what everyone wants.

[carakey]

QA round 3:

Setup:

• From my admin account, I deleted the previous test work for this ticket, and created and reviewed a new test work
  • work: https://staging.oregondigital.org/concern/images/4x51hj066
  • fileset: https://staging.oregondigital.org/concern/parent/4x51hj066/file_sets/f04x51hj066
• From a personal "other affiliation" account in a private tab, I verified the work was public and findable in search results, and added it to My Collection titled "Cara Test"

Testing:

• [x] (As admin) I used the Tombstone button on the work. This operation completed without error and the work was updated. QA Pass with notes:
  • I received an email notification that "Requested deletion has been approved" which could be confusing, since "tombstone" and "delete" are not the same.
• [x] (As admin) I used the Tombstone Message Editor to update the tombstone display message. The message was updated. QA Pass with notes:
  • The default display message should still be updated from the "Default" text
  • Text formatting is needed, whether the default or custom message, as the text is easily missed -- has very little spacing before the bigger, bolder "CITATIONS" header
• [x] (Private tab / not logged in) I searched for the item by its title, and it did not appear in search results. QA Pass
• [ ] (Private tab / not logged in) I used the direct URL for the work and the one for the FileSet, expecting to see show pages with clear tombstone messaging. QA criteria not met
  • [ ] Criteria: The show page clearly communicates to a user that an object was formerly at this location but has been removed. Current behavior:
    • The tombstone page shows the title. No viewer or thumbnail is displayed. The item metadata is not displayed. There is a "Tombstoned" label on the work, though no label on the fileset.
    • The text "(Image)" and "(File)" appears on the two show pages, respectively. Neither the "Default" text nor the custom tombstone message shows. The citations also do not appear, as they do when logged in as admin.
    • The pages have the banner message "The work is not currently available because it has not yet completed the approval process." This is not accurate.
• [ ] (Private tab / non-admin account) I accessed My Collections expecting the tombstoned work to have been removed. The work is still listed and the thumbnail appears. QA criteria not met
  • [ ] Criteria: Tombstoned works should be removed from My Collections
    • Note: Clicking into the work from the collection leads to the tombstone show page as above, with the message about being in review and the "(Image)" text only.
• [ ] (As admin) I tested management of the tombstoned work. QA criteria not met
  • [x] Criteria: An admin should be able to restore the work
    • Current behavior is same as before, with options to change the tombstone message, edit or delete the work, but not to change the tombstone status.
  • [ ] Criteria: a separate dashboard page to manage tombstoned works (this should potentially be a separate ticket)

[straleyb]

• [x] Keep visibility as it was

• [x] Remove from search results

• [x] Make sure it doesn't show up in facets.

• [x] Remove from collection view as a non owner of collection

• [x] Tombstone page for user needs to display tombstone message and citations

• [x] Notice on Tombstone Page should reflect proper information: Not "This work is under review"

• [x] Add button on tombstone page to revert similar to tombstone button.

• [x] Default Message: "This item has been deleted"

[carakey]

QA round 4:

Setup:

• From my admin account, I came back to the previous test work and changed it back to active (Review --> Approve) so I could step through the process from the beginning. It now shows as Public/Deposited.
  • work: https://staging.oregondigital.org/concern/images/4x51hj066
  • fileset: https://staging.oregondigital.org/concern/parent/4x51hj066/file_sets/f04x51hj066
• From a personal "other affiliation" account in a private tab, I verified the work is public and findable in search results, and that it is still in a My Collection titled "Cara Test".
• Since my test work had previously been deleted with a custom message, it still showed the same custom message after being restored and then re-tombstoned. To check the default message, I tested tombstoning a different work to see the new default message, which has been fixed to say "This item has been deleted."

Testing:

• [ ] (As admin) I used the Tombstone button on the work. This operation completed without error and the work was updated. QA Pass for basic functionality, but improvement still needed to be production-ready:
  • The email notification wording was changed but not improved. Previously "Requested deletion has been approved"; now "The deletion for [item] was approved by [user]" which remains confusing for the same reason as before. My complaint is that it should say "tombstone" rather than "delete".
    • Why? From the admin side, tombstoned and deleted are different states. A deleted work is gone and its URL no longer exists. A tombstoned work still exists in the system, its URL loads a tombstone page, and it can be restored if needed. So the email that the admin gets should show the correct state. (However the "deleted" terminology is desired for user-facing messaging because users don't need to know the difference.)
• [ ] (As admin) I used the Tombstone Message Editor to update the tombstone display message. The message was updated. QA Pass for basic functionality, but improvement still needed to be production-ready:
  • Text formatting is still needed to make larger/bolder/more colorful, and/or add space before the bigger, bolder "CITATIONS" header -- in other words make it be visible.
• [x] (Private tab / not logged in) I searched for the item by its title, and it did not appear in search results. QA Pass
• [ ] (Private tab / not logged in) I used the direct URL for the work and the one for the FileSet, expecting to see show pages with clear tombstone messaging. QA criteria not met (only slight change from previous QA round)
  • [ ] Criteria: The show page clearly communicates to a user that an object was formerly at this location but has been removed. Current behavior:
    • The tombstone page shows the title. No viewer or thumbnail is displayed. The item metadata is not displayed. There is a "Tombstoned" label on the work, though no label on the fileset.
    • The text "(Image)" and "(File)" appears on the two show pages, respectively. Neither the "Default" text nor the custom tombstone message shows. The citations also do not appear, as they do when logged in as admin. The purpose of having a tombstone message is to communicate to the user! But that message isn't making it to the user.
    • The work show page has an updated banner message "This item is unavailable due to being deleted." I think this is awkward English (sorry). Can we go with "This item has been deleted. It is no longer available."
    • The fileset pages still have the banner message "The work is not currently available because it has not yet completed the approval process."
• [ ] (Private tab / non-admin account) I accessed My Collections expecting the tombstoned work to have been removed. The work is still listed and the thumbnail appears. QA criteria not met (no change from previous QA round)
  • [ ] Criteria: Tombstoned works should be removed from My Collections
    • Note: Clicking into the work from the collection leads to the tombstone show page as above, with the message about being in review and the "(Image)" text only.
• [ ] Criteria: An admin should be able to restore the work (As admin) I tested management of the tombstoned work. I did this at the beginning of the QA using Review --> Approve and it worked. I tested at the end with the Approve button and got an error message. QA fail on button
  • Error message: <span class="translation_missing" title="translation missing: en.hyrax.tombstone.revert">Revert</span>
  • We also still want a separate dashboard page to manage tombstoned works. Should I make this a new ticket?

[carakey]

Here's the TL;DR list of remaining changes to get to QA Pass (for Cara):

• [x] Email message to admin: Change "The deletion request..." to "The tombstone request..."
• [x] Make Tombstone Message appear on user view.
• [x] Format Tombstone Message to be visible above the Citations on both admin and user view.
• [x] Update banner message on both work and fileset show pages to "This item has been deleted. It is no longer available."
• [x] Remove tombstoned works from user collections (and all types of collections)
• [ ] Fix Approve button

[straleyb]

@carakey Thanks for the Tl;Dr list. The layout + needs written out in that way works way better on my end. Ill go through and make sure those get knocked out.

For my notes: The banner message is for a user that cannot see the work and fileset. Different view needs added tombstone partial.

[carakey]

@carakey Thanks for the Tl;Dr list. The layout + needs written out in that way works way better on my end. Ill go through and make sure those get knocked out.

Sure thing - but I see the long format as legacy documentation, helpful for capturing the full current state, for replicating the steps in future QA rounds, and capturing the rationale. So I'm gonna keep doing that too.

[straleyb]

Tombstoned work that is unavailable

Tombstoned FileSet that is unavailable

Tombstoned work

d6"

FileSet of Tombstoned work

Search Before Work is Tombstoned

Search After Work is Tombstoned

With Creator in Facets [Aea, J. K]

With Creator in Facets Tombstoned [Aea, J. K]

Work not Tombstoned in Collection

Work Tombstoned in Collection

[straleyb]

@carakey This is ready for another pass through. I posted pictures of my testing on staging for the different areas of functionality based on the Tl;Dr from last week and december 19th. Let me know if there are other hanging bits. I haven't been able to get those pesky buttons laid out proper, but I don't want that specific thing to hold up any QA for the core functionality of what we need for Tombstones to be operating within the system.

[straleyb]

One bug I noticed is its sending two notifications out. One deletion and one approval. Need to find the source of where this extra notification is coming from.

[straleyb]

Ok I think the search error I was having is actually random issue with a work. I created a new work, approved it, searched, it showed up, then tombstoned it and searched again and it was gone. Search worked properly.

[carakey]

QA round 5:

TL;DR list

• [ ] Get rid of the second notification email; keep the one that uses the "tombstone" wording.

• [ ] Remove the text string with the work type in parentheses from the show page of a tombstoned work (top right, just below title).

• [ ] Prevent tombstoned works + thumbnails from appearing in user collections

• ---

Full QA Notes:

Setup:

• From my admin account, I came back to the previous test work and changed it back to active (Review --> Approve) so I could step through the process from the beginning. It now shows as Public/Deposited.
  • work: https://staging.oregondigital.org/concern/images/4x51hj066
  • fileset: https://staging.oregondigital.org/concern/parent/4x51hj066/file_sets/f04x51hj066
• I added a creator (Aa, A. J. van der) to my test work to QA the faceting piece.
• From a personal "other affiliation" account in a private tab, I verified the work is public and findable in search results, and that it is still in a My Collection titled "Cara Test".

Testing:

• [ ] (As admin) I used the Tombstone button on the work. This operation completed without error and the work was updated. **QA Pass for basic functionality, but still seeing that bug with the two emails, as Brandon noted. Email should use "Tombstone" wording.
• [x] (As admin) I used the Tombstone Message Editor to update the tombstone display message. The message was updated. The message has been formatted and is more visible than previous QA rounds. QA Pass
• [x] (Private tab / not logged in) I searched for the item by its title, and it did not appear in search results. QA Pass
• [x] (Private tab / not logged in) I used the direct URL for the work and the one for the FileSet, expecting to see show pages with clear tombstone messaging. Criteria: The show page clearly communicates to a user that an object was formerly at this location but has been removed. QA close but could still be cleaned up Current behavior:
  • [x] The tombstone page shows the title. No viewer or thumbnail is displayed. The item metadata is not displayed. There is a "Tombstoned" label on the work and now also on the fileset. Good stuff!
  • [x] Show page for work and fileset still has the worktype in parentheses off to the far right side - (Image) on my test item, and I also see (Generic) in @straleyb 's screenshots.
  • [x] Updated banner message on both work and fileset: "This item has been deleted. It is no longer available."
• Criteria: Tombstoned works should be removed from collections - public/digital curated collections, and private/users' My Collections.
  • [x] (Private tab / non-admin account) I accessed the public collection that contains my second test item and it was not listed
  • [ ] (Private tab / non-admin account) I accessed My Collections expecting the tombstoned work to have been removed. The work is still listed and the thumbnail appears. QA criteria not met
    • Note: Clicking into the work from My Collection leads to the (QA Almost Passing) tombstone show page as above.
• [x] Criteria: Tombstoned works' metadata do not appear in facets.
  • QA Pass The one-off creator that I added to my test work does not appear in facet list when the work is tombstoned.
• [x] Criteria: An admin should be able to restore the work (As admin) I tested management of the tombstoned work. I did this at the beginning of the QA using Review --> Approve and it worked. I tested at the end with the Reapprove button and it worked. QA Pass

[straleyb]

@carakey Is the expectation for the (Image) text just to be removed? Or is there some sort of layout that is more desired? Also, could you provide a screenshot for the my collection thing? Im confused about being able to reproduce this error.

[carakey]

@straleyb Just to remove the (Image) text. I don't know any reason why it should be there.

re: My Collection showing tombstoned work:

^ The tombstoned test work is only thing that's been added to this My Collection (which is a private user collection created with my personal gmail address). This item is in tombstoned state but the collection still shows as having 1 item in the list of my collections.

^ In the collection view, the thumbnail for the tombstoned item still appears and the title shows on hover. If I click this I get to the tombstone show page, but it would be better if it were removed from all collections including user collections.

[carakey]

I am getting different results in my unaffiliated user collection from what @straleyb is seeing. Trying to reproduce:

1. As unaffiliated cara_test user, add a test work (this one) to a user collection "Cara Test." Also added another work that won't be tombstoned for this test. Verify both appear in my Cara Test collection, along with previously tombstoned test work.
2. As admin user, tombstone the new test work.
3. As cara_test user, navigate to Cara Test collection. See three thumbnails showing, 2 of them for currently tombstoned works.

[carakey]

QA round 6:

1. Right away as I started QA, I saw that my "Cara Test" User Collection thumbnails still included two tombstoned works. However, after going through the process of re-approving and then re-tombstoning the test works, the thumbnails did disappear from my collection. FWIW I re-approved one yet again and it did show back up in the list of works in my collection, but did not reappear as a thumbnail (no other thumbnail replaced it, either).

2. The string displaying the work type in parentheses has been removed from the tombstoned work's show page.

3. I'm still getting the two emails when I tombstone a work - one with the "Tombstone request" and the other with the "deletion request" language.

Assuming that the my collection issue is only with works tombstoned prior to the most recent changes, and since the emails aren't user facing, I can QA pass this as MVP. We'd still like to see the emails fixed and the tombstone management feature in the dashboard.

[straleyb]

1: This one should be fine once we deploy. There shouldn't be any tombstoned works attached to any collections so the process should work of tombstoning an item and it being removed should function as expected. The current functionality of re-adding a work that was previously featured as featured if it is reapproved wasn't discussed. Currently you can just re-add it back as a featured fileset from the collection edit page, so it should be fine. If this is a piece of functionality we want then it should be discussed as an enhancement for this rather than MVP.

2: I think this is what was discussed previously here Show page for work and fileset still has the worktype in parentheses off to the far right side - (Image) on my test item, and I also see (Generic) in @straleyb 's screenshots.

3: This problem shouldn't be an issue when we deploy. While updating and working on tombstones, multiple notifications were used. Once this gets deployed, production will only know about the tombstone notification and shouldn't have the delete one.

The tombstone management feature should be discussed by POSM for needs and requirements, then a ticket should be made for it. Thats another quite significant chunk of work and should be discussed and requirements laid out before I start tackling that.

[jsimic]

@straleyb How hard would it be to add Tombstoned to the filter to the Works > Suppressed Bsi?

[carakey]

If those two items will be resolved on deploy, then this is ready to QA pass except for Julia's question.