updates made in admin pages not showing in mockup

only1chunts commented 10 months ago

User story

As a curator I want to see my changes appear in the pre-publication view of the dataset page So that I can confirm the changes are correct and show the relevant users the private mockup page displaying the correct information

Acceptance criteria

Given I make an update to any dataset metadata in the admin pages When I click view "open private URL" Then the dataset page is displayed at the private token url with those changes displayed

Given I make an update to any sample metadata in the admin pages When I click view "open private URL" Then the dataset page is displayed at the private token url with those changes displayed

Given I make an update to any file metadata in the admin pages When I click view "open private URL" Then the dataset page is displayed at the private token url with those changes displayed

Given I make an update to any author metadata in the admin pages When I click view "open private URL" Then the dataset page is displayed at the private token url with those changes displayed

Additional Info

may be related to #963

Product Backlog Item Ready Checklist

[ ] Business value is clearly articulated
[ ] Item is understood enough by the IT team so it can make an informed decision as to whether it can complete this item
[ ] Dependencies are identified and no external dependencies would block this item from being completed
[ ] At the time of the scheduled sprint, the IT team has the appropriate composition to complete this item
[ ] This item is estimated and small enough to comfortably be completed in one sprint
[ ] Acceptance criteria are clear and testable
[ ] Performance criteria, if any, are defined and testable
[ ] The Scrum team understands how to demonstrate this item at the sprint review

Product Backlog Item Done Checklist

[ ] Item(s) in increment pass all Acceptance Criteria
[ ] Code is refactored to best practices and coding standards
[ ] Documentation is updated as needed
[ ] Data security has not been compromised (with particular reference to the personal information we hold in GigaDB)
[ ] No deviation from the team technology stack and software architecture has been introduced
[ ] The product is in a releasable state (i.e. the increment has not broken anything)

rija commented 10 months ago

Additional notes: @only1chunts can reproduce it on Chrome. Neither Yannan, nor @tuli have noticed the issue

rija commented 10 months ago

Hi @only1chunts

1) The main problem:

I was able to replicate the experiment you showed us during last meeting that reproduce the issue whereby the mockup is not reflecting the changes to the database.

The reason it is happening is because of the way the application decide wether the cached data has expired or not. It is entirely based on whether there is a new entry added to the curation log associated with the dataset for which data (included related data) is being updated.

So if you change authors, files, or even the dataset description but without adding a curation log entry, the mockup will still use cached data, because none of those actions results in a curation_log entry.

I'm guessing this is not a behaviour desired by curators trying to enter a dataset in the system with back-and-forth liaising with authors and with need to preview content at every step.

So, it sounds like disabling use of cached data all-together when opening up a mockup page would be reasonable remedy for the problem at hand?

2) Other issues that can potentially make operators perceive the problem more often

a. Clicking the "Open Private URL" url does NOT save the form, so if the operator forgets to click "Save" to save changes, and click the URL directly, changes are not shown in the mockup even if problem 1) didn't exist

b. If multiple operators work on changing the same dataset at the same time, both the dataset updates and the mockup rendering may become inconsistent

only1chunts commented 10 months ago

So this ticket will results in a fix to poin 1 above.
Ticket #1533 should fix point 2a above
will you (@rija ) create a ticket to address point 2b above? are there easy ways to put some sort of lock to prevent multiple people working on the same admin page at the same time? or maybe just a warning when a save is attempted to let the user know data is not able to be saved at this moment in time and why, and maybe try again later? While we are a small team its not likely to be an issue, but if/when we grow the chances will increase so it needs to be addressed at some point. (i.e. the new ticket for this part does not need to have the "must-have-for-go-live" label)

tuli commented 10 months ago

_So if you change authors, files, or even the dataset description but without adding a curation log entry, the mockup will still use cached data, because none of those actions results in a curationlog entry.

Could we discuss these (and other) actions resulting in an automatic curation log entry?

rija commented 10 months ago

Hi @only1chunts,

I will create a ticket for 2b, but I don't think 2b is a must for going live, so it should be scheduled as a regular product backlog item. It's not a trivial functionality to implement. It can be become quite complex depending on how much UI feedback and control is needed. The ticket I'll create will assume the simplest thing that can work, and we can discuss it during PB refinement workshop.

And yes, #1533 seems to take care of 2a

rija commented 10 months ago

_So if you change authors, files, or even the dataset description but without adding a curation log entry, the mockup will still use cached data, because none of those actions results in a curationlog entry.

Could we discuss these (and other) actions resulting in an automatic curation log entry?

Hi @tuli,

Yes, I also think it's a worthy topic of discussion, clarification and maybe changing some behaviours, I just wonder when and how is the right context to do so?

In most database based applications I've been working on, all the tables have a created_at and updated_at columns automatically populated with the right values at the right time (automated time stamping) without human intervention and caching invalidation is often based on changes to these values. That mechanism and manual logging have their distinct purposes and are not mutually exclusive, and can both be used together.

But the current situation is that we tried to do global caching invalidation based of manual curation logging which really doesn't work well.

For the scope of this ticket though, I'll make changes to have the mockup to always fetch what's exactly in the database whenever it is summoned, irrespective of workflows and caching rules.

Maybe you can point at a ticket (or create one if not existing) where we can discuss the scope of manual curation logging and when it's good enough to let the automated time stamping do its thing?

gigascience / gigadb-website