Edit to collection relationship doesn't update on save

[HKativa]

Cat and I were testing user roles for relating this record to a collection: record ID 7m01bm63b. Record was already related to collection ID 2j62s582j, which I created. Wanted to test relating record to collection that Cat had created. Related object to collection ID sj139253d and record updated on save. We then edited the record again to unselect collection ID sj139253d, but on save, both collections are still related to object. We think this may be a bug, but could also be permissions-related - not sure.

[HKativa]

Update: Was able to remove the record from Collection (sj139253d) page.

[HKativa]

@eddierubeiz I was able to replicate this bug so just follow-up with me whenever you want to see my workflow.

[eddierubeiz]

I have some intermediate results to report. This works just fine if the collections are small. If I create three empty test collections in staging, as well as a fresh test work that's not part of any collections, this bug doesn't occur at all.

On the other hand, if you try to add or remove an item from a large collection, the bug is flagrantly obvious. The change is not showing up on the item record after you save because the change is actually still underway.

Removing an item from the Beckman collection in Staging from the console appears to take between five and seven minutes and causes a flood of log messages. It appears that the entire collection is being traversed at least once in the process, even though only two items need to be adjusted -- the collection and the item.

In short: something ain't right.

[jrochkind]

This is now ringing a bell (sorry I didn't remember this initially), and I think @sanfordd thought so too -- I think it's possible this is a long-standing problem which has existed always in sufia, to be run into as soon as your collections get that big, which we've actually long been aware of and @catlu maybe had some way of working around, which for her workflow was good enough.

From what i remember of the internals of the problem, fixing it the "right" way may not really be feasible for us right now.

Might be worth asking @catlu for a favor to check in and see what she knows and if she had a workaround.

[HKativa]

Interestingly, @catlu and I discovered and documented this behavior together when we stumbled across it while experimenting with user roles and permissions issues. I can follow up with Cat to see if she has a work-around, but ultimately feel based on this thread and further conversation with @eddierubeiz, that this is a low-priority beyond what Eddie's already documented and as a tool for learning more about queuing behavior. For one, changing the collection to which a work belongs isn't a typical user behavior; switching a work from one collection to another was just what Cat and I used to test whether I could add works to a collection I didn't create. Also, it's primarily an issue for larger collections, which again diminishes the value of devoting more work to this.

[jrochkind]

Ah, that's good info, thanks @HKativa ! I suspect that workflow never/rarely involved doing this before, and that it has been always broken all along. Or rather, once our collections got big enough to break it.

Sufia, unfortunately, is full of bugs, there are certainly many we have yet to find. :)

[HKativa]

@jrochkind That sounds perfectly plausible. As I told @eddierubeiz, based on our workflow, it's quite possible that we might not have noticed this for years, just a funny coincidence that Cat and I found it while doing something else entirely.

[eddierubeiz]

Yup. For what it's worth, it's trivially easy to write a Rake task to add / remove items from a collection. If a) this functionality is genuinely broken (as opposed to really slow) for large collections and b) we seldom if ever perform this operation, a Rake task might be the perfect band-aid. But I still haven't proven a) and @jrochkind gives me reason to suspect it's not really as broken as I thought it was.

[eddierubeiz]

A couple updates from my work on Friday:

• Adding and removing items from collections is not done in a background job.
• Removing a test item from the Beckman collection in staging fails silently.
• Meanwhile, the proposed workaround is also not practical for large collections -- you have to go to the collection page, click "Edit", and then laboriously look through every item in the collection until you find the one you want to remove.
• I did write a background job that does this, and I know how to call it at the right time with the right arguments. But I still need to figure out how to disable the existing broken functionality.
• At today's tech meeting, we decided to table this issue; as noted above we simply don't do this enough to warrant really fixing the bug.
• I will, however, add some sample lines to the wiki that explain how to do this from the Rails console.

[eddierubeiz]

How to do this from the console: https://chemheritage.atlassian.net/wiki/spaces/HDCSD/pages/15368194/Application+administration?atlOrigin=eyJpIjoiZGIxMTlkNDI0ZmVmNDQzZThkODk0ZWRiYjEyYWYwOGMiLCJwIjoiYyJ9

[jrochkind]

Decided not to handle for now, long-standing issue, difficult to fix because of sufia architecture, not part of our current workflows.