DSpace / DSpace

(Official) The DSpace digital asset management system that powers your Institutional Repository
https://wiki.lyrasis.org/display/DSDOC7x/
BSD 3-Clause "New" or "Revised" License
853 stars 1.29k forks source link

Projections break object cache #9552

Open hutattedonmyarm opened 1 month ago

hutattedonmyarm commented 1 month ago

Projections, according to their own description are supposed to be used to add or omit information from objects on their route from a JPA entity to a HALResource.

Actually doing so is a pain and prone to break things: The object cache used by the RequestService only cares about an object's UUID/self-link and does not respect projections. That is, requesting an object with projection X adds it to the cache and then requesting it with projection Y loads the exact response from the cache, disregarding that the response might be different.

This works fine if all you care about are embeds, because they have tons and tons of special-case handling everywhere. But it completely falls apart if custom projections are desired which actually add or omit data.

Here's the concrete use case and issue we're having: Due to the abysmal performance with embedding we created a custom projection to use with search result lists (related entities on a person page, the actual search, and so on). This projection adds the thumbnail bitstream id (if it exists) to the item. We're storing this in Solr, so it's fast to retrieve as well. The projection also omits a whole slew of metadata which is not needed for the limited search result component. This saves us from using embeds in the results and measurably increases our search performance.

However, clicking a result opens the item page. The item page loads the item (with the default projection), fetches it from the cache and uses that to populate the item page. The cached item is only partial (due to our projection), causing an incomplete item page. Ideally, the cache should recognize that another projection has been used and refrain from serving its content. We triaged three solutions, all with their own problems:

tdonohue commented 1 month ago

Moving this to DSpace/DSpace as "projections" are a backend feature which we've borrowed from Spring Data JPA Projections (provided in Spring Boot).

In any case, this likely will require more analysis on best practices from Spring for how to better cache projections (or even whether all projections may be cacheable, especially if they return modified objects)