search

k-int / gokb-phase1

Original GOKb repo - Moving to https://github.com/openlibraryenvironment/gokb
http://www.gokb.org
Other
11 stars 5 forks source link

Package OAI response is too static #605

Open hornmo opened 7 years ago

hornmo commented 7 years ago

The current OAI response layout for packages always consists of 3 full packages with all of their TIPPs. This response can get (arbitrarily) large for big packages (i.e. eBooks), which may cause memory/performance issues at both ends. It should be considered to split up this response in some way (maybe return single TIPPs with detailed package information each?). This will of course introduce significant breaking changes, but it may be necessary to ensure a reliable performance of the API.

ianibo commented 7 years ago

This is fair feedback, but I think better solution is to devise a more granular mechanism, and then persuade users to adopt it, than to change the package endpoint.

Large packages are a problem, but they are a reality we need to deal with, and if chunking three packages per request is too many then sooner or later we will see a single package which is too big on it's own.

The most obvious solution seems to be development of functionality around the tipp endpoint. Allowing users to essentially "Subscribe" to changes to a package at the tipp level would make the process much more granular. We could, in fact, remove all references to tipps from the package endpoint. This would mean that the package endpoint could be used to indicate CRUD operations on the packages themselves, but that the detail of package contents could only be derived from the tipp endpoint.

For actual usage, the tipp endpoint is likely to be more useful anyway - and it has the added advantage of already existing :)

A less appealing approach might be to have a "Package Events" feed - with package and tipp level CRUD events mixed together like PACKAGE-1 CREATED {Package details} PACKAGE-1 TIPP-ADDED {Tipp details} PACAKGE-1 TIPP-ADDED {Tipp details} PACKAGE-1 TIPP-UPDATED {Tipp details} PACKAGE-1 PACKAGE-UPDATED {Package details}

This feels pretty ugly tho, as it mixes up the resource metaphor with an event based approach, and would involve us putting action verbs into the resource.

Not sure how to move this forward just at the moment tho - it certainly impacts how we might approach FOLIO KB work as it relates to GOKb.

I intend to mark this as backlog and close it out if thats OK as it's not a current bug - unless someone has a better suggestion?

hornmo commented 7 years ago

I think it's fine to put it aside for now. It seems to me, that the solution of providing detailed changes via the tipp endpoint should be the way to go.