search

caltechlibrary / irdmtools

A Go and Python package for working with InvenioRDM repositories.
https://caltechlibrary.github.io/irdmtools
Other
1 stars 1 forks source link

Narrow the focus of rdm tooling for Go based tools to working with Postgres #76

Open rsdoiel opened 6 months ago

rsdoiel commented 6 months ago

RDM has a wonderful JSON API but it is (by necessity) rate limited. This makes balancing the performances characteristics for what is needed for internal processing as compared to public access a challenge. The RDM database schema and architecture is well constructed via the ORM Invenio employees. This means that with a little reverse engineering we can recreate the JSON input and puts right from the database itself by taking advantage of Postgres SELECT, VIEWS and FUNCTIONS. irdmtools for the Go elements should work directly with the database archicture except in record creation or updating. The Go based tooling can be simplify by removing support for the JSON API when not required for reading data.

rsdoiel commented 4 months ago

I've removed JSON API from many of the funcs in invenio_api.go. Need to update tests to reflect that. Also need to consolidate duplicate code shapes then cleanup the remaning codebase and update the package level documentation comments to reflect the changes.