The goal of this RFC is to clearly separate the data flow for Invenio into several layers: a data access layer, a business logic layer and view layer.
The data flow in Invenio has been shown to suffer from several issues that overall causes confusion as well as possibly produce hard-to-main code due to lack of very clear separation of responsibilities. Examples of issues with the data flow:
Serializers can fetch records from both the database and Elasticsearch but these representations are not identical causing serializers to having to first transform them to a standard format.
Marshmallow schemas in loaders have dependencies on the Flask request context making them impossible to use outside of a request.
The $schema attribute has not clear place to be injected - sometimes it is added to the marshmallow loader other times not.
Indexing receivers receive all signals instead of only for records they actually can handle.
Indexing is based on the $schema attribute, but sometimes this needs further customization.
Deposit needs to preserve some fields in the JSON that should not be overwritten by the user.
A record class can be used in several modules (example Invenio-Records-REST and Invenio-GitHub), but does not provide proper business-level validation, just structural validation via the JSONSchema.
The $schema attribute is supposed to be used for versioning, but it hides the feature.
Co-writing on Codimd
The goal of this RFC is to clearly separate the data flow for Invenio into several layers: a data access layer, a business logic layer and view layer.
The data flow in Invenio has been shown to suffer from several issues that overall causes confusion as well as possibly produce hard-to-main code due to lack of very clear separation of responsibilities. Examples of issues with the data flow:
$schemaattribute has not clear place to be injected - sometimes it is added to the marshmallow loader other times not.$schemaattribute, but sometimes this needs further customization.$schemaattribute is supposed to be used for versioning, but it hides the feature.