search

jvalue / ods

Open Data Service - Make consuming open data easy, safe, and reliable
GNU Affero General Public License v3.0
36 stars 23 forks source link

Adapter preview persistence #283

Open mathiaszinnen opened 3 years ago

mathiaszinnen commented 3 years ago

Currently all imported data is persisted in the adapter, even if the result is returned directly and not a reference to the persisted data. I suggest we adapt the /preview and /preview/raw endpoints so that data is always included there and never persisted. Another possibility would be to move the persistence to the stateful datasource package altogether and never persist anything in the adapter package like proposed in #267 . Any opinions @georg-schwarz , @lunedis , @sonallux ?

georg-schwarz commented 3 years ago

I like the second idea. We only need to persist data that is part of the internal flows, so it should be part of the datasource domain instead of the stateless adapter domain.

lunedis commented 3 years ago

How does this issue relate to #135 ?

It seems like we should discuss a refactoring and/or redesign of the adapter/datasource service.

georg-schwarz commented 3 years ago

Resembles the last bullet point of #135. I think this issue #283 is the first step to complete the refactoring in #135 afterwards for better paths to the imports.

mathiaszinnen commented 3 years ago

It seems like we should discuss a refactoring and/or redesign of the adapter/datasource service.

Agreed, I think both issues are closely related and it probably makes sense to handle both doing a major refactoring..

georg-schwarz commented 3 years ago

Duplicate of #267 btw if we handle it like this.