search

reTHINK-project / core-framework

The main goal of WP3 is to provide the reTHINK core framework comprised by the runtime environment where Hyperties are executed and the messaging nodes used to support messages exchange between Hyperties.
Apache License 2.0
1 stars 0 forks source link

How to keep the Runtime Catalogue synchronised with Catalogue Server #162

Closed pchainho closed 8 years ago

pchainho commented 8 years ago

It is planned to support persistence for the catalogue runtime through a generic persistence component:

https://github.com/reTHINK-project/dev-runtime-core/issues/39

This issue is to discuss a mechanism that keeps data in the runtime catalogue synchronised with the Catalogue Server, which avoids to download the full data object, every time it is requested.

Different options:

Other options?

@emmelmann-fokus , @antonroman , others?

rjflp commented 8 years ago

I think the last would be hard to implement, as several Message Node can connect to a single Catalogue Server, right. This would require the Catalogue Server to maintain state about the Message Nodes it had interacted with. Deal with Message Nodes not being available temporarily, etc. Much more complex and harder to implement than, e.g. the second option.

antonroman commented 8 years ago

I also think that the second option is easier to implement and support. We could use the signature attribute which can also be used as the key for the persistence component (I have not calculated it but I guess a collision is very unlikely to happen). For the sake of debugging and code maintenance I think it would be also good to include also a new version parameter, which combined to the signature can be used as key for the persistence module.

The process would be like this:

  1. When the runtime needs to use a element from the Catalogue Server it would check for the signature and current version of the element. This request is supposed to be much lighter than the request to ask for the code.
  2. We would receive a JSON with both values from the Catalogue Server.
  3. Once it gets the JSON with the current version and the signature, we pass it to the persistence module.
  4. If the persistence module has already the same version then it will return the Object as a response.
  5. If not it will return an error and the runtime Catalogue we'll have to ask for the actual element to the Catalogue Server.

Does this make sense to you? Do you see any problem in this approach?

pchainho commented 8 years ago

Agree however I believe the CatalogueURL would be better for the key.

antonroman commented 8 years ago

You mean to use the CatalogueURL as key for the persistence module, right? I agree, but we should use the version in the key. This way we can update the Catalogue Object in the persistence module if the version of the Object stored is different from the version we already have. I will update issue reTHINK-project/dev-runtime-core#39 to explain how it will work.

To achieve this is has to be possible to ask for the Catalogue Server for the version of a Catalogue Object. I'll update issue https://github.com/reTHINK-project/dev-catalogue/issues/3 to ask for this new requirement to be included in the next release.

In the diagram in https://github.com/reTHINK-project/dev-service-framework/tree/develop/docs/datamodel/hyperty-catalogue the version attribute is not present in the CatalogueDataObject but it appears in the text below. I see that in the current version of the Catalogue Server the version attribute is included.

pchainho commented 8 years ago

ok, I'll update the hyperty catalogue data model

emmelmann-fokus commented 8 years ago

I basically agree with the approach that @antonroman suggests. I had the same line of thought.

I would suggest that we extend every catalogue object with a "version" entry that contains a strictly monotonic version number (e.g., reverse date-time stamp, i.e.: 20160129102954).

@antonroman, yes, please update the issue in dev-catalogue.