update the LDES client (when it becomes available)

[marc-portier]

There has been quite some recent activity on the used LDES client at https://github.com/TREEcg/event-stream-client/issues

Specially there has been some work to

• support kickstarting the ldes sync with a pre-made dump at certain date (for MRG these are at https://www.marineregions.org/export/)
• ensure the ldes client checks the entity state (lastmodified) to avoid going to the end (

Also note: there is docker-packaging foreseen as well, might be worthwhile to investigate? https://github.com/TREEcg/event-stream-client/issues/67

[nvdk]

should we consider using the VSDS building blocks? LDIO seems interesting

[marc-portier]

as discussed: yes this does make sense

might be wise to upfront what elements / features / aspects we are hoping/looking for? when you investigate this, it might help to then also have some report focus on these in order to see what gaps would need to get closed ?

from our side:

• easy deployment inside docker-compose setup
• basic configuration per ldes feed to harvest would need to include
  • target sparql endpoint to have the triples stored
  • (assuming graph context per member identifier for updates?)
  • if needed: separate write sparql path & write credentials
  • uri to the feed
  • rate request limitation (requests per minute)
  • something to control de-referencing depth --> preferably through (list of) required property-paths to fetch, alternatively some shape-description ?
• advanced config
  • understanding of how to handle setup/config for multiple distinct feeds (e.g. through one or multiple docker instances?)
  • allow specifying URL to bootstrap dump (introducing initial state of all available last versions at a certain time in history)
• expected way of working
  • including smart state management to avoid crawling into history needlessly
  • supporting some easy full reload from empty state on demand
  • able to consume (gzipped) bootstrap-dumps (see advanced config)