SwissDataScienceCenter / renku-search

2 stars 0 forks source link

Consolidate redis streams #182

Closed eikek closed 2 weeks ago

eikek commented 1 month ago

Currently, there are the following redis streams

Now we want to have fewer streams with the idea of one per entity type. This would be then

As a first step, search can read from all streams combined and once data services pushes only to the new streams, the previous ones can be removed here.

eikek commented 4 weeks ago

The next discussion brought us to use only a single stream to sync data between data-services and search components.