search

futurewei-cloud / alcor

Alcor: Cloud native SDN platform powered by Kubernetes and Istio
MIT License
32 stars 33 forks source link

[MQ Scaling Path] Support production grade deployment for MQ-specific mapping. #490

Closed xieus closed 3 years ago

xieus commented 3 years ago

Context

The DPM pulsar client needs two mappings in the following to enable message multicast, one is group topic to host ip mapping, and the other is multicast topic to group topic mapping. 

host.ip.to.group.topic.map=group-topic1:192.168.131.131,10.10.10.11 group-topic2:192.168.131.131,11.11.11.12
group.topic.to.multicast.topic.map=multicast-topic1:group-topic1,group-topic3 multicast-topic2:group-topic2,group-topic4

The mappings are currently stored as configurations in application.properties. For actual production deployment, this solution will not sustain due to the large number of production nodes.

Request

Think of a long-term solution to support production grade deployment. One possibility to store the mapping in node metadata manager, as this is a host-level resource.

xieus commented 3 years ago

@VanderChen @chenpiaoping Based on discussion in open-source meeting, we will need to break down the task into three parts: (1) Add two new fields on web entity (NodeInfo) and UTs change (2) Node Metadata Manager => DPM (async call) and new API in DPM for node registration (3) Fallback mechanism: DPM to retrieve missing data from NMM in case of cache miss

xieus commented 3 years ago

Addressed by PR #532