I've made a configuration for Loki that consists of 2 Loki clusters that are separated from eachother in each network zone and they use the same storage platform.
This architecture somewhat works.. But there's a problem when using a single querier and trying to retrieving in-memory/WAL logs from seperate Loki clusters.
Will refer to a picture below..
Protected Network-Zone are allowed to send egress traffic to Exposed Network-Zone.
Exposed Network-Zone are not allowed to send ingress traffic to Protected Network-Zone.
My goal is I want to be able to read live-logs from either ingester in each network-zone from the querier.
Problem is I can't read in-memory/WAL logs from the Exposed Network-Zone. I have to wait until ingester-B have filled out it's Chunks and sent it to the cloud storage.
For querier-A to query ingester-B inmemory/WAL logs I have to add ingester-B to querier-A's memberlist, i could not figure out any other way, the querier will however unpremediated tell the other Loki components to add it to their ring, and that declares all the components in every cluster to think they're in the same cluster
And it will error and make the cluster unhealthy since the components in Exposed Network-Zone can't start and send new network traffic to components in the Protected Network Zone.
I'm looking for a suggestion on a alternative configuration on how to accomplish my goal with the same architecture.
Otherwise the solution i'd like would be a configuration option for querier component to read from additional ingester without forcing membership or a ring.
I've made a configuration for Loki that consists of 2 Loki clusters that are separated from eachother in each network zone and they use the same storage platform. This architecture somewhat works.. But there's a problem when using a single querier and trying to retrieving in-memory/WAL logs from seperate Loki clusters. Will refer to a picture below.. Protected Network-Zone are allowed to send egress traffic to Exposed Network-Zone. Exposed Network-Zone are not allowed to send ingress traffic to Protected Network-Zone. My goal is I want to be able to read live-logs from either ingester in each network-zone from the querier. Problem is I can't read in-memory/WAL logs from the Exposed Network-Zone. I have to wait until ingester-B have filled out it's Chunks and sent it to the cloud storage.
For querier-A to query ingester-B inmemory/WAL logs I have to add ingester-B to querier-A's memberlist, i could not figure out any other way, the querier will however unpremediated tell the other Loki components to add it to their ring, and that declares all the components in every cluster to think they're in the same cluster And it will error and make the cluster unhealthy since the components in Exposed Network-Zone can't start and send new network traffic to components in the Protected Network Zone.
I'm looking for a suggestion on a alternative configuration on how to accomplish my goal with the same architecture. Otherwise the solution i'd like would be a configuration option for querier component to read from additional ingester without forcing membership or a ring.