[loki-distributed] Loki Microservices Deployment is not working as expected.Getting "empty ring" error from distributor

Hi Team,

I have deployed Loki in Micro Services Mode.

All the Loki components are up and Running. But i could not forwarded the logs from promtail to Loki.

Below are my config for

distributor: ring: kvstore: store: inmemory

ingester:
  lifecycler:
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
  chunk_idle_period: 30m
  chunk_block_size: 262144
  chunk_encoding: snappy
  chunk_retain_period: 1m
  max_transfer_retries: 0

Initially, i have configured the loki-distributed gateway to promtail push link:

http://loki-distributed-gateway.avx.svc.cluster.local:80

Getting the below error log messages:

**cal:80 msg="final error sending batch" status=426 error="server returned HTTP status 426 Upgrade Required (426): "

127.0.0.1 - - [15/Jul/2021:12:12:07 +0000] 426 "POST /loki/api/v1/push HTTP/1.1" 0 "-" "promtail/2.2.0" "-"**

And after changing it to directly connecting the distributor address like below:

http://loki-distributed-distributor.avx.svc.cluster.local:3100

I am getting the below error messages:

level=warn ts=2021-07-15T11:21:47.817142708Z caller=logging.go:71 traceID=5ff38756cb099188 msg="POST /loki/api/v1/push (500) 2.535511ms Response: \"empty ring\n\" ws: false; Content-Length: 129429; Content-Type: application/x-protobuf; User-Agent: promtail/2.2.0; X-B3-Parentspanid: 5597fba85e06f9c2; X-B3-Sampled: 1; X-B3-Spanid: 7741e05cbc81b074; X-B3-Traceid: 79276c5f9d60a41f5597fba85e06f9c2; X-Forwarded-Client-Cert: By=spiffe://cluster.local/ns/avx/sa/loki-distributed;Hash=1a0d7951946ae7805f9ae9115e5a17096b6481fa7bfd5e0fa9f67969c8d907df;Subject=\"\";URI=spiffe://cluster.local/ns/avx/sa/promtail; X-Forwarded-Proto: http; X-Request-Id: 579ef2ac-ea21-9f88-923f-7e63f4d64dd7; "

Your input will be much appreciated.

grafana / helm-charts

[loki-distributed] Loki Microservices Deployment is not working as expected.Getting "empty ring" error from distributor #565