Hi all. I found a very strange behaviour in export service if adding multiple routes. Seems that sometimes the workers goroutines remains waiting also if there is data in the channel. This issue appears randomly, but i managed to reproduce it with workers = 1 and GOMAXPROCS= 1 (but the issue can appear also with default settings).
This is my setup
i also have an external process that publishes on nats "measures" topic.
if i remove the second route, it works
if i switch the 2 routes, it works
if the first route defined is the "measures" one , then the r.Consume() worker goroutine remains waiting on the for msg := range r.Messages, and never process messages also if the channel is full / partially full
Seems a strange concurrency issue, but i cannot understand it. Tried some workarounds to better understand the issue that works:
change the for := range with a select and added a default case with a sleep. In that case it works
subscribe to nats with a callback instead of channel, and manually insert values in the channel:
for _, r := range e.consumers {
//_, err := nc.ChanQueueSubscribe(r.NatsTopic, exportGroup, r.Messages)
_, err := nc.QueueSubscribe(r.NatsTopic, exportGroup,func(msg *nats.Msg){
r.Messages <- msg
})
if err != nil {
e.logger.Error(fmt.Sprintf("Failed to subscribe to NATS %s: %s", r.NatsTopic, err))
}
for i := 0; i < r.Workers; i++ {
go r.Consume()
}
}
Hi all. I found a very strange behaviour in export service if adding multiple routes. Seems that sometimes the workers goroutines remains waiting also if there is data in the channel. This issue appears randomly, but i managed to reproduce it with workers = 1 and GOMAXPROCS= 1 (but the issue can appear also with default settings). This is my setup
i also have an external process that publishes on nats "measures" topic.
if i remove the second route, it works
if i switch the 2 routes, it works
if the first route defined is the "measures" one , then the
r.Consume()
worker goroutine remains waiting on thefor msg := range r.Messages
, and never process messages also if the channel is full / partially fullSeems a strange concurrency issue, but i cannot understand it. Tried some workarounds to better understand the issue that works: