search

nsqio / go-nsq

The official Go package for NSQ
MIT License
2.59k stars 444 forks source link

Ensuring a topic and channel are registered prior to consumption #188

Closed isadon closed 8 years ago

isadon commented 8 years ago

Im getting the following error on a consumer to that im adding to my app:

2016/08/25 23:06:09 consumer.go:459: INF    1 [sns_chat/send] querying nsqlookupd http://:4161/lookup?topic=sns_chat
2016/08/25 23:06:09 consumer.go:464: ERR    1 [sns_chat/send] error querying nsqlookupd (http://:4161/lookup?topic=sns_chat) - got response 404 Not Found "{\"message\":\"TOPIC_NOT_FOUND\"}"

Obviously the consumer.go code is expecting my topic "sns_chat" to exist and hence why it shows the message. But of course that topic doesnt exist because the producer on that topic hasn't sent its first message yet, at some point it will. Why Is the above message being returned? If a topic and ch don't get created until after the first publish why is creating a new consumer not just transparently handling that.

Can the errors above just be ignored and can I just expect that consumer will receive the message later when it does get sent?

ploxiln commented 8 years ago

Consumers which have been configured with nsqlookupd addresses should continue to periodically poll the nsqlookupd instances, every 30 seconds or so, and find the topic after it is created.

The message will queue under the topic if there are no channels. Then, the first channel created for that topic will get the message, and the next (different) channel created will not get the message, because it drained from the topic at a time when the second channel did not exist.

If that is a problem, you can pre-create topics and channels in nsqd, see http://nsq.io/components/nsqd.html#post-topiccreate

Also, when a topic first comes into existence in nsqd, if it is configured with nsqlookupd addresses, it will query nsqlookupd for any known channels for that topic, and create the channels simultaneously. This avoids the "first channel" problem in the case of an existing busy cluster. https://github.com/nsqio/nsq/blob/master/nsqd/nsqd.go#L443

mreiferson commented 8 years ago

You can safely ignore these errors until the topic exists.

gm42 commented 6 years ago

Does this Go package offer an API for /topic/create? Ignoring errors is not a good practice, because it effectively hides the problem in case of misconfiguration.

ploxiln commented 6 years ago

There is no helper function in this package for /topic/create - you should just use http requests.