Open jreyeshdez opened 4 years ago
I'd call it a known limitation - but nsqadmin should probably handle it more gracefully.
The purpose of creating topics with nsqadmin, is to also be able to pre-create channels (these are all registered in nsqlookupd). When some service starts producing/publishing messages of that topic, it will send it to a pre-configured nsqd (or pool), often "localhost". That nsqd will create the topic locally, and pre-populate the channels with the list of existing channels from nsqlookupd. This way, if there will be multiple channels, messages won't be lost on channels other than the first to be created on an nsqd, by consumers when they connect. (Consumers typically use nsqlookupd to find all nsqd instances/servers which have a topic.)
nsqadmin expects that topics must exist on some nsqd ... and that's usually true ... but it does enable the case when it is briefly false, so that should be made more graceful.
@jreyeshdez can you share the logs from the other daemons running (e.g. nsqd
and nsqlookupd
)
@jreyeshdez can you share the logs from the other daemons running (e.g.
nsqd
andnsqlookupd
)
Sorry for the formatting. Edit (Splitting it into two comments)
@jreyeshdez can you share the logs from the other daemons running (e.g.
nsqd
andnsqlookupd
)
I was also able to reproduce the issue by setting up nsq via docker compose so you might get more valuable logs in there too.
You should connected to the nsqd to create topic
Any info on this one? I get this error as well. this happening when I change the port. when I use the default port I have no problem.
From @jreyeshdez's logs, this stands out (from nsqd
):
[nsqd] 2020/04/27 17:47:09.976307 INFO: CI: querying nsqlookupd http://nsqlookupd-01:9011/channels?topic=test
[nsqd] 2020/04/27 17:47:09.994447 WARNING: failed to query nsqlookupd for channels to pre-create for topic test - Failed to query any nsqlookupd: Get http://nsqlookupd-01:9011/channels?topic=test: dial tcp: lookup nsqlookupd-01 on 127.0.0.53:53: server misbehaving
Get http://nsqlookupd-02:9012/channels?topic=test: dial tcp: lookup nsqlookupd-02 on 127.0.0.53:53: server misbehaving
[nsqd] 2020/04/27 17:47:09.994526 INFO: TOPIC(test): new channel(test)
It looks like there's at least some DNS oddness going on (it's always DNS).
Also, the logs I'm looking for are from boot through manual topic creation, without consumers connecting.
I have a set up that starts
nsqlookupd, nsqd and nsqadmin
from an bash script that gets called from a cloud-init file in a EC2 instance.The whole setup works. I can browse to
http://ec2-public-ip.aws-region.compute.amazonaws.com:port/lookup
and create a Topic however when I click over the inactive topic I get the following error in the UIUPSTREAM_ERROR: Failed to query any nsqd:
and following error in the nsqadmin logs:This is part of the script that starts nsq:
The
$ThisPublicDnsName, $ThisPrivateDnsName, $ThisPrivateIpAddress
are retrieved by using aws cli ec2describe-instances
.If I create a consumer and connect to any of the tcp addresses for nsqd and subscribe to a topic then it works, however topic gets created as it does not exist and the previous
GET /api/topics/test
works in the UI where I can see the info about consumers/producers for that topic. If the topic got created via UI also works if connecting the consumer/producer. On the other hand if I try to create another topic viahttp://ec2-public-ip.aws-region.compute.amazonaws.com:port/lookup
then same problem occurs.From above script I am mapping the ip hostname (-broadcast-address) in
/etc/hosts
so not sure what I am missing.