search

s12v / sns

Fake Amazon SNS
https://hub.docker.com/r/s12v/sns/
Apache License 2.0
126 stars 40 forks source link

Docker SNS is not able to publish message to another docker instance via docker link #21

Closed kcjl519 closed 6 years ago

kcjl519 commented 7 years ago

I have a very basic docker-compose configuration, with 2 nodes. nodejs docker is linked via docker link as well as dependency

The following are the api calls I'm using to test.

Create new topic

aws sns --endpoint-url http://socialcrm.dev:9911 create-topic --name mrt_rules_
{
    "TopicArn": "arn:aws:sns:us-east-1:1485239739350:mrt_rules"
}

Export ARN into env

ARN=arn:aws:sns:us-east-1:1485239739350:mrt_rules ; export ARN

Subscribe nodejs to topic

aws sns --endpoint-url http://socialcrm.dev:9911 subscribe --topic-arn $ARN --protocol http --notification-endpoint http:mrt_rules:8080/sns
{
    "SubscriptionArn": "89540363-acaf-47f6-945b-e86bc03b94d1"
}

Post new message

aws sns --endpoint-url http://socialcrm.dev:9911 publish --topic-arn $ARN --message "New message"
{
    "MessageId": "c6e10fb4-6891-4522-bf2e-e4906091d783"
}

And I get the following errors in Akka (with debug turned on)

06:50:18.125 [sns-akka.actor.default-dispatcher-20] DEBUG akka.io.TcpListener - New connection accepted
06:50:18.132 [sns-akka.actor.default-dispatcher-14] WARN  akka.actor.ActorSystemImpl - Illegal request header: Illegal 'authorization' header: Invalid input '/', expected tchar, OWS, listSep or 'EOI' (line 1, column 49): AWS4-HMAC-SHA256 Credential=AKIAJRU7EX22W6MFQFAA/20170124/ap-southeast-2/sns/aws4_request, SignedHeaders=host;x-amz-date, Signature=a41a998abb3a6c7c25ec4cfd8b9b16aa12888e3497c4a620440b472dc74db3e5
                                                ^
06:50:18.142 [sns-akka.actor.default-dispatcher-15] DEBUG me.snov.sns.actor.SubscribeActor - Sending message c6e10fb4-6891-4522-bf2e-e4906091d783 to http:mrt_rules:8080/sns
06:50:18.143 [sns-akka.actor.default-dispatcher-15] DEBUG me.snov.sns.actor.SubscribeActor - Sending message c6e10fb4-6891-4522-bf2e-e4906091d783 to http:172.18.0.4:8080/sns
06:50:18.147 [sns-akka.actor.default-dispatcher-17] DEBUG akka.actor.ActorSystemImpl - akka-http-sns: Response for
  Request : HttpRequest(HttpMethod(POST),http://socialcrm.dev:9911/,List(Host: socialcrm.dev:9911, Accept-Encoding: identity, X-Amz-Date: 20170124T065018Z, authorization: AWS4-HMAC-SHA256 Credential=AKIAJRU7EX22W6MFQFAA/20170124/ap-southeast-2/sns/aws4_request, SignedHeaders=host;x-amz-date, Signature=a41a998abb3a6c7c25ec4cfd8b9b16aa12888e3497c4a620440b472dc74db3e5, User-Agent: aws-cli/1.10.25 Python/2.7.10 Darwin/16.3.0 botocore/1.5.2, Timeout-Access: <function1>),HttpEntity.Strict(application/x-www-form-urlencoded; charset=UTF-8,Action=Publish&Message=New+message&Version=2010-03-31&TopicArn=arn%3Aaws%3Asns%3Aus-east-1%3A1485239739350%3Amrt_rules),HttpProtocol(HTTP/1.1))
  Response: Complete(HttpResponse(200 OK,List(),HttpEntity.Strict(text/xml; charset=UTF-8,<PublishResponse xmlns="http://sns.amazonaws.com/doc/2010-03-31/"><PublishResult><MessageId>c6e10fb4-6891-4522-bf2e-e4906091d783</MessageId></PublishResult><ResponseMetadata><RequestId>a21f6deb-2f4a-4ca5-bc27-95c70ca6dd17</RequestId></ResponseMetadata></PublishResponse>),HttpProtocol(HTTP/1.1)))
Jan 24, 2017 6:50:18 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: **Connection refused (Connection refused)**
Jan 24, 2017 6:50:18 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Jan 24, 2017 6:50:18 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: **Connection refused (Connection refused)**
Jan 24, 2017 6:50:18 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Jan 24, 2017 6:50:18 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: **Connection refused (Connection refused)**
Jan 24, 2017 6:50:18 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
06:50:18.250 [sns-akka.actor.default-dispatcher-20] ERROR akka.actor.OneForOneStrategy - Connection refused (Connection refused)
akka.camel.AkkaCamelException: Connection refused (Connection refused)
    at akka.camel.ProducerSupport$$anonfun$produce$1.applyOrElse(Producer.scala:73) ~[sns.jar:0.1.3]
    at akka.actor.Actor$class.aroundReceive(Actor.scala:484) ~[sns.jar:0.1.3]
    at me.snov.sns.actor.ProducerActor.aroundReceive(ProducerActor.scala:10) ~[sns.jar:0.1.3]
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) [sns.jar:0.1.3]
    at akka.actor.ActorCell.invoke(ActorCell.scala:495) [sns.jar:0.1.3]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [sns.jar:0.1.3]
    at akka.dispatch.Mailbox.run(Mailbox.scala:224) [sns.jar:0.1.3]
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [sns.jar:0.1.3]
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [sns.jar:0.1.3]
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [sns.jar:0.1.3]
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [sns.jar:0.1.3]
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [sns.jar:0.1.3]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_111-internal]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_111-internal]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_111-internal]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_111-internal]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_111-internal]
    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_111-internal]
    at java.net.Socket.connect(Socket.java:538) ~[na:1.8.0_111-internal]
    at java.net.Socket.<init>(Socket.java:434) ~[na:1.8.0_111-internal]
    at java.net.Socket.<init>(Socket.java:286) ~[na:1.8.0_111-internal]
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) ~[sns.jar:0.1.3]
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) ~[sns.jar:0.1.3]
    at org.apache.camel.component.http.HttpProducer.executeMethod(HttpProducer.java:253) ~[sns.jar:0.1.3]
    at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:148) ~[sns.jar:0.1.3]
    at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) ~[sns.jar:0.1.3]
    at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:145) ~[sns.jar:0.1.3]
    at akka.camel.ProducerSupport$ProducerChild.produce(Producer.scala:137) ~[sns.jar:0.1.3]
    at akka.camel.ProducerSupport$ProducerChild$$anonfun$receive$1.applyOrElse(Producer.scala:111) ~[sns.jar:0.1.3]
    at akka.actor.Actor$class.aroundReceive(Actor.scala:484) ~[sns.jar:0.1.3]
    at akka.camel.ProducerSupport$ProducerChild.aroundReceive(Producer.scala:108) ~[sns.jar:0.1.3]
    ... 9 common frames omitted

I have manually subscribed the nodejs's IP, and it published message fine, I have also subscribed google.com to it, and it published fine. It seems Akka is unable to resolve the DNS of the docker link for nodejs.

A-Scott-Rowe commented 7 years ago

@kcjl519 Did you ever resolve this issue? I am experiencing the same error using 2 rails apps and propono.

s12v commented 7 years ago

@A-Scott-Rowe, no, I was stuck actually. It was working a year ago and now it doesn't. I can't see why.

Could you give more details, please? Do you use the same docker-compose configuration?

A-Scott-Rowe commented 7 years ago

@s12v I think my situation was actually a little different. I was using this in a docker container and also a fake sqs in another docker container. The two services worked great independently but when sns tried to connect to sqs to push the message it received a similar(but not the same) error as above.

I was using the same config as in your example but could not resolve it. I ended up finding an all-in-one docker image that works very well. https://github.com/p4tin/GoAws

scott-r-lindsey commented 6 years ago

@s12v is this project abandoned? I'm having the problem above and I don't know how to debug this. I have verified that my ElasticMQ container is accessible from my s12v/sns container on the correct port, but I'm still getting the "Connection refused" error message above.

Any hints on how to debug would be greatly appreciated.

s12v commented 6 years ago

@scott-r-lindsey, it's not abandoned, but I couldn't find the root cause of this problem and didn't have time to look into this again.

sam-io commented 6 years ago

@s12v I can reproduce this issue using the example docker-compose file that is include in your project. The problem appears to be that when the camel component that is used to send the message to sqs starts up it lists the queues to get their urls. If elasticmq has not been configured with a host name that matches the docker link then it will try to send the message to the wrong url.

I have submitted a pull request that shows the fix.

s12v commented 6 years ago

this should be solved now, thanks @sam-io