Upgrading from 1.0.3 to 1.1.1 results in http 301 error

[ryanhall07]

We've been using the latest docker image, which was 1.0.3 until 4 days ago. Our CI build started failing when the latest image changed.

https://travis-ci.org/square/misk/builds/473409993

    com.amazonaws.services.sqs.model.AmazonSQSException: null (Service: AmazonSQS; Status Code: 301; Error Code: 301 Moved Permanently; Request ID: null)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
        at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1740)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1716)
        at com.amazonaws.services.sqs.AmazonSQSClient.executeSendMessage(AmazonSQSClient.java:1503)
        at com.amazonaws.services.sqs.AmazonSQSClient.sendMessage(AmazonSQSClient.java:1480)
        at misk.jobqueue.sqs.SqsJobQueue$enqueue$1.invoke(SqsJobQueue.kt:31)

Any ideas what might have broken this? Going to start digging into what changed in the latest release.

[p4tin]

@ryanhall07 Can you give me more details on the test that is failing? or are they all failing?

if you look at the release notes for 1.1.0 you'll find a list of things that changed between 1.0.3 and 1.1.1 - yes it's a very long list I was pretty busy the past few months. As much information as you can give me (maybe your java code) can help me troubleshoot this.

[ryanhall07]

@p4tin I was able to track down the issue to this commit https://github.com/p4tin/goaws/pull/183/files

This is a dump of the SendMessageRequest from the Intellij debugger

{QueueUrl: http://://my_queue,MessageBody: this is job 1,MessageAttributes: {},}

It should be noted that all tests and all calls to SQS (e.g ReceiveMessage) also fail with 301.

[p4tin]

@ryanhall07 - I think the issue you are facing is with the renaming in the queue URLs - try to add an entry in the '/etc/hosts' file as such: 127.0.0.1 us-east-1.goaws.com

See if that helps?

[slang25]

We have seen a similar thing: https://github.com/justeat/JustSaying/issues/494

[p4tin]

@slang25 See if what I proposed to @ryanhall07 works for you as well? If it does I will try to fix it tomorrow so it is backwards compatible with that old version

[slang25]

Thanks, that makes sense. I'm coding from a phone, will be at my laptop later: https://github.com/justeat/JustSaying/pull/496/files

[ryanhall07]

@p4tin changing /etc/hosts locally did not help.

As I continue to debug, the results fromGetQueueUrl changed from http://:/queue/my_queue to http://://my_queue for a queue named my_queue

[ryanhall07]

so it looks like app.CurrentEnvironment.AccountID is currently empty?

https://github.com/p4tin/goaws/pull/183/files#diff-a27de3fe2c38e3f55e788b0fc5a513b5

is that coming from an env variable we don't have set?

[ryanhall07]

so I'm not specifying a config file and it appears the default location is wrong, so AccountID resolves to the empty string.

https://github.com/p4tin/goaws/blob/master/app/conf/config.go#L22

I think that should be "./app/conf/goaws.yaml"

When I run it locally with a file it works

Local:
  AccountID: queue

Any other value for AccountID results in a 404 instead

[zanturik]

404 is given by router.go, aws sdk (at least for php) is trying to send POST data to full url, like "http://sqs.central-ca-1.goaws.local:4100/100010001000/test", but goaws doesn't have route for this kind of url. I added r.HandleFunc("/{account}/{queueName}", actionHandler).Methods("GET", "POST") to router/router.go as a workaround solution.

[p4tin]

I am working on a fix for goaws to make the router problem go away... Stil doing some tests to make sure and updating the unit tests should be out with version 1.1.2 by tomorrow.

[p4tin]

@zanturik + @ryanhall07 + @slang25 --> ok new version (1.1.2) is out (also new docker image) - please give it a try before I close this ticket.

[zanturik]

@p4tin works fine, thanks.

[ryanhall07]

@p4tin confirmed 1.1.2 fixes the issue. thanks for the quick fix!