using string :endpoint rather than :region results in NPE

[sirobinson]

Using [io.nervous/fink-nottle "0.2.0"]

If we define creds as follows: (def aws-creds (assoc-some {:region (env :aws-region)} :access-key (env :aws-access-key) :secret-key (env :aws-secret-key)))

where (env :aws-region) resolves as "eu-west-1"

Then requiring [fink-nottle.sqs :as sqs] and using (defn get-queue-url [queue-name](go %28sqs/create-queue! aws-creds queue-name%29)) Will successfully (create a queue and) get the url of queue.

If we define creds as (def aws-creds (assoc-some {:endpoint "https://sqs.eu-west-1.amazonaws.com/"} :access-key (env :aws-access-key) :secret-key (env :aws-secret-key)))

Then, on compilation, we get an error Caused by: java.lang.NullPointerException at eulalie.util$to_last_match.invoke(util.clj:88) at eulalie.sign_util$parse_region_name.invoke(sign_util.clj:60) at eulalie.sign_util$get_or_calc_region.invoke(sign_util.clj:68) at eulalie.sign$get_scope.invoke(sign.clj:29) at eulalie.sign$aws4_sign.invoke(sign.clj:97) at eulalie$eval23908$fn23910.invoke(eulalie.clj:37) at clojure.lang.MultiFn.invoke(MultiFn.java:229) at eulalie$issue_requestBANG$fn__23986$state_machine20651auto__23987$fn23989.invoke(eulalie.clj:83) at eulalie$issue_requestBANG$fn__23986$state_machine20651auto__23987.invoke(eulalie.clj:80) at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:940) at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:944) at eulalie$issue_requestBANG$fn__23986.invoke(eulalie.clj:80) at clojure.lang.AFn.run(AFn.java:22) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

Not sure if this is a problem with eulalie or fink?

[sirobinson]

Sorry - markdown mangled my fn. The get-queue-url fn looks like this:

(defn get-queue-url
  [queue-name]
  (sqs/create-queue! aws-creds queue-name))

[moea]

Simon,

Thanks for filing these. It's not written down anywhere, but unhelpfully, the version of Eulalie targeted by version 0.4.0 requires that :endpoint be an instance of cemerick.url/url (where cemerick.url is a transitive dependency of Fink-Nottle, via Eulalie). :endpoint wasn't initially intended to be user-facing, and I didn't clean up the interface after exposing it.

Does passing your string to url fix the issue?

FWIW, 1.0.0-SNAPSHOT of Fink-Nottle/Eulalie allow strings, though I don't recommend switching over yet unless you're feeling (more) adventurous - I've been holding off on an actual release until I finish the documentation.

[sirobinson]

Hi Moe

Thanks (again) for the prompt response :)

I've just kicked it around a bit and can get a cemerick url for what I'm assuming is the required endpoint (https://sqs.eu-west-1.amazonaws.com):

cemerick.url.URL{:protocol "https", :username nil, :password nil, :host "sqs.eu-west-1.amazonaws.com", :port -1, :path "", :query nil, :anchor nil}

...but trying to use it results in clojure.lang.ExceptionInfo: invalid-address {:type :invalid-address, :message "The address clojure.core.async.impl.channels.ManyToManyChannel@3f7035cb is not valid for this endpoint.

It's late though and I could be doing something wrong x/ Will look again in the morning.

[moea]

Ouch, that looks nasty. I'm also exhausted - I did try this though:

(sqs/create-queue!
  (assoc creds :endpoint 
               (cemerick.url/url "https://sqs.eu-west-1.amazonaws.com/")) 
  "endpoint-test")
;; => Channel containing 
;;     "https://sqs.eu-west-1.amazonaws.com/151963828411/endpoint-test"

With 0.2.0, which seems to work OK - something else is likely going on above.

Hazarding a wild guess, is it possible your create queue call is succeeding, and you're passing the unresolved channel it returns as the queue url to a subsequent API call (which is expecting a string)?

[sirobinson]

So, that works :)

[sirobinson]

Had been playing around with a few bits, trying to get things off channels etc and had changed the call to create-queue!
rather than
create-queue!!
8-/
Hazards of late-night programming! Thanks for your continued help here.
RE ":endpoint wasn't initially intended to be user-facing" - In case it's of interest, what I'm working towards is using fink-nottle against AWS for production service and also against a locally run SQS for acceptance-type testing (the lib I'm looking at requires that we can use endpoint rather than region as region wouldn't allow us to connect locally...).
I dare say I may have a couple more questions at some point!
Thanks again!

[moea]

Simon,

Cool. Please don't hesitate with questions - it's useful to get a sense of how the library is being used.

Are you using https://github.com/iain/fake_sqs or something else? The Dynamo tests use a local instance where they can, but wasn't sure which way to go with SNS/SQS.

[sirobinson]

Right now, am looking at elasticmq server (https://github.com/adamw/elasticmq) - will let you know how I get on!

[sirobinson]

So, that seems to work flawlessly!
I'm using this:
https://github.com/mdaley/lein-sqs
...which sets up an in-memory sqs using elasticmq. If you fancy giving it a go, it's as simple as adding

[lein-sqs "0.1.0"]  

to the plugins section of prooject.clj
and I added this in the env[iron] section:

        :sqs {:port ~(get (System/getenv) "SQS_PORT" "8084")
              :limits "relaxed"}

Then, for fink, I use http://localhost:8084/ as the endpoint to pass to cemerick.url and, bingo, local sqs for testing :)

[moea]

Simon,

Thanks, I'll give it ago for the next release!

I'll leave this issue open until 1.0.0 is deployed.

[sirobinson]

Cool froods

[moea]

Fixed in 0.4.0