search

mcohen01 / amazonica

A comprehensive Clojure client for the entire Amazon AWS api.
1k stars 202 forks source link

CloudSearchDomain examples? #237

Open acron0 opened 8 years ago

acron0 commented 8 years ago

The Java SDK is wrapped already. However, I'm pretty stumped as to how to use this :/ Would anyone be able to provide examples/give advice? I have looked at the SDK docs but I might as well be reading an alien dialect.

mcohen01 commented 8 years ago

It's going to look very similar to http://docs.aws.amazon.com/cli/latest/reference/cloudsearchdomain/index.html

acron0 commented 8 years ago

The error I get, regardless of my input, seems to be NullPointerException com.amazonaws.http.ExecutionContext.getSigner (ExecutionContext.java:135)

mcohen01 commented 8 years ago

Can you post your code and the full stacktrace?

acron0 commented 8 years ago

Code

(csd/upload-documents {:profile "my-profile"}
      :endpoint-url cs-doc-endpoint
      :content-type "application/json"
      :documents (io/input-stream "/path/to/data"))

ST

Unhandled java.lang.NullPointerException
   (No message)

     ExecutionContext.java:  135  com.amazonaws.http.ExecutionContext/getSigner
     AmazonHttpClient.java: 1636  com.amazonaws.http.AmazonHttpClient$ExecOneRequestParams/newSigner
     AmazonHttpClient.java:  841  com.amazonaws.http.AmazonHttpClient/executeOneRequest
     AmazonHttpClient.java:  723  com.amazonaws.http.AmazonHttpClient/executeHelper
     AmazonHttpClient.java:  475  com.amazonaws.http.AmazonHttpClient/doExecute
     AmazonHttpClient.java:  437  com.amazonaws.http.AmazonHttpClient/executeWithTimer
     AmazonHttpClient.java:  386  com.amazonaws.http.AmazonHttpClient/execute
AmazonCloudSearchDomainClient.java:  581  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/doInvoke
AmazonCloudSearchDomainClient.java:  551  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/invoke
AmazonCloudSearchDomainClient.java:  505  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/uploadDocuments
NativeMethodAccessorImpl.java:   -2  sun.reflect.NativeMethodAccessorImpl/invoke0
NativeMethodAccessorImpl.java:   62  sun.reflect.NativeMethodAccessorImpl/invoke
DelegatingMethodAccessorImpl.java:   43  sun.reflect.DelegatingMethodAccessorImpl/invoke
               Method.java:  498  java.lang.reflect.Method/invoke
                       nil:   -1  sun.reflect.GeneratedMethodAccessor20/invoke
DelegatingMethodAccessorImpl.java:   43  sun.reflect.DelegatingMethodAccessorImpl/invoke
               Method.java:  498  java.lang.reflect.Method/invoke
            Reflector.java:   93  clojure.lang.Reflector/invokeMatchingMethod
            Reflector.java:   28  clojure.lang.Reflector/invokeInstanceMethod
                  core.clj:  834  amazonica.core/fn-call/fn
                  core.clj:  905  amazonica.core/intern-function/fn
               RestFn.java:  421  clojure.lang.RestFn/invoke
mcohen01 commented 8 years ago

v0.3.75 is on clojars and the readme has been updated

acron0 commented 8 years ago

@mcohen01 This is superb, thank you!

acron0 commented 8 years ago

EDIT: Updated, still getting an Exception and would appreciate some guidance.

2. Unhandled com.amazonaws.AmazonClientException
   Unable to reset stream after calculating AWS4 signature

           AWS4Signer.java:  524  com.amazonaws.auth.AWS4Signer/calculateContentHash
           AWS4Signer.java:  195  com.amazonaws.auth.AWS4Signer/sign
     AmazonHttpClient.java:  856  com.amazonaws.http.AmazonHttpClient/executeOneRequest
     AmazonHttpClient.java:  723  com.amazonaws.http.AmazonHttpClient/executeHelper
     AmazonHttpClient.java:  475  com.amazonaws.http.AmazonHttpClient/doExecute
     AmazonHttpClient.java:  437  com.amazonaws.http.AmazonHttpClient/executeWithTimer
     AmazonHttpClient.java:  386  com.amazonaws.http.AmazonHttpClient/execute
AmazonCloudSearchDomainClient.java:  581  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/doInvoke
AmazonCloudSearchDomainClient.java:  551  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/invoke
AmazonCloudSearchDomainClient.java:  505  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/uploadDocuments
NativeMethodAccessorImpl.java:   -2  sun.reflect.NativeMethodAccessorImpl/invoke0
NativeMethodAccessorImpl.java:   62  sun.reflect.NativeMethodAccessorImpl/invoke
DelegatingMethodAccessorImpl.java:   43  sun.reflect.DelegatingMethodAccessorImpl/invoke
               Method.java:  498  java.lang.reflect.Method/invoke
                       nil:   -1  sun.reflect.GeneratedMethodAccessor10/invoke
DelegatingMethodAccessorImpl.java:   43  sun.reflect.DelegatingMethodAccessorImpl/invoke
               Method.java:  498  java.lang.reflect.Method/invoke
            Reflector.java:   93  clojure.lang.Reflector/invokeMatchingMethod
            Reflector.java:   28  clojure.lang.Reflector/invokeInstanceMethod
                  core.clj:  835  amazonica.core/fn-call/fn
                  core.clj:  906  amazonica.core/intern-function/fn
               RestFn.java:  457  clojure.lang.RestFn/invoke
                  core.clj:  108  sbsk.core/upload*
                  core.clj:  101  sbsk.core/upload*
                      REPL:   51  sbsk.core/eval24663
                      REPL:   51  sbsk.core/eval24663
             Compiler.java: 6927  clojure.lang.Compiler/eval
             Compiler.java: 6890  clojure.lang.Compiler/eval
                  core.clj: 3105  clojure.core/eval
                  core.clj: 3101  clojure.core/eval
                  main.clj:  240  clojure.main/repl/read-eval-print/fn
                  main.clj:  240  clojure.main/repl/read-eval-print
                  main.clj:  258  clojure.main/repl/fn
                  main.clj:  258  clojure.main/repl
                  main.clj:  174  clojure.main/repl
               RestFn.java:  137  clojure.lang.RestFn/applyTo
                  core.clj:  646  clojure.core/apply
                  core.clj:  641  clojure.core/apply
                regrow.clj:   18  refactor-nrepl.ns.slam.hound.regrow/wrap-clojure-repl/fn
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  clojure.tools.nrepl.middleware.interruptible-eval/evaluate/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  646  clojure.core/apply
                  core.clj: 1881  clojure.core/with-bindings*
                  core.clj: 1881  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   85  clojure.tools.nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   55  clojure.tools.nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  222  clojure.tools.nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
    interruptible_eval.clj:  190  clojure.tools.nrepl.middleware.interruptible-eval/run-next/fn
                  AFn.java:   22  clojure.lang.AFn/run
   ThreadPoolExecutor.java: 1142  java.util.concurrent.ThreadPoolExecutor/runWorker
   ThreadPoolExecutor.java:  617  java.util.concurrent.ThreadPoolExecutor$Worker/run
               Thread.java:  745  java.lang.Thread/run

1. Caused by java.io.IOException
   Resetting to invalid mark

  BufferedInputStream.java:  448  java.io.BufferedInputStream/reset
 SdkFilterInputStream.java:  102  com.amazonaws.internal.SdkFilterInputStream/reset
  ProgressInputStream.java:  139  com.amazonaws.event.ProgressInputStream/reset
 SdkFilterInputStream.java:  102  com.amazonaws.internal.SdkFilterInputStream/reset
           AWS4Signer.java:  522  com.amazonaws.auth.AWS4Signer/calculateContentHash
           AWS4Signer.java:  195  com.amazonaws.auth.AWS4Signer/sign
     AmazonHttpClient.java:  856  com.amazonaws.http.AmazonHttpClient/executeOneRequest
     AmazonHttpClient.java:  723  com.amazonaws.http.AmazonHttpClient/executeHelper
     AmazonHttpClient.java:  475  com.amazonaws.http.AmazonHttpClient/doExecute
     AmazonHttpClient.java:  437  com.amazonaws.http.AmazonHttpClient/executeWithTimer
     AmazonHttpClient.java:  386  com.amazonaws.http.AmazonHttpClient/execute
AmazonCloudSearchDomainClient.java:  581  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/doInvoke
AmazonCloudSearchDomainClient.java:  551  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/invoke
AmazonCloudSearchDomainClient.java:  505  com.amazonaws.services.cloudsearchdomain.AmazonCloudSearchDomainClient/uploadDocuments

Code

(defn upload*
  []
  (csd/set-endpoint doc-domain)
  (let [records (->> (fetch)
                     (map scrub))
        tf      (fs/temp-file "fbrecords")]
    (spit tf (generate-string records))
    (csd/upload-documents
     :content-type "application/json"
     :documents (io/input-stream tf))))