mcohen01 / amazonica

A comprehensive Clojure client for the entire Amazon AWS api.
1k stars 202 forks source link

How to get the progress in uploading to an s3 bucket? #443

Open zendevil opened 3 years ago

zendevil commented 3 years ago

I using amazonica to put an object in a bucket like so:

(s3/put-object
           :endpoint "humboi-videos.s3-accelerate.amazonaws.com"
           :bucket-name "humboi-videos"
           :key thumbnail-name
           :file "./resources/public/thumbnail.png"
           :access-control-list {:grant-permission ["AllUsers" "Read"]})

What I want to know is how much progress has been made in putting the object in the bucket. For example if the total data is 1000 bytes, I want to know how many bytes has been uploaded to the s3 bucket. How can I achieve this?

mcohen01 commented 3 years ago

You want to use S3 Transfer for progress updates.

(require '[amazonica.aws.s3transfer :as s3t])

(-> (s3t/upload :bucket-name "humboi-videos"
                :key thumbnail-name"
                :file (java.io.File. "./resources/public/thumbnail.png")) 
    :add-progress-listener
    (apply [println]))
zendevil commented 3 years ago

Using this the following way:

(->
           (s3t/upload
           :endpoint "humboi-videos.s3-accelerate.amazonaws.com"
           :bucket-name "humboi-videos"
           :key filename
           :file (:tempfile (get multipart-params "video"))
           :access-control-list {:grant-permission ["AllUsers" "Read"]}
           :region-name "us-west-1")
           :add-progress-listener
           (apply [println]))

is giving me this exception:

java.lang.IllegalArgumentException: wrong number of arguments
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at amazonica.core$fn_call$fn__18886.invoke(core.clj:876)
    at amazonica.core$intern_function$fn__18926.doInvoke(core.clj:1032)
    at clojure.lang.RestFn.invoke(RestFn.java:1096)
    at humboiserver.routes.home$video_and_info_upload.invokeStatic(home.clj:50)
    at humboiserver.routes.home$video_and_info_upload.invoke(home.clj:37)
    at ring.middleware.multipart_params$wrap_multipart_params$fn__9126.invoke(multipart_params.clj:171)
    at muuntaja.middleware$wrap_params$fn__8281.invoke(middleware.clj:52)
    at muuntaja.middleware$wrap_format$fn__8285.invoke(middleware.clj:73)
    at humboiserver.middleware$wrap_formats$fn__9418.invoke(middleware.clj:38)
    at reitit.ring$ring_handler$fn__20931.invoke(ring.cljc:326)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.lang.AFunction$1.doInvoke(AFunction.java:31)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.lang.Var.invoke(Var.java:384)
    at ring.middleware.reload$wrap_reload$fn__4463.invoke(reload.clj:39)
    at selmer.middleware$wrap_error_page$fn__4478.invoke(middleware.clj:18)
    at prone.middleware$wrap_exceptions$fn__4719.invoke(middleware.clj:159)
    at ring.middleware.flash$wrap_flash$fn__8523.invoke(flash.clj:39)
    at ring.middleware.session$wrap_session$fn__8890.invoke(session.clj:108)
    at ring.middleware.keyword_params$wrap_keyword_params$fn__8936.invoke(keyword_params.clj:53)
    at ring.middleware.nested_params$wrap_nested_params$fn__8994.invoke(nested_params.clj:89)
    at ring.middleware.multipart_params$wrap_multipart_params$fn__9126.invoke(multipart_params.clj:171)
    at ring.middleware.params$wrap_params$fn__9150.invoke(params.clj:67)
    at ring.middleware.cookies$wrap_cookies$fn__8841.invoke(cookies.clj:214)
    at ring.middleware.absolute_redirects$wrap_absolute_redirects$fn__9338.invoke(absolute_redirects.clj:47)
    at ring.middleware.resource$wrap_resource_prefer_resources$fn__9186.invoke(resource.clj:25)
    at ring.middleware.content_type$wrap_content_type$fn__9286.invoke(content_type.clj:34)
    at ring.middleware.default_charset$wrap_default_charset$fn__9310.invoke(default_charset.clj:31)
    at ring.middleware.not_modified$wrap_not_modified$fn__9252.invoke(not_modified.clj:61)
    at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
    at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
    at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
    at humboiserver.middleware$wrap_internal_error$fn__9412.invoke(middleware.clj:17)
    at aleph.http.server$handle_request$fn__28538$f__22839__auto____28539.invoke(server.clj:158)
    at clojure.lang.AFn.run(AFn.java:22)
    at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
    at manifold.executor$thread_factory$reify__22721$f__22722.invoke(executor.clj:44)
    at clojure.lang.AFn.run(AFn.java:22)
    at java.lang.Thread.run(Thread.java:748)