search

ooyala / spark-jobserver

REST job server for Spark. Note that this is *not* the mainline open source version. For that, go to https://github.com/spark-jobserver/spark-jobserver. This fork now serves as a semi-private repo for Ooyala.
Other
344 stars 135 forks source link

Unable to upload fat jar. #119

Closed FK7 closed 8 years ago

FK7 commented 8 years ago

I am getting following error while uploading fat jar using curl command.

[2016-05-16 13:42:31,589] WARN erver.HttpServerConnection [] [akka.tcp://JobServer@127.0.0.1:42033/user/IO-HTTP/listener-0/2] - Illegal request, responding with status '413 Request Entity Too Large': Request Content-Length 38175252 exceeds the configured limit of 31457280

I've configured the parsing.max-content-lengthvalue to 100m in emr.conf

spray.can.server {
   # uncomment the next lines for making this an HTTPS example
   # ssl-encryption = on
   idle-timeout = 60 s
   request-timeout = 40 s
   pipelining-limit = 2 # for maximum performance (prevents StopReading / ResumeReading messages to the IOBridge)
   # Needed for HTTP/1.0 requests with missing Host headers
   default-host-header = "spray.io:8765"

   # Increase this in order to upload bigger job jars
   parsing.max-content-length = 100m
 }

And also is showing the correct value in logs 

    # /mnt/lib/spark-jobserver/emr.conf: 61
    "spray" : {
        # /mnt/lib/spark-jobserver/emr.conf: 61
        "can" : {
            # /mnt/lib/spark-jobserver/emr.conf: 61
            # check the reference.conf in spray-can/src/main/resources for all defined settings
            "server" : {
                # /mnt/lib/spark-jobserver/emr.conf: 68
                # Needed for HTTP/1.0 requests with missing Host headers
                "default-host-header" : "spray.io:8765",
                # /mnt/lib/spark-jobserver/emr.conf: 64
                # uncomment the next lines for making this an HTTPS example
                # ssl-encryption = on
                "idle-timeout" : "60 s",
                # /mnt/lib/spark-jobserver/emr.conf: 71
                "parsing" : {
                    # /mnt/lib/spark-jobserver/emr.conf: 71
                    # Increase this in order to upload bigger job jars
                    "max-content-length" : "100m"
                },
                # /mnt/lib/spark-jobserver/emr.conf: 66
                # for maximum performance (prevents StopReading / ResumeReading messages to the IOBridge)
                "pipelining-limit" : 2,
                # /mnt/lib/spark-jobserver/emr.conf: 65
                "request-timeout" : "40 s"
            }
        }

But still getting error, dont know what is happening. Kindly let me know am I missing something ?