search

databricks / sbt-databricks

An sbt plugin for deploying code to Databricks Cloud
http://go.databricks.com/register-for-dbc
Other
71 stars 27 forks source link

`dbcDeploy` Fails with java.net.SocketException: Broken pipe #30

Closed EvanOman closed 8 years ago

EvanOman commented 8 years ago

All,

I am using sbt-databricks and I am running into a broken pipe error when I run dbcDeploy. I can do a dbcListClusters just fine but the deploy command is not working.

Any thoughts?

Full output:

...
Uploading <package-name>.jar
Apr 20, 2016 11:16:17 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://<instance-name>.cloud.databricks.com:443: Broken pipe
Apr 20, 2016 11:16:17 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://<instance-name>.cloud.databricks.com:443
Apr 20, 2016 11:16:17 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://<instance-name>.cloud.databricks.com:443: Broken pipe
Apr 20, 2016 11:16:17 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://<instance-name>.cloud.databricks.com:443
Apr 20, 2016 11:16:17 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://<instance-name>.cloud.databricks.com:443: Broken pipe
Apr 20, 2016 11:16:17 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://<instance-name>.cloud.databricks.com:443
[trace] Stack trace suppressed: run last *:dbcDeploy for the full output.
[error] (*:dbcDeploy) java.net.SocketException: Broken pipe
[error] Total time: 11 s, completed Apr 20, 2016 11:16:18 PM
> last *:dbcDeploy
java.net.SocketException: Broken pipe
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
        at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
        at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
        at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:876)
        at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:847)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
        at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:123)
        at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:135)
        at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:164)
        at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:115)
        at org.apache.http.entity.mime.content.FileBody.writeTo(FileBody.java:120)
        at org.apache.http.entity.mime.AbstractMultipartForm.doWriteTo(AbstractMultipartForm.java:150)
        at org.apache.http.entity.mime.AbstractMultipartForm.writeTo(AbstractMultipartForm.java:173)
        at org.apache.http.entity.mime.MultipartFormEntity.writeTo(MultipartFormEntity.java:97)
        at org.apache.http.entity.mime.MultipartEntity.writeTo(MultipartEntity.java:171)
        at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:155)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
        at com.sun.proxy.$Proxy4.sendRequestEntity(Unknown Source)
        at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:236)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
        at sbtdatabricks.DatabricksHttp.uploadJar(DatabricksHttp.scala:76)
        at sbtdatabricks.DatabricksPlugin$$anonfun$8.apply(DatabricksPlugin.scala:151)
        at sbtdatabricks.DatabricksPlugin$$anonfun$8.apply(DatabricksPlugin.scala:150)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.immutable.Set$Set1.foreach(Set.scala:74)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
        at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
        at scala.collection.SetLike$class.map(SetLike.scala:93)
        at scala.collection.AbstractSet.map(Set.scala:47)
        at sbtdatabricks.DatabricksPlugin$.sbtdatabricks$DatabricksPlugin$$uploadImpl1(DatabricksPlugin.scala:150)
        at sbtdatabricks.DatabricksPlugin$$anonfun$deployImpl$1$$anonfun$apply$6$$anonfun$apply$7.apply(DatabricksPlugin.scala:194)
        at sbtdatabricks.DatabricksPlugin$$anonfun$deployImpl$1$$anonfun$apply$6$$anonfun$apply$7.apply(DatabricksPlugin.scala:192)
        at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
        at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
        at sbt.std.Transform$$anon$4.work(System.scala:63)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
        at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
        at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
        at sbt.Execute.work(Execute.scala:237)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
        at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
        at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
[error] (*:dbcDeploy) java.net.SocketException: Broken pipe
brkyvz commented 8 years ago

@EvanOman I'm looking into the issue. I will let you know when the issue should be resolved.

brkyvz commented 8 years ago

@EvanOman Which Java version do you use? I'm planning to upgrade the http client to Jetty 9, but that requires Java 8. Is that okay?

EvanOman commented 8 years ago

I am using Java 8 on my machine so that is no problem for me.

Thanks for looking into this!

brkyvz commented 8 years ago

Released version 0.1.5 with the fix. Just to be clear, the fix was simpler and does not require Java 8.