search

Petersoj / alpaca-java

A Java API for Alpaca, the commission free, algo friendly, stock trading broker. https://alpaca.markets
https://petersoj.github.io/alpaca-java/
MIT License
197 stars 82 forks source link

Broken pipe in PolygonWebsocketClientEndpoint #61

Closed liulinglll closed 3 years ago

liulinglll commented 4 years ago

got "broken pipe" error as below. Seems like this only happens when subscribing a lot of symbols - like SP100.

2020-05-04 13:28:58.808:WARN:oejwja.JsrEvents:WebSocketClient@2092801316-67: Unable to report throwable to websocket (no @OnError handler declared): io.github.mainstringargs.polygon.websocket.client.PolygonWebsocketClientEndpoint java.io.IOException: Broken pipe at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.flush(SslConnection.java:1089) at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422) at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277) at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381) at org.eclipse.jetty.websocket.common.io.FrameFlusher.flush(FrameFlusher.java:264) at org.eclipse.jetty.websocket.common.io.FrameFlusher.process(FrameFlusher.java:193) at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241) at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223) at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.outgoingFrame(AbstractWebSocketConnection.java:584) at org.eclipse.jetty.websocket.client.io.WebSocketClientConnection.outgoingFrame(WebSocketClientConnection.java:58) at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.close(AbstractWebSocketConnection.java:181) at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:511) at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:441) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang.Thread.run(Thread.java:748)

mainstringargs commented 4 years ago

Hello -- can you provide which version of alpaca-java you're using and the version of the JRE/JDK? Can you give more examples of tickers that you're seeing this with?

liulinglll commented 4 years ago

i am using alpaca-java 5.0.12 and JDK 1.8.

tickers are sp100 - ABBV ABT ACN ADBE AGN AIG ALL AMGN AMZN BAC BIIB BK BKNG BLK BMY BRK.B C CHTR CL CMCSA COF COP COST CVS DD DHR DUK EMR EXC F FB FDX GD GE GILD GM GOOG GOOGL HON KHC KMI LLY LMT LOW MA MDLZ MDT MET MO MS NEE NFLX NVDA ORCL OXY PEP PM PYPL QCOM RTN SBUX SLB SO SPG T TGT TMO TXN UNP UPS USB WFC AAPL AXP BA CAT CSCO CVX DIS DOW GS HD IBM INTC JNJ JPM KO MCD MMM MRK MSFT NKE PFE PG TRV UNH UTX V VZ WBA WMT XOM

mainstringargs commented 4 years ago

I think a lot of issues you're seeing have to do with a problem from Polygon.io, they have been seemingly randomly changing the i field on their Trade message between integer and String. This is wreaking havoc on our schemas. See some discussion here: https://alpaca-community.slack.com/archives/CKKL6MP7H/p1585675561106600. We are waiting for conformation from them to when they will decide which is the preferred way or if we need to handle it on our end.

mainstringargs commented 4 years ago

Have you heard/seen any updates on the 'i' field, @Petersoj?

Petersoj commented 4 years ago

As @liulinglll mentioned here, the problem seemed to occur strictly on the Polygon side of things. We currently don't have the OnError WS method implemented yet, but I'll look into it this weekend. @liulinglll If you can do a few more tests and let us know what you find tomorrow, that'd be great (maybe try listening to less channels?). As far as the 'i' field goes in trade schemas for the "T.*" channel, I have not heard anything yet @mainstringargs

mainstringargs commented 4 years ago

Ah ok. I was going down the 'i' field route because I tried all of those tickers for SP100, let it run on each for a bit and the only errors I was getting had to do with that field -- I did not see the 'java.io.IOException: Broken pipe' issue.

Petersoj commented 3 years ago

If this keeps happening, please open a new issue.