hitbtc-com / hitbtc-api

HitBTC API
GNU Lesser General Public License v3.0
322 stars 153 forks source link

Streaming Incomplete frame received #193

Closed Hydrorastaman closed 5 years ago

Hydrorastaman commented 6 years ago

Hello! I'm trying to integrate our software with your service and have an issue. When trying to subscribe to "subscribeTrades" and "subscribeOrderbook" streams for all of 825 instruments have received frequently disconnects and Poco library exception "Incomplete frame received". For connection used "wss://api.hitbtc.com/api/2/ws" endpoint:

const Context::Ptr context = new Context(Context::CLIENT_USE, "", "", "", Context::VERIFY_NONE);
HTTPSClientSession cs("api.hitbtc.com", 443, context);
HTTPRequest request(HTTPRequest::HTTP_GET, "/api/2/ws", HTTPMessage::HTTP_1_1);
HTTPResponse response;
response.setChunkedTransferEncoding(true);

websocket_.reset(new Poco::Net::WebSocket(cs, request, response));
reactor_.reset(new SocketReactor());
reactor_->addEventHandler(*websocket_.get(), NObserver<Connection, ReadableNotification>(*this, &Connection::onSocketReadable));
reactor_->addEventHandler(*websocket_.get(), NObserver<Connection, ShutdownNotification>(*this, &Connection::onSocketShutdown));
reactor_->addEventHandler(*websocket_.get(), NObserver<Connection, ErrorNotification>(*this, &Connection::onSocketError));

Subscription to streams:

    void Connection::Subscribe(MessageType type, std::string symbol) const {
        std::transform(std::begin(symbol), std::end(symbol), std::begin(symbol), ::toupper);

        std::string message;

        switch (type)
        {
            case MessageType::Trades:
                message = "{\"method\": \"subscribeTrades\", \"params\": {\"symbol\": \"" + symbol + "\"}}";
                break;

            case MessageType::Books:
                message = "{\"method\": \"subscribeOrderbook\", \"params\": {\"symbol\": \"" + symbol + "\"}}";
                break;

            default:
                return;
        }

        SendString(message);
    }

How I can handle changes for all instruments via streaming API?

Hydrorastaman commented 5 years ago

@hitbtc-com could you specify the status of the issue?

hitbtc-com commented 5 years ago

see example https://gist.github.com/hitbtc-com/fc738c1b926d9d7aa7e3bd5247f792a1 you should subscribe for next symbol after the previous response, otherwise, you can limit over network bandwidth

Hydrorastaman commented 5 years ago

see example https://gist.github.com/hitbtc-com/fc738c1b926d9d7aa7e3bd5247f792a1 you should subscribe for next symbol after the previous response, otherwise, you can limit over network bandwidth

The issue is not in the subscription process, it comes right after subscriptions. I successfully subscribed to all instruments (one connection or nine connections with 100 instruments per connection) and received quotes. And in 2-3 minutes I received "WebSocket Exception: Incomplete frame received" on socket read.

hitbtc-com commented 5 years ago

perhaps you are using the WebSocket library with errors. Can you test with other library?

Hydrorastaman commented 5 years ago

@hitbtc-com I've investigated the reason of the issue. int rc = SSL_read(_pSSL, buffer, length); returns zero and SSL_get_error(_pSSL, rc); returns SSL_ERROR_ZERO_RETURN which means that connection was terminated. OpenSSL version is # define OPENSSL_VERSION_TEXT "OpenSSL 1.0.1t 3 May 2016" . Should I try to check with the latest OpenSSL?

hitbtc-com commented 5 years ago

Be necessary to update for latest stable version. For your security, keep openssl library updated

Hydrorastaman commented 5 years ago

@hitbtc-com have updated OpenSSL to the latest, now does not receive incomplete frame. But receive PING frame with zero length. Have tried to send back PONG|FINAL frame or don't make any response. In any case receive CONTinuation frame w/o any data till reconnection. What response should be send on PING frame to the server?

hitbtc-com commented 5 years ago

Most of libraries automatically respond PONG frame