search

LUCIT-Systems-and-Development / unicorn-binance-websocket-api

A Python SDK by LUCIT to use the Binance Websocket API`s (com+testnet, com-margin+testnet, com-isolated_margin+testnet, com-futures+testnet, com-coin_futures, us, tr, dex/chain+testnet) in a simple, fast, flexible, robust and fully-featured way.
https://unicorn-binance-websocket-api.docs.lucit.tech/
Other
692 stars 164 forks source link

Refactoring manager for ease testing #305

Closed dima-dmytruk23 closed 1 year ago

dima-dmytruk23 commented 1 year ago

PR Details

I made a more flexible manager constructor so that you can replace the exchange with your own mock server.

Description

To replace the exchange with your mock server, just specify exchange=Exchanges.LOCALHOST, exchange_type='dex' (or cex) when initializing the manager. exchange_type must be specified only for Localhost. You can also set websocket_base_uri and max_subscriptions_per_stream otherwise the default values ​​will be applied.

Related Issue

https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/304

Motivation and Context

With the previous implementation - replacing the server with your own mock - was unreal.

How Has This Been Tested

Tested on localhost (MacOS) and in docker. I worked with a real exchange and with my own mock server.

Types of changes

Checklist

oliver-zehentleitner commented 1 year ago

Hello!

First of all thanks for the PR, the code looks very neat and valid.

Is it support for Apache MockServer? https://www.mock-server.com/

I didn't know that, I do similar with a SSH redirect: https://www.ssh.com/academy/ssh/tunneling-example

Could you provide more information for other users on how to use it with this lib?

What I noticed, REST is not sent to the proxy: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/unicorn_binance_websocket_api/restclient.py

Thank you!

dima-dmytruk23 commented 1 year ago

Привет!

Прежде всего спасибо за пиар, код выглядит очень аккуратно и валидно.

Это поддержка Apache MockServer? https://www.mock-server.com/

Я этого не знал, я делаю то же самое с перенаправлением SSH: https://www.ssh.com/academy/ssh/tunneling-example

Не могли бы вы предоставить дополнительную информацию для других пользователей о том, как использовать ее с этой библиотекой?

Что я заметил, REST не отправляется на прокси: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/unicorn_binance_websocket_api/restclient.py

Спасибо!

@oliver-zehentleitner

Initially, I tried to raise JMeter, but there were difficulties, and I decided to write my own simple websocket server. For example, here it sends 1000 messages in ~1 second.

import asyncio
import json
import logging
import random
from datetime import datetime

import websockets
from binance import Client

logging.basicConfig(level=logging.INFO,
                    format="{asctime} {msecs} | {levelname} | {process} | {thread} | {module} | {filename}:{lineno} "
                           "| {message}",
                    style="{",
                    datefmt='%Y-%m-%d %H:%M:%S')
logger = logging.getLogger(__name__)

WS_HOST = 'localhost'
WS_PORT = 8765
MESSAGES_COUNT = 1000
MESSAGE_TIMING = 0.00063

def get_symbols():
    binance_client = Client()
    exchange_info = binance_client.futures_exchange_info()
    return [symbol_info['symbol'] for symbol_info in exchange_info['symbols']]

async def get_message():
    symbol = random.choice(symbols)
    return json.dumps({
        "data": {
            "e": "kline",
            "E": datetime.now().timestamp() * 1000000,
            "s": symbol,
            "k": {
                "t": 1638747660000,
                "T": 1638747719999,
                "s": symbol,
                "i": "1m",
                "f": 100,
                "L": 200,
                "o": "0.0010",
                "c": "0.0020",
                "h": "0.0025",
                "l": "0.0015",
                "v": "1000",
                "n": 100,
                "x": False,
                "q": "1.0000",
                "V": "500",
                "Q": "0.500",
                "B": "123456",
            }
        }
    })

async def handler(websocket, path):
    await websocket.recv()

    tm = datetime.now()
    for i in range(MESSAGES_COUNT):
        message = await get_message()
        await websocket.send(message)
        logger.info(f"{message=}")
        await asyncio.sleep(MESSAGE_TIMING)

    logger.info((datetime.now() - tm).total_seconds())
    return

def start_ws_server():
    start_server = websockets.serve(handler, WS_HOST, WS_PORT)
    asyncio.get_event_loop().run_until_complete(start_server)
    asyncio.get_event_loop().run_forever()

if __name__ == '__main__':
    symbols = get_symbols()
    start_ws_server()
oliver-zehentleitner commented 1 year ago

Is it possible to add support for REST API in this package too?

This would be needed to run userData streams!

https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/unicorn_binance_websocket_api/restclient.py

dima-dmytruk23 commented 1 year ago

Is it possible to add support for REST API in this package too?

This would be needed to run userData streams!

https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/unicorn_binance_websocket_api/restclient.py

Yes, sure. You can do it.

I just limited myself to web sockets in my task. The REST did not look. If you have the desire and opportunity - you can change the PR and tag me. I won't have time next week.

oliver-zehentleitner commented 1 year ago

I am also full of tasks :/ i started preparing a new update for full UBS packages...

I would appreciate if we could add this to the next release but from my point of view we have to do:

I can do unittests and docs, because i want to learn and understand both.

Could you do the REST implementation?

dima-dmytruk23 commented 1 year ago

I am also full of tasks :/ i started preparing a new update for full UBS packages...

I would appreciate if we could add this to the next release but from my point of view we have to do:

  • Documenation: what is this implementation, how can it be used
  • Unittests: Thanks for the script above, this should help to test. I general we can apply a new kind of testing now because we can test against our self via localhost...
  • Adding REST support, i dont want merge half implementations...

I can do unittests and docs, because i want to learn and understand both.

Could you do the REST implementation?

Ok. Let me try on the weekend to implement for REST. Will you do documentation and tests?

oliver-zehentleitner commented 1 year ago

deal! i will do my part afterwards when this is merged.

oliver-zehentleitner commented 1 year ago

f"{self.exchange_type=}"

Awesome, did not know that :)

Is it only possible to use localhost with the hardcoded port or can this be overwritten?

dima-dmytruk23 commented 1 year ago

f"{self.exchange_type=}"

Awesome, did not know that :)

Yeah. It's from python3.8

Is it only possible to use localhost with the hardcoded port or can this be overwritten?

Hardcoded. To flexibly set the port, you need to rewrite it here (separate the key with the URI separately into host and port (tuple/dictionary)) and process it in the constructor of manager.

https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/pull/305/files#diff-540fe6a75df813cec7157aacac6de4268fd66a84f6bd3b2daaf7310f4202277eR43

oliver-zehentleitner commented 1 year ago

Thanks , I will try when this PR is ready

oliver-zehentleitner commented 1 year ago

I will merge it, we also add socks server support and your code is usefull. thanks!