Can't extract data from ticker stream

[just-a-normal-human]

Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])

• [x] I have searched for other issues with the same problem or similar feature requests.

Select one:

• [x] Bug
• [ ] Feature Request
• [ ] Technical Help
• [ ] Other

Environment

• [ ] Are you using the module on a VPS or other Cloud hosting?
• [ ] If so, have you tried running the code on your local machine?
• [ ] Are you using the module on a Raspberry Pi?

Operating System? (include version)

• [x] macOS
• [ ] Windows
• [ ] Linux (include flavour)

Python Version Requirement

• [x] I am using Python 3.6.1 or above

Exact Python Version?

Python 3.8.3

Pip Version?

pip 20.2.2

Dependencies

Run pip list > pip_list.txt and upload the file.

UNICORN Binance WebSocket API Version?

1.16.7

I'm not sure if I'm misunderstanding something, fairly new to this, but I can't manage to extract the data from the stream. I've seen some people having problem with the 1m kline but I'm facing the same issue with ticker data. Let's say I'm looking to only get the symbol and price and print it. It seems to not matter how I approach this and the result will always be something in the lines of string indices must be integers I've tried both with and without unicorn_fy. Maybe I should post the issue at unicorn_fy instead?

I'm adding one example what I've tried to extract price and error traceback.

Thanks alot for this awesome library and for your help!

binance_websocket_api_manager = BinanceWebSocketApiManager(exchange="binance.com-futures")
binance_websocket_api_manager.create_stream(['ticker'], ['linkusdt', 'btcusdt'] )

while True:
    oldest_stream_data_from_stream_buffer = binance_websocket_api_manager.pop_stream_data_from_stream_buffer()
    if oldest_stream_data_from_stream_buffer:
        unicorn_fied_stream_data = UnicornFy.binance_com_websocket(oldest_stream_data_from_stream_buffer)
        stream_price_change = unicorn_fied_stream_data['data'][0]['price_change']
        print(stream_price_change)

ERROR:root:BinanceWebSocketApiConnection->__enter__(2ed883cb-7a51-4f6e-8707-f23538d57e0c, ['miniTicker'], ['linkusdt', 'btcusdt']) connecting to wss://fstream.binance.com/stream?streams=btcusdt@miniTicker error: 2 - string indices must be integers
ERROR:root:BinanceWebSocketApiConnection->__enter__(b9f3c8ae-532c-4f9a-b8d1-f7ba44d1bdeb, ['ticker'], ['linkusdt', 'btcusdt']) connecting to wss://fstream.binance.com/stream?streams=btcusdt@ticker error: 2 - string indices must be integers
Traceback (most recent call last):

  File "/Users/d/opt/anaconda3/envs/futures/lib/python3.8/site-packages/unicorn_fy/unicorn_fy.py", line 353, in binance_websocket
    for item in stream_data['items']:

KeyError: 'items'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/Users/d/Doc/Code/Python/SPYDER/Binance_Futures_python-master/bread/untitled6.py", line 85, in <module>
    unicorn_fied_stream_data = UnicornFy.binance_com_websocket(oldest_stream_data_from_stream_buffer)

  File "/Users/d/opt/anaconda3/envs/futures/lib/python3.8/site-packages/unicorn_fy/unicorn_fy.py", line 125, in binance_com_websocket
    return UnicornFy.binance_websocket(stream_data_json, exchange="binance.com", show_deprecated_warning=False)

  File "/Users/d/opt/anaconda3/envs/futures/lib/python3.8/site-packages/unicorn_fy/unicorn_fy.py", line 387, in binance_websocket
    'trade_before_24h_window': stream_data['data']['x'],

KeyError: 'x'

[just-a-normal-human]

A workaround is offcourse constructing a dataframe out of the stream and not using unicorn_fy, not sure how optimal this is tho (since it's making the code alot slower) . Here is what I did:

binance_websocket_api_manager = BinanceWebSocketApiManager(exchange="binance.com-futures")
binance_websocket_api_manager.create_stream(['ticker'], ['linkusdt', 'btcusdt'] )

import pandas as pd
from time import sleep

def ticker_data(stream):

   frame= pd.DataFrame(stream, columns=['stream','data'])

   try:
       frame=frame.drop(['e','p','P','w','Q','o','h','l','v','q','O','C','F','L','n'])
       frame=frame.drop(['stream'], 1)
       frame=frame.rename(index={'E':'Date', 'c':'Price', 's':'Symbol'})
       frame=frame.T
       frame['Date'] = pd.to_datetime(frame['Date'], unit='ms')
       frame = frame.reset_index(drop=True)
       frame.set_index('Symbol', inplace=True)

   except KeyError:
       pass           

   return frame

while True:
    oldest_stream_data_from_stream_buffer = binance_websocket_api_manager.pop_stream_data_from_stream_buffer()
    if oldest_stream_data_from_stream_buffer:
        #unicorn_fied_stream_data = UnicornFy.binance_com_websocket(oldest_stream_data_from_stream_buffer)

        new=json.loads(oldest_stream_data_from_stream_buffer)
        tick=ticker_data(new)
        print(tick)
        print ("\n")
        try:
            print(tick['Price'])
        except KeyError:
            pass
        sleep(7)
        binance_websocket_api_manager.stop_manager_with_all_streams()

Which gives an output like so:

                           Date     Price
Symbol                                   
BTCUSDT 2020-09-05 11:56:55.412  10231.09

Symbol
BTCUSDT    10231.09

                            Date   Price
Symbol                                  
LINKUSDT 2020-09-05 11:56:55.412  11.573

Symbol
LINKUSDT    11.573

[just-a-normal-human]

Okey my mistake, had to just load the stream with json and then wrap it with a try: except KeyError: statement. Guess I made it more complicated than it had to be 🙃

while True:
    oldest_tick_stream_data = binance_websocket_api_manager.pop_stream_data_from_stream_buffer(tick_stream)
    if oldest_tick_stream_data:

        json_d=json.loads(oldest_tick_stream_data)

        try:
            print(json_d['data']['c'])
        except KeyError:
            pass

[oliver-zehentleitner]

Hello!

Yes its actually very easy. You receive a string structured in JSON. You can convert json to a python dict.

I think this problem of unicorn_fy and not of this lib.

I know the bugs inside, but i dont have the time at the moment.

Just a hint, there are 2 versions of unicorn_fy available which have two different bugs (solving the one was causing the other) - you can try the other one if you still need it.

I close this topic in this repository.

Best regards, Oliver