bmoscon / cryptofeed

Cryptocurrency Exchange Websocket Data Feed Handler
Other
2.14k stars 668 forks source link

cryptofeed failed in case when 'time' is missing in ticker message from Coinbase #909

Closed dkozlov closed 1 year ago

dkozlov commented 1 year ago

Describe the bug cryptofeed failed in case when 'time' is missing in ticker message from Coinbase

To Reproduce

import os
import sys
from cryptofeed import FeedHandler
from cryptofeed.backends.influxdb import BookInflux, CandlesInflux, FundingInflux, TickerInflux, TradeInflux
from cryptofeed.defines import CANDLES, FUNDING, L1_BOOK, L2_BOOK, TICKER, TRADES
from cryptofeed.exchanges import Coinbase

f = FeedHandler()
    coinbase_pairs = ['HBAR-USDT','HBAR-USD']
    f.add_feed(Coinbase(channels=[TICKER], symbols=coinbase_pairs, callbacks={TICKER: TickerInflux(INFLUX_ADDR, ORG, BUCKET, TOKEN)}))
{'type': 'ticker', 'product_id': 'HBAR-USD', 'price': '0', 'open_24h': '0', 'volume_24h': '0', 'low_24h': '0', 'high_24h': '0', 'volume_30d': '0', 'best_bid': '0.00000', 'best_ask': '0.00000'}
2022-09-23 13:10:20,593 : ERROR : COINBASE.ws.9: encountered an exception, reconnecting in 1.0 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/cryptofeed/connection_handler.py", line 69, in _create_connection
    await self._handler(connection, self.handler)
  File "/usr/local/lib/python3.8/dist-packages/cryptofeed/connection_handler.py", line 119, in _handler
    await handler(message, connection, self.conn.last_message)
  File "/usr/local/lib/python3.8/dist-packages/cryptofeed/exchanges/coinbase.py", line 352, in message_handler
    await self._ticker(msg, timestamp)
  File "/usr/local/lib/python3.8/dist-packages/cryptofeed/exchanges/coinbase.py", line 109, in _ticker
    await self.callback(TICKER, Ticker(self.id, self.exchange_symbol_to_std_symbol(msg['product_id']), Decimal(msg['best_bid']), Decimal(msg['best_ask']), self.timestamp_normalize(msg['time']), raw=msg),
 timestamp)
KeyError: 'time'

Expected behavior I expected that missing 'time' in 'HBAR-USD' ticker message shouldn't affect cryptofeed runtime and this issue should be handled automatically or this ticker should be automatically skipped with warning.

Operating System:

Cryptofeed Version pypi https://github.com/bmoscon/cryptofeed/releases/tag/v2.3.0

bmoscon commented 1 year ago

fixed - thanks for raising the issue

dkozlov commented 1 year ago

@bmoscon where can I see the code changes related to this issue?

bmoscon commented 1 year ago

https://github.com/bmoscon/cryptofeed/commit/96acc3c2c2f28025ef477553d7f002bf300693d9

KamranBiglari commented 1 year ago

@bmoscon I do believe it's not really fixed. I'm using version 2.3.1 and sometimes for coinbase I'm getting empty timestamp.

python /cryptostore.py
2022-11-24 10:51:39 - exchange: COINBASE symbol: BTC-USD bid: 0.00 ask: 0.00 timestamp: None