erdewit / ib_insync

Python sync/async framework for Interactive Brokers API
BSD 2-Clause "Simplified" License
2.8k stars 744 forks source link

ib_insync\decoder.py error when downloading LME metal prices #239

Closed An-naili closed 4 years ago

An-naili commented 4 years ago

Hi all,

Starting yesterday (before all worked fine), I get the following decoder problem when downloading LME metal prices from IB:

Traceback (most recent call last): File "C:\Users\Natalia Vasileva\AppData\Local\Programs\Python\Python37-32\lib\site-packages\ib_insync\decoder.py", line 189, in interpret handler(fields) File "C:\Users\Natalia Vasileva\AppData\Local\Programs\Python\Python37-32\lib\site-packages\ib_insync\decoder.py", line 413, in historicalData open=float(get()), ValueError: could not convert string to float: '?'

That's parts of error handling fields:

Error handling fields: ['17', '3', '20180423 14:00:22', '20200423 14:00:22', '507', '20180424', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180425', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180426', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180427', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180430', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180501', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180502', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180503', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180504', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180508', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180509', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180510', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180511', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180514', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180515', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180516', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180517', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180518', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180521', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180522', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180523', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180524', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180525', '?', '?', '?', '?', '-1', '-1.00', '-1', '20180529', '?', '?', '?', '?', '-1', '-1.00', '-1',

That's my code for reference:


"""
Import IB data
"""

from ib_insync import *

import csv
import pandas as pd
from pathlib import Path
from datetime import datetime, timedelta

ib = IB()
ib.connect('127.0.0.1', 7497, clientId=0)

contract2 = Contract()
contract2.symbol = 'CA'
contract2.secType = 'CONTFUT'
contract2.exchange = 'LMEOTC'

bars = ib.reqHistoricalData(contract2, endDateTime='', durationStr='2 Y',
    barSizeSetting='1 day', whatToShow='MIDPOINT', useRTH=True)

# convert to pandas dataframe:
dfLME = util.df(bars)
dfLME = dfLME[['date', 'close']]
LMEcolumns = ['Date', 'LME Cu 3m']
dfLME.columns = LMEcolumns

for symbol in ('AH', 'ZSLME', 'NI', 'PB', 'SNLME'):
    contract = Contract()
    contract.symbol = symbol
    contract.secType = 'CONTFUT'
    contract.exchange = 'LMEOTC'

    bars = ib.reqHistoricalData(contract, endDateTime='', durationStr='2 Y',
        barSizeSetting='1 day', whatToShow='MIDPOINT', useRTH=True)
    lme = util.df(bars)
    lme = lme[['date', 'close']]
    lmecolumns = ['Date', symbol]
    lme.columns = lmecolumns
    dfLME = pd.merge(dfLME, lme, how='left', left_on='Date', right_on='Date')

LMEcolumns = ['Date', 'LME Cu 3m', 'LME Al 1m', 'LME Zn 1m', 'LME Ni 1m', 'LME Lead 1m', 'LME Tin 1m']

dfLME.columns = LMEcolumns
dfLME = dfLME.set_index('Date')

dfLME = dfLME.ffill()
erdewit commented 4 years ago

The prices are sent as question marks instead of numbers. This clearly is very wrong and the current behaviour of spitting out an error message seems appropriate to me. This as opposed to silently converting the question marks to NaN values and ending up with a spreadsheet filled with NaNs.

It's probably a temporary IB data server issue, if it persists then contact IB.