csingley / ibflex

Python parser for Interactive Brokers Flex XML statements
MIT License
85 stars 42 forks source link

Struggling #40

Closed HawkridgeV closed 3 years ago

HawkridgeV commented 3 years ago

Hello,

I have jsut started using Python and am new to programming. I am struggling to make use of this project. Could anyone explain how to get started/set up this thing? I have the. token + an xml, however I have no idea what to do with makefile, setup, etc. I have tried: pythong ibflex build and install but the sample code does not work for me (its in Jypiter format and I am using pycharm.

Sorry for the beginer questions! Thanks a lot in advance.

csingley commented 3 years ago

Hey! The project README is intended to function as documentation. As it says there under the section entitled "Installation", you should be able just to type

pip install ibflex

assuming you have a recent version of Python and/or you've already installed pip.

If you don't want to use pip, or it doesn't work for you, ibflex has got all your old-school installation needs covered. Download & unzip the library (or git clone it if you're one of the cool kids), open a shell inside that directory, and type:

python setup.py install

If there are no errors with this process, you can verify the installation by opening a Python interpreter and typing:

import ibflex
print(ibflex.__file__)

That will report where the library got installed. Inside the directory, you'll find client.py, which can be run as a script, e.g. python client.py --help. For your convenience, this script actually should have been installed to your $PATH under the name flexget, so you can try just typing flexget --help and take it from there.

HTH

HawkridgeV commented 3 years ago

Hi Chris,

thank you very much for the detailed description, it looks like I did the right thing from the start. I had done pip install already ;(

I was simply not aware that I could ignore the build/makefile if pip worked. Now I have tried your client and that worked like a charm. I put my token + query ID and got the response. I saved it all with element tree as an xml directly:

token = '2800143wwer43099584' query_id = '534344'

root = ET.fromstring(response)

tree = ET.ElementTree(root)

tree.write("output3.xml")

I have attached the xml. Now I am struggling with your code for the parser. I have no clue how to use it. I have tried: response = parser.parse( 'IBFlexStatement.xml')

but the rest of your example does not work. (I think I am not getting something with the functions as I am a beginner) . I figured out that the parser puts the data in this structure and i can access it by:

print(response.FlexStatements) print(response.type) print(response.queryName)

but the rest is not working. the response structure does not have all the other fields I believe. Do you know what is going wrong? The response.Trade is completely empty ;(

Sorry for all the questions and thanks a lot in advance. Kind regards Vitus

Am Sa., 4. Sept. 2021 um 22:06 Uhr schrieb Christopher Singley < @.***>:

Hey! The project *README** https://github.com/csingley/ibflex is intended to function as documentation. As it says there under the section entitled "Installation", you should be able just to type

pip install ibflex

assuming you have a recent version of Python and/or you've already installed pip.

If you don't want to use pip, or it doesn't work for you, ibflex has got all your old-school installation needs covered. Download & unzip the library (or git clone it if you're one of the cool kids), open a shell inside that directory, and type:

python setup.py install

If there are no errors with this process, you can verify the installation by opening a Python interpreter and typing:

import ibflex print(ibflex.file)

That will report where the library got installed. Inside the directory, you'll find client.py, which can be run as a script, e.g. python client.py --help. For your convenience, this script actually should have been installed to your $PATH under the name flexget, so you can try just typing flexget --help and take it from there.

HTH

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/csingley/ibflex/issues/40#issuecomment-913033347, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGAS2QQSFE5IXQZAW4DFK6DUAJ34VANCNFSM5DNSRY5Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

csingley commented 3 years ago

I was simply not aware that I could ignore the build/makefile if pip worked.

If you open the Makefile in a text editor, I think you'll find it very easy to understand... it's mostly there as a convenience for developers (i.e. me) working on the library code. Users of the library don't need to bother with it; they can just stick to the usual Python installation practices (PyPI, pip, etc.)

I figured out that the parser puts the data in this structure and i can access it by:

print(response.FlexStatements) print(response.type) print(response.queryName)

Sound like it is working then, no? That's good!

It's possible that my example from the README docs is out of date; IBKR changes the data model quite frequently. The other main possibility is that you haven't configured your Flex Query to include the data that you want to look at.

Have a look at your XML file... I'd just open it up in a text editor and examine it closely. Does the output of ibflex.parser.parse() directly map to what's contained in your XML file?

As part of this process, I'd also recommend opening up ibflex.parser.Types in a text editor and looking at the source code. I have gone to quite a bit of trouble to make these data structures very easy to understand, modify, and extend by people who have no special experience.

At the top of the file, you'll see this:

class FlexQueryResponse(FlexElement):
    """ Root element """

    queryName: str
    type: str
    FlexStatements: Tuple["FlexStatement", ...]

So you should be able to do print(len(response.FlexStatements)) and stmt = response.FlexStatements[0]. Moving on to that class definition, you'll see this:

class FlexStatement(FlexElement):
    """ Wrapped in <FlexStatements> """

    accountId: str
    fromDate: datetime.date
    toDate: datetime.date
    period: str
    whenGenerated: datetime.datetime

    <snip>

    Trades: Tuple["Trade", ...] = ()

So you should be able to say print(len(stmt.Trades)) and trade = stmt.Trades[0] (if it's not empty). Then you can go look at the class definition for Trade, and keep on going. But again, if you haven't configured the Flex Query builder on IB's website to download trade data, it won't be there in the XML file for the parser to find.

If you get data in your XML file that is not showing up in the parser output, that is a bug!

HawkridgeV commented 3 years ago

Hi Chris,

thank you very much once again! It was my bad, I did not find the Trades in the Flexstatement and had no idea how to access it. Response.Flexstatement.0 obviously does not work.. With your little code I can now access all the trades in stmt! Now I can work on visualizing the data ;)

print(stmt.Trades[20])

For example spits out the whole thing for one trade:

Trade(transactionType=<TradeType.EXCHTRADE: 'ExchTrade'>, openCloseIndicator=None, buySell=None, orderType=None, assetCategory=<AssetClass.OPTION: 'OPT'>, accountId='U6asdf1', currency='USD', fxRateToBase=None, symbol='APA 210903P00017500', description='APA 03SEP21 17.5 P', conid=None, cusip=None, isin=None, listingExchange=None, multiplier=None, strike=Decimal('17.5'), expiry=datetime.date(2021, 9, 3), putCall=<PutCall.PUT: 'P'>, tradeID='174235', reportDate=None, tradeDate=None, tradeTime=None, settleDateTarget=None, exchange=None, quantity=Decimal('6'), tradePrice=None, tradeMoney=None, taxes=None, ibCommission=Decimal('-5.118'), ibCommissionCurrency='USD', netCash=Decimal('-23.118'), netCashInBase=None, closePrice=Decimal('0.035'), notes=(), cost=Decimal('23.118'), mtmPnl=None, origTradePrice=None, origTradeDate=None, origTradeID=None, origOrderID=None, openDateTime=None, fifoPnlRealized=Decimal('0'), capitalGainsPnl=None, levelOfDetail=None, ibOrderID=None, orderTime=datetime.datetime(2021, 8, 31, 15, 2, 30), changeInPrice=None, changeInQuantity=None, proceeds=None, fxPnl=None, clearingFirmID=None, transactionID=None, holdingPeriodDateTime=None, ibExecID=None, brokerageOrderID=None, orderReference=None, volatilityOrderLink=None, exchOrderId=None, extExecID=None, traderID=None, isAPIOrder=None, acctAlias=None, model=None, securityID=None, securityIDType=None, principalAdjustFactor=None, dateTime=datetime.datetime(2021, 8, 31, 15, 2, 30), underlyingConid=None, underlyingSecurityID='US03743Q1085', underlyingSymbol='APA', underlyingListingExchange=None, issuer=None, sedol=None, whenRealized=None, whenReopened=None, accruedInt=Decimal('0'))

I hope this is the correct result.

Kind regards Vitus

Am Mo., 6. Sept. 2021 um 00:50 Uhr schrieb Christopher Singley < @.***>:

I was simply not aware that I could ignore the build/makefile if pip worked.

If you open the Makefile in a text editor, I think you'll find it very easy to understand... it's mostly there as a convenience for developers (i.e. me) working on the library code. Users of the library don't need to bother with it; they can just stick to the usual Python installation practices (PyPI, pip, etc.)

I figured out that the parser puts the data in this structure and i can access it by:

print(response.FlexStatements) print(response.type) print(response.queryName)

Sound like it is working then, no? That's good!

It's possible that my example from the README docs is out of date; IBKR changes the data model quite frequently. The other main possibility is that you haven't configured your Flex Query to include the data that you want to look at.

Have a look at your XML file... I'd just open it up in a text editor and examine it closely. Does the output of ibflex.parser.parse() directly map to what's contained in your XML file?

As part of this process, I'd also recommend opening up ibflex.parser.Types in a text editor and looking at the source code. I have gone to quite a bit of trouble to make these data structures very easy to understand, modify, and extend by people who have no special experience.

At the top of the file, you'll see this:

class FlexQueryResponse(FlexElement): """ Root element """

queryName: str
type: str
FlexStatements: Tuple["FlexStatement", ...]

So you should be able to do print(len(response.FlexStatements)) and stmt = response.FlexStatements[0]. Moving on to that class definition, you'll see this:

class FlexStatement(FlexElement): """ Wrapped in """

accountId: str
fromDate: datetime.date
toDate: datetime.date
period: str
whenGenerated: datetime.datetime

<snip>

Trades: Tuple["Trade", ...] = ()

So you should be able to say print(len(stmt.Trades)) and trade = stmt.Trades[0] (if it's not empty). Then you can go look at the class definition for Trade, and keep on going. But again, if you haven't configured the Flex Query builder on IB's website to download trade data, it won't be there in the XML file for the parser to find.

If you get data in your XML file that is not showing up in the parser output, that is a bug!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/csingley/ibflex/issues/40#issuecomment-913240660, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGAS2QVW22FF3V36V3ZDHGLUAPX4LANCNFSM5DNSRY5Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

csingley commented 3 years ago

You've got it. You should be good to go.

BTW those fields listed inside <angle brackets> are enums; you'll find them in ibflex.enums.py if you need to handle different cases in your code.

Have fun!