search

pytr-org / pytr

Use TradeRepublic in terminal and mass download all documents
https://pypi.org/project/pytr/
MIT License
422 stars 74 forks source link

addtional variables from extracted-information in events for file name #114

Open b166er opened 2 months ago

b166er commented 2 months ago

Is your feature request related to a problem? Please describe. For some events, such as the purchase of bonds, options and several simultaneous notifications with customer letters, the company name is not specified in the event_title or subtitle. The only way to get the company name of the underlying product is to extract the ISIN from the “icon” field or from specific section-block. Here are a few examples where no company name is returned by the API, except for the ISIN in the icon field.

  1. multiple company information in a single api response.json
  2. Share gift.json
  3. sell Bond.json
  4. buy Option contract.json in this api answer is a special situation, when buying an option there are two ISIN values, that of the option itself and that of the underlying product. The option has no real name other than the action and the underlying value, for example: “Call 80€” or “Put 120€”. I think the underlying company is also important, so we need the ISIN from the section block.

Describe the solution you'd like My proposed solution is as follows. Introduce new variables (naming are just for example): {event_isin} - this is a parsed value from event_icon field. {event_companyname} - this is a retrieved name from parsed isin

For option purchases you also need additional information from the section block. The two pieces of information are almost always identical to the two variables above. The only difference comes with bond and option purchases, where you need the underlying companies, which are stored in this section block: {section_action_isin} - parsed from section_action_payload field {section_action_companyname} - translated from above isin

Here are a few example scenarios with input and output where the variables can be useful. _

  1. multiple company informations.

pattern block (before):

  notice_multiple_documents:
    pattern: [{event_type: "GESH_CORPORATE_ACTION_MULTIPLE_POSITIONS"}] #event_subtitle: Gesellschaftshinweis
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {event_subtitle} - {event_title} - {document_title}"

pattern block (after):

  notice_multiple_documents:
    pattern: [{event_type: "GESH_CORPORATE_ACTION_MULTIPLE_POSITIONS"}] #event_subtitle: Gesellschaftshinweis
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {event_subtitle} - {event_companyname} - {document_title}"

result file name (before): Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 1.pdf Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 2.pdf

result file name (after): Gesellschaftshinweis - Company AAA - Kundenanschreiben 1.pdf Gesellschaftshinweis - Company AAA - Kundenanschreiben 2.pdf

_

  1. Share gift

pattern line (before):

    pattern: [{event_type: "STOCK_PERK_REFUNDED",  document_title: "Abrechnung(.\\d+)?"}] # Aktiengeschenk
    filename: "{iso_date}.{iso_time} {event_title}"

pattern line (after):

    pattern: [{event_type: "STOCK_PERK_REFUNDED",  document_title: "Abrechnung(.\\d+)?"}] # Aktiengeschenk
    filename: "{iso_date}.{iso_time} {event_title} - {event_companyname}"

result file name (before): 2020-01-02.34-56 Aktiengeschenk.pdf

result file name (after): 2020-01-02.34-56 Aktiengeschenk - Company BBB.pdf

_

  1. trading Bonds

pattern line (before):

  bond_repayment:
    pattern: [{event_type: "REPAYMENT"}]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Repayment {event_title}"

  bond_interest:
    pattern: [{event_type: "COUPON_PAYMENT"}]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Interest {event_title}"

pattern line (after):

  bond_repayment:
    pattern: [{event_type: "REPAYMENT"}]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Repayment {event_title} - {event_companyname}"

  bond_interest:
    pattern: [{event_type: "COUPON_PAYMENT"}]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Interest {event_title} - {event_companyname}"

result file name (before): 2020-01-02.34-56 Repayment Anleihe Feb. 2024.pdf 2020-01-02.34-56 Interest Anleihe Feb. 2024.pdf

result file name (before): 2020-01-02.34-56 Repayment Anleihe Feb. 2024 - Company CCC.pdf 2020-01-02.34-56 Interest Anleihe Feb. 2024 - Company CCC.pdf

_

  1. trading Option contracts

pattern line (before):

  stock_order_settlement:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Abrechnung(.\\d+)?"}]
    path: "Stocks/Settlement/{iso_date_year}/"

  stock_order_cost_report:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Kosteninformation(.\\d+)?"}]
    path: "Stocks/Cost report/{iso_date_year}/"

  stock_order_created:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Auftragsbestätigung(.\\d+)?"}]
    path: "Stocks/Order created/{iso_date_year}/"

  notice_option_contract:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Basisinformationsblatt(.\\d+)?"}]
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {document_title} Option"

pattern line (after):

  stock_order_settlement:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Abrechnung(.\\d+)?"}]
    path: "Stocks/Settlement/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {section_action_companyname} - {event_title}"

  stock_order_cost_report:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Kosteninformation(.\\d+)?"}]
    path: "Stocks/Cost report/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {section_action_companyname} - {event_title}"

  stock_order_created:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Auftragsbestätigung(.\\d+)?"}]
    path: "Stocks/Order created/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {section_action_companyname} - {event_title}"

  notice_option_contract:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Basisinformationsblatt(.\\d+)?"}]
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {document_title} Option - {event_isin} {section_action_companyname}"

result file name (before): /Stocks/Settlement/2020/2020-01-02.34-56 Call 123 €.pdf /Stocks/Cost report/2020/2020-01-02.34-56 Call 123 €.pdf /Stocks/Order created/2020/2020-01-02.34-56 Call 123 €.pdf /Notice/2020/2020-01-02.34-56 Basisinformationsblatt Option.pdf

result file name (before): /Stocks/Settlement/2020/2020-01-02.34-56 Company DDD - Call 123 €.pdf /Stocks/Cost report/2020/2020-01-02.34-56 Company DDD - Call 123 €.pdf /Stocks/Order created/2020/2020-01-02.34-56 Company DDD - Call 123 €.pdf /Notice/2020/2020-01-02.34-56 Basisinformationsblatt Option - XX0000123456 - Company DDD.pdf

Additional context just a pseudo code as example

diff --git "a/pytr/timeline.py" "b/pytr/timeline.py"
index f809023..7bd1e64 100644
--- "a/pytr/timeline.py"
+++ "b/pytr/timeline.py"
@@ -1,3 +1,4 @@
+import re
 from datetime import datetime
 import json

@@ -129,12 +131,35 @@ class Timeline:
             + f"{event['title']} -- {event['subtitle']} - {event['timestamp'][:19]}"
         )

+        # set default value
+        event_isin = section_action_isin = 0
+        event_companyname = event['title']
+        section_action_companyname = ''

+        # we need company name from isin only for specific events where no company name is given
+        if event['eventType'] == 'COUPON_PAYMENT' or event['eventType'] == 'REPAYMENT' or event['eventType'] == 'STOCK_PERK_REFUNDED':
+          event_isin = re.fullmatch("logos/([A-Z]{2}[A-Z0-9]{9}[\d]{1})/v2", "", event['icon'])
+          # retrieve detail information from isin, maybe caching in dict [isin => name] and serialize for future access or sqlite?
+          event_companyname = await self.tr.stock_details(event_isin) # just pseudo code example, not implemented yet

         event['has_docs'] = False
         for section in response['sections']:
+
+            # set isin from section-action-payload
+            if section['type'] == 'header' and section['action']:
+                section_action_isin = section['action']['payload']
+                if event_isin == 0: # if event is not specific, set isin from regular value
+                    event_isin = section_action_isin
+
+            # set company name from underlying product (if available)
+            if section['title'] == 'Übersicht':
+                for data in section['data']:
+                    if data['title'] == 'Basiswert':
+                        section_action_companyname = data['detail']['text']
+
             if section['type'] != 'documents':
                 continue
+
             for doc in section['data']:
                 event['has_docs'] = True
                 try:
@@ -142,7 +167,12 @@ class Timeline:
                 except (ValueError, KeyError):
                     timestamp = datetime.now().timestamp()
                 if self.max_age_timestamp == 0 or self.max_age_timestamp < timestamp:
-                    dl.dl_doc(doc, event['eventType'], event['title'], event['subtitle'], section['title'], datetime.fromisoformat(event['timestamp'][:19]))
+
+                    # set section_action_companyname if empty
+                    if not section_action_companyname and section_action_isin != 0:
+                        section_action_companyname = await self.tr.stock_details(section_action_isin) # retrieve detail information from section-isin - just pseudo code example, not implemented yet
+
+                    dl.dl_doc(doc, event['eventType'], event['title'], event['subtitle'], section['title'], datetime.fromisoformat(event['timestamp'][:19]), event_isin, event_companyname, section_action_isin, section_action_companyname)

         if event['has_docs']:
             self.events_with_docs.append(event)