pytr-org / pytr

Use TradeRepublic in terminal and mass download all documents
https://pypi.org/project/pytr/
MIT License
422 stars 74 forks source link

Customizable destination folder structure #98

Open b166er opened 4 months ago

b166er commented 4 months ago

Is your feature request related to a problem? Please describe. I would finally like a fixed folder structure, independent of API changes (like #91), which everyone can set according to their own preferences.

Describe the solution you'd like I suggest defining the target folders using several search criteria/search patterns. This search-target mapping should then be saved in a suitable configuration file. An example configuration file is attached.

for most search patterns you only need the following json nodes from the API-response: in root-event node: “type” ("event_type"), “subtitle” ("event_subtitle"), for a few documents you also need the “title” ("event_title"). In section-node: “title” ("section_title"), and finaly in "document"-node the “title” ("document_title").

most search patterns contain 1:1 strings. rarely, dynamic regex number patterns are also available, e.g. document number “\d” or for larger numbers e.g. years “\d+”.

There should also be a possibility for dynamic names in the target path, as predefined variables. In addition to the existing search patterns that can be reused in the path {event_type} - {event_title}, {event_subtitle}, {document_title}, there should also be environment variables, {iso_date} and {time} already exist, it would be nice to have the individual components, year: {%Y} => 2024, and of course month and day.

I personally find YAML format much more human-readable (especially because there is a comment option), nevertheless, in order to stay with the current technology and not want to introduce new dependencies into the project, here is the example configuration also in JSON format foldermap.json

Additional context

destination:
# valid for all blocks without explicit filename
  all:
    filename: "{iso_date}.{time} {event_title}" # {event_title} = Wertpapier-/ETF-/Produkt-Name

# if pattern not found, use this block
  unknown:
    path: "Unknown/{section_title}/"
    filename: "{iso_date}.{time} {event_type} - {event_subtitle} - {document_title} - {event_title}"

# Aktien
  aktien_kauf_abrechnung:
    pattern: [
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Kauforder",        section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Limit-Buy-Order",  section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Stop-Sell-Order",  section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit verkauft
      { event_type: "TRADE_INVOICE",        event_subtitle: "Limit-Buy-Order",  section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Verkaufsorder",    section_title: "Dokumente", document_title: "Abrechnung" }, # verkauft
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",        section_title: "Dokumente", document_title: "Abrechnung" }, # Aktiengeschenk
      { event_type: "SHAREBOOKING",         event_subtitle: "Verkauf",          section_title: "Dokumente", document_title: "Abrechnung \\d" }, # Kapitalmassnahme
    ]
    path: "Wertpapiere/Abrechnung/{%Y}/"

  aktien_kauf_kosteninfo:
    pattern: [
      { event_type: "ORDER_CREATED",        event_subtitle: "Limit-Buy-Order erstellt", section_title: "Dokumente", document_title: "Kosteninformation" }, # limit erstellt
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Kosteninformation" },
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Stop-Sell-Order",          section_title: "Dokumente", document_title: "Kosteninformation" },
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Kauforder",                section_title: "Dokumente", document_title: "Kosteninformation" }, # ohne limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Verkaufsorder",            section_title: "Dokumente", document_title: "Kosteninformation" }, # verkauft
      { event_type: "TRADE_INVOICE",        event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Kosteninformation" },
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",                section_title: "Dokumente", document_title: "Kosteninformation" }, # Aktiengeschenk
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",                section_title: "Dokumente", document_title: "Kosteninformation \\d" }, # Aktiengeschenk
      { event_type: "EX_POST_COST_REPORT",  event_subtitle: , event_title: "Ex-Post Kosteninformation", section_title: "Dokumente", document_title: "Ex-Post Kosteninformation \\d+" },
    ]
    path: "Wertpapiere/Kosteninfo/{%Y}/"

  aktien_kauf_limit_erstellt:
    pattern: [
      { event_type: "ORDER_CREATED",  event_subtitle: "Limit-Buy-Order erstellt", section_title: "Dokumente", document_title: "Auftragsbestätigung" },
      { event_type: "ORDER_EXECUTED", event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Auftragsbestätigung" },
      { event_type: "ORDER_EXECUTED", event_subtitle: "Stop-Sell-Order",          section_title: "Dokumente", document_title: "Auftragsbestätigung" },
      { event_type: "TRADE_INVOICE",  event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Auftragsbestätigung" },
    ]
    path: "Wertpapiere/Order erstellt/{%Y}/"

  aktien_limit_storno:
    pattern: [
      { event_type: "TRADE_CANCELED",         event_subtitle: "Verkauf-Abrechnung storniert", section_title: "Dokumente", document_title: "Abrechnung \\d" },
      { event_type: "ORDER_CANCELED",         event_subtitle: "Limit-Buy-Order storniert",    section_title: "Dokumente", document_title: "Löschbestätigung" },
      { event_type: "SAVINGS_PLAN_CANCELED",  event_subtitle: "Sparplan storniert",           section_title: "Dokumente", document_title: "Stornierungsbestätigung" },
    ]
    path: "Wertpapiere/Order storno/{%Y}/"

# Kapitalmaßnahmen
  info_anzeigen1:
    pattern: [
      { event_type: "EXERCISE",     event_subtitle: "Ausübung",                                 section_title: "Dokumente", document_title: "Kundenanschreiben" },
      { event_type: "SHAREBOOKING", event_subtitle: "Split",                                    section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Fusion",                                   section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Verkauf",                                  section_title: "Dokumente", document_title: "Ausführungsanzeige \\d" },
      { event_type: "SHAREBOOKING", event_subtitle: "Ausbuchung",                               section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "ISIN Wechsel",                             section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Entflechtung",                             section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Reverse Split",                            section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Titelumtausch",                            section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Umtausch/Bezug",                           section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Reorganisation",                           section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Stockdividende",                           section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Kapitalreduktion",                         section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Titelgleichstellung",                      section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Wertlose Ausbuchung",                      section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Effektive Auslieferung",                   section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Kapitalerhöhung gegen Bar",                section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Kapitalerhöhung aus Gesellschaftsmitteln", section_title: "Dokumente", document_title: "Ausführungsanzeige" },
    ]
    path: "Wertpapiere/Anzeigen/{%Y}/"
    filename: "{iso_date}.{time} {event_subtitle} {event_title}"

# split for better readability
  info_anzeigen2:
    pattern: [
      { event_type: "CORPORATE_ACTION",      event_subtitle: "Kapitalherabsetzung",                 section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_CANCELED", event_subtitle: "Entflechtung storniert",              section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING_CANCELED", event_subtitle: "Stockdividende storniert",            section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING_CANCELED", event_subtitle: "Kapitalerhöhung gegen Bar storniert", section_title: "Dokumente", document_title: "Ausführungsanzeige" },
    ]
    path: "Wertpapiere/Anzeigen/{%Y}/"
    filename: "{iso_date}.{time} {event_subtitle} {event_title}"

# split for better readability
  info_anzeigen3:
    pattern: [
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Fusion",                      section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Reinvestierung",              section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Umtausch/Bezug",              section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Zwangsübernahme",             section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Kapitalreduktion",            section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "INSTRUCTION_CORPORATE_ACTION", event_subtitle: "Weisung zur Aktiendividende", section_title: "Dokumente", document_title: "Kundenanschreiben" }, #Kapitalerhöhung gegen Bar
      { event_type: "INSTRUCTION_CORPORATE_ACTION", event_subtitle: "Weisung zur Aktiendividende", section_title: "Dokumente", document_title: "Kundenanschreiben \\d" }, #Kapitalerhöhung gegen Bar
    ]
    path: "Wertpapiere/Anzeigen/{%Y}/"
    filename: "{iso_date}.{time} {event_subtitle} {event_title}"

# split for better readability
  info_anzeigen4:
    pattern: [
      { event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Spin-off",      section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Reverse Split", section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Wechsel",       section_title: "Dokumente", document_title: "Information" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Information",   section_title: "Dokumente", document_title: "Information" },
    ]
    path: "Wertpapiere/Anzeigen/{%Y}/"
    filename: "{iso_date}.{time} {event_subtitle} {event_title}"

  portfolio_abschluess:
    pattern: [
      { event_type: "QUARTERLY_REPORT", event_subtitle: , event_title: "Q\\d/\\d+ Abschluss", section_title: "Dokumente", document_title: "Kontoauszug" },
      { event_type: "QUARTERLY_REPORT", event_subtitle: , event_title: "Q\\d/\\d+ Abschluss", section_title: "Dokumente", document_title: "Depotauszug" },
      { event_type: "QUARTERLY_REPORT", event_subtitle: , event_title: "Q\\d/\\d+ Abschluss", section_title: "Dokumente", document_title: "Cryptoauszug" },
    ]
    path: "Wertpapiere/Abschluss/{%Y}/"
    filename: "{iso_date} {document_title} {event_title}"

# Dividenden
  dividende_erhalten:
    pattern: [
      { event_type: "CREDIT",                             event_subtitle: "Dividende",    section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "CREDIT",                             event_subtitle: "Ausschüttung", section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "ssp_corporate_action_invoice_cash",  event_subtitle: "Bardividende", section_title: "Dokumente", document_title: "Dokumente" },
    ]
    path: "Dividenden/{%Y}/"

# Dividenden Kapitalmaßnahme wahl
  dividende_wahl:
    pattern: [
      { event_type: "INSTRUCTION_CORPORATE_ACTION",                   event_subtitle: "Weisung zur Aktiendividende",  section_title: "Dokumente", document_title: "Dividende Wahlweise" },
      { event_type: "ssp_dividend_option_customer_instruction",       event_subtitle: "Cash oder Aktie",              section_title: "Dokumente", document_title: "Dividende Wahlweise" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Dividende Wahlweise",          section_title: "Dokumente", document_title: "Kapitalmaßnahmen" },
    ]
    path: "Dividendenwahl/{%Y}/"

# Hauptversammlungen
  hauptversammlungen:
    pattern: [
      { event_type: "GENERAL_MEETING",                                event_subtitle: "Hauptversammlung",           section_title: "Dokumente", document_title: "Hauptversammlung" },
      { event_type: "GENERAL_MEETING",                                event_subtitle: "Hauptversammlung",           section_title: "Dokumente", document_title: "Hauptversammlung \\d" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Jährliche Hauptversammlung", section_title: "Dokumente", document_title: "Information" },
    ]
    path: "Wertpapiere/Hauptversammlungen/{%Y}/"

# Sparplan
  sparplan:
    pattern: [
      { event_type: "SAVINGS_PLAN_INVOICE_CREATED", event_subtitle: "Sparplan ausgeführt", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
      { event_type: "SAVINGS_PLAN_EXECUTED",        event_subtitle: "Sparplan ausgeführt", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
    ]
    path: "Wertpapiere/Sparplan/{%Y}/"

# Vorabpauschale
  vorabpauschale:
    pattern: [
      { event_type: "PRE_DETERMINED_TAX_BASE_EARNING", event_subtitle: "Vorabpauschale", section_title: "Dokumente", document_title: "Vorabpauschale" },
    ]
    path: "Wertpapiere/Vorabpauschale/{%Y}/"

# Anleihen
  anleihe_tilgung:
    pattern: [
      { event_type: "REPAYMENT", event_subtitle: "Tilgung", section_title: "Dokumente", document_title: "Abrechnung" },
    ]
    path: "Anleihen/Tilgungen/{%Y}/"

  anleihe_zinsen:
    pattern: [
      { event_type: "COUPON_PAYMENT", event_subtitle: "Coupon Zahlung", section_title: "Dokumente", document_title: "Abrechnung" },
    ]
    path: "Anleihen/Zinsen/{%Y}/"

# Saveback
  saveback_aktiviert:
    pattern: [
      { event_type: "benefits_saveback_execution", event_subtitle: "Saveback", section_title: "Dokumente", document_title: "Aktivierung" },
    ]
    path: "Saveback/aktiviert/"

  saveback_abrechnung:
    pattern: [
      { event_type: "benefits_saveback_execution", event_subtitle: "Saveback", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
    ]
    path: "Saveback/Abrechnung/"

  saveback_kosteninfo:
    pattern: [
      { event_type: "benefits_saveback_execution", event_subtitle: "Saveback", section_title: "Dokumente", document_title: "Kosteninformation" },
    ]
    path: "Saveback/Kosteninfo/"

# Round up
  roundup_aktiviert:
    pattern: [
      { event_type: "benefits_spare_change_execution", event_subtitle: "Round up", section_title: "Dokumente", document_title: "Aktivierung" }, # same files - multiple times at once
    ]
    path: "Roundup/aktiviert/{%Y}/"

  roundup_ausgefuert:
    pattern: [
      { event_type: "benefits_spare_change_execution", event_subtitle: "Round up", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
    ]
    path: "Roundup/Abrechnung/{%Y}/"

  roundup_kosteninfo:
    pattern: [
      { event_type: "benefits_spare_change_execution", event_subtitle: "Round up", section_title: "Dokumente", document_title: "Kosteninformation" },
    ]
    path: "Roundup/Kosteninfo/{%Y}/"

# Konto
  zinsen:
    pattern: [
      { event_type: "INTEREST_PAYOUT",          event_subtitle: , event_title: "Zinsen", section_title: "Dokument", document_title: "Abrechnung" },
      { event_type: "INTEREST_PAYOUT_CREATED",  event_subtitle: , event_title: "Zinsen", section_title: "Dokument", document_title: "Abrechnung" },
    ]
    path: "Zinsen/"
    filename: "{iso_date} Zinsabrechnung"

  uberweisung_ankommend:
    pattern: [
      { event_type: "INCOMING_TRANSFER", event_subtitle: "Erhalten", section_title: "Dokument", document_title: "Transaktionsbestätigung" },
    ]
    path: "Konto/"
    filename: "{iso_date}.{time} {document_title}" # {event_title} = Personal name

# Tax
  steuer_report:
    pattern: [
      { event_type: "TAX_REFUND",          event_subtitle: "Steuerbuchung", event_title: "Steuerabrechnung",        section_title: "Dokument", document_title: "Steuerabrechnung" },
      { event_type: "YEAR_END_TAX_REPORT", event_subtitle: "Jahr \\d+",     event_title: "Jährlicher Steuerreport", section_title: "Dokument", document_title: "Steuerreport \\d+" },
    ]
    path: "Steuern/"
    filename: "{iso_date} {document_title}"

  steuer_korrektur:
    pattern: [
      { event_type: "ssp_tax_correction_invoice", event_subtitle: , event_title: "Steuerkorrektur", section_title: "Dokument", document_title: "Steuerabrechnung" },
    ]
    path: "Steuern/"
    filename: "{iso_date} {event_title}"

# Allgemeine Infos
  infos1_wertpapiere:
    pattern: [
      { event_type: "CUSTOMER_CREATED",  event_subtitle: "Erhalten", section_title: "Dokumente",               document_title: "Basisinformationen über Wertpapiere" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Hinweise zu den Handelsplätzen" }, # same file as below
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Hinweise zu den Handelsplätzen" }, # same file as above
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Kundenvereinbarung" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Kundenvereinbarung" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Datenschutzinformationen" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Datenschutzinformationen" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Info zur Einlagensicherung" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Info zur Einlagensicherung" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Preis- und Leistungsverzeichnis" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Preis- und Leistungsverzeichnis" },
    ]
    path: "Infos/"
    filename: "{iso_date} {document_title}"

  infos2_crypto:
    pattern: [
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Kundenvereinbarung" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Customer Agreement" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Mistrade Regelungen" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Risikohinweise Crypto" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Crypto Verwahrbedingungen" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Vorvertragliche Informationen" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Widerrufsbelehrung Cryptoverwahrer" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Datenschutzinformationen Cryptoverwahrer" },
    ]
    path: "Infos/"
    filename: "{iso_date} {document_title}"

  infos3_optionen_etfs:
    pattern: [
      { event_type: "ORDER_EXECUTED",                           event_subtitle: "Kauforder",            section_title: "Dokumente", document_title: "Basisinformationsblatt" }, # ohne limit gekauft
      { event_type: "GESH_CORPORATE_ACTION",                    event_subtitle: "Unternehmensmeldung",  section_title: "Dokumente", document_title: "Kundenanschreiben" },
      { event_type: "GESH_CORPORATE_ACTION_MULTIPLE_POSITIONS", event_subtitle: "Gesellschaftshinweis", section_title: "Dokumente", document_title: "Kundenanschreiben \\d" },
    ]
    path: "Infos/"
    filename: "{iso_date} {document_title} - {event_title}"

  karte_bestellt:
    pattern: [
      { event_type: "card_order_billed", event_subtitle: , event_title: "Trade Republic Card",  section_title: "Dokumente", document_title: "Bestellung Trade Republic Karte" },
    ]
    path: "Infos/"
    filename: "{iso_date} {document_title}"

# enthalten keine Dokumente, werden übersprungen:
#
#  uberweisung_ankommend:
#    pattern: [
#      { event_type: "PAYMENT_INBOUND", event_subtitle: , section_title: , document_title: },
#      { event_type: "INCOMING_TRANSFER", event_subtitle: "Erhalten", section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  uberweisung_ausgehend:
#    pattern: [
#      { event_type: "PAYMENT_OUTBOUND", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  karte_geld_erhalten:
#    pattern: [
#      { event_type: "card_refund", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  karte_bezahlt:
#    pattern: [
#      { event_type: "card_successful_transaction", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  karte_geld_abheben:
#    pattern: [
#      { event_type: "card_successful_atm_withdrawal", event_subtitle: , event_title: "Abhebung", section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#
#  karte_transaktions_fehler:
#    pattern: [
#      { event_type: "card_failed_transaction", event_subtitle: "Abgebrochen", section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  karte_verifikation_erfolgreich:
#    pattern: [
#      { event_type: "card_successful_verification", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  neue_iban:
#    pattern: [
#      { event_type: "new_tr_iban", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Girokonto/"
#
#  order_abgelehnt:
#    pattern: [
#      { event_type: "ORDER_REJECTED", event_subtitle: "Kauforder abgelehnt", section_title: , document_title: },
#    ]
#    path: "Wertpapiere/Order storno/{%Y}/"
#
#
#  order_abgelehnt:
#    pattern: [
#      { event_type: "ORDER_EXPIRED", event_subtitle: "Kauforder abgelaufen", section_title: , document_title: },
#    ]
#    path: "Wertpapiere/Order storno/{%Y}/"
#

hope, this is possible.

Katzmann1983 commented 4 months ago

The idea is great. At the moment the folder structure is a mess, especially in case of non english accounts. My dl_docs has folders like this: Abrechnung Abrechnung Ausführung Abschlüsse Auftragsbestätigung Ausführungsanzeige Basisinformationsblatt Contrato con el cliente Cryptoauszug Dividende Ex-Post Kosteninformation Hauptversammlung Información de protección de datos Información sobre la garantía de depósitos Kapitalmaßnahmen Kosteninformation Kundenanschreiben Kundenvereinbarung Listado de precios y servicios Löschbestätigung Saveback Sparplan Steuerreport Términos y condiciones de la oferta acción gratis Your terms of service for instant trading volume Zinsen

We should be using folder and file names without space, everything in English, with a proper translation to German, Spanish, etc.

b166er commented 4 months ago

and that is the goal of my suggestion, with a config-map-file every user can decide for himself where which file goes into which folder and how the folders are named and the structure looks like. the config shown above is only an example of how it could look like, the folder names can definitely be specified, and every user can then decide for himself at the end what the folder should be called. I think the “uknown” block is also important in the config-map-file, so that API changes can be recognized immediately (by placing documents in the “unknown” folder) and, if necessary, users can adjust them themselves in the config.

BjBe82 commented 4 months ago

I like the idea too and implemented a first draft based on @b166er yaml (with some small changes to the variables) here: https://github.com/BjBe82/pytr/tree/Issue-98-Customizable-destination-folder-structure

If someone is willing to review/test. There are some patterns in unknown still which need a pattern in the yaml file, but for now i use this for testing.

This introduces two new dependencies:

The script will copy the default template from "config/file_destination_config__template.yaml" to the user directory "/.pytr/file_destination_config.yaml" if it is missing (where also pytr stores the credentials). If it is present it will just read the config from there.

b166er commented 4 months ago

good work @BjBe82 in most cases your solution works great, by your solution, I could already find more documents that were missing in the config-map! i noticed a few small things.

  1. in the pattern “info_anzeigen1” there is an event_subtitle: “Umtausch/Bezug”, the “/” is interpreted as a directory delimiter and thus a subdirectory is created for this document. The same as in the pattern “portfolio_abschluess” with event_title: "Q\d/\d+ Abschluss". My suggestion is to clean all node element names (event_type, event_title, event_subtitle, section_title, documenttitle) to get a valid file name. the following special characters must be removed or replaced with e.g. "" or "-"..., to get a proper file name (at least in Windows): \ / : * ? “ < > |

  2. After a run, some documents were stored in the “unknown” folder, although the patterns are present. After a short troubleshooting, I found out that the string “SHAREBOOKING” with re.match() is equal to both “SHAREBOOKING” and “SHAREBOOKING_CANCELED”! i think we need to rethink the strategy how we compare the node names. there is definitely the requirement to match with regex, but not all strings are allowed to be matched with regex. i added “_regex”-postfix in the patterns for testing to get a clear distinction in the patterns which should be matched 1:1 with strings and which are allowed to be matched with regex.

  3. I think it would be nice if we could distinguish whether documents have no pattern found or several patterns were found at the same time for an event, this makes it easier to search for the errors later. i have therefore introduced another rule block that is only used for multiple results.

  4. As @Katzmann1983 has already correctly criticized, I have tried to rename all rule names and target folder names in English so that the config is uniformly readable by everyone. I think the users can then later customize their own folder names in their preferred language.

@todo what else occurs to me, are the json node names (e.g. event_subtitle or document_title) dependent on the selected language in the app from the user account or do they always come in German? if the nodes are also available in other languages, then we need a translation table.

unfortunately i am not allowed to upload files with yaml-extension to tickets in this github project, so i am adding the content of the config file directly to the ticket. here is the updated config:

destination:
# valid for all blocks without explicit filename
  default:
    filename: "{iso_date}.{iso_time} {event_title}" # {event_title} = Wertpapier-/ETF-/Produkt-Name

# if pattern not found, use this block
  unknown:
    path: "Unknown/{section_title}/"
    filename: "{iso_date}.{iso_time} {event_type} - {event_subtitle} - {document_title} - {event_title}"

# if pattern found multiple times, use this block
  multiple_match:
    path: "MultipleMatchPattern/{section_title}/"
    filename: "{iso_date}.{iso_time} {event_type} - {event_subtitle} - {document_title} - {event_title}"

# stocks
  stock_order_settlement:
    pattern: [
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Kauforder",        section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Limit-Buy-Order",  section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Stop-Sell-Order",  section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit verkauft
      { event_type: "TRADE_INVOICE",        event_subtitle: "Limit-Buy-Order",  section_title: "Dokumente", document_title: "Abrechnung" }, # mit limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Verkaufsorder",    section_title: "Dokumente", document_title: "Abrechnung" }, # verkauft
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",        section_title: "Dokumente", document_title: "Abrechnung" }, # Aktiengeschenk
      { event_type: "SHAREBOOKING",         event_subtitle: "Verkauf",          section_title: "Dokumente", document_title_regex: "Abrechnung \\d" }, # Kapitalmassnahme
    ]
    path: "Stocks/Settlement/{iso_date_year}/"

  stock_order_cost_report:
    pattern: [
      { event_type: "ORDER_CREATED",        event_subtitle: "Limit-Buy-Order erstellt", section_title: "Dokumente", document_title: "Kosteninformation" }, # limit erstellt
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Kosteninformation" },
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Stop-Sell-Order",          section_title: "Dokumente", document_title: "Kosteninformation" },
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Kauforder",                section_title: "Dokumente", document_title: "Kosteninformation" }, # ohne limit gekauft
      { event_type: "ORDER_EXECUTED",       event_subtitle: "Verkaufsorder",            section_title: "Dokumente", document_title: "Kosteninformation" }, # verkauft
      { event_type: "TRADE_INVOICE",        event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Kosteninformation" },
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",                section_title: "Dokumente", document_title: "Kosteninformation" }, # Aktiengeschenk
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",                section_title: "Dokumente", document_title_regex: "Kosteninformation \\d" }, # Aktiengeschenk
      { event_type: "EX_POST_COST_REPORT",  event_title: "Ex-Post Kosteninformation",   section_title: "Dokumente", document_title_regex: "Ex-Post Kosteninformation \\d+" },
    ]
    path: "Stocks/Cost report/{iso_date_year}/"

  stock_order_created:
    pattern: [
      { event_type: "TRADE_INVOICE",  event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Auftragsbestätigung" },
      { event_type: "ORDER_CREATED",  event_subtitle: "Limit-Buy-Order erstellt", section_title: "Dokumente", document_title: "Auftragsbestätigung" },
      { event_type: "ORDER_EXECUTED", event_subtitle: "Limit-Buy-Order",          section_title: "Dokumente", document_title: "Auftragsbestätigung" },
      { event_type: "ORDER_EXECUTED", event_subtitle: "Stop-Sell-Order",          section_title: "Dokumente", document_title: "Auftragsbestätigung" },
    ]
    path: "Stocks/Order created/{iso_date_year}/"

  stock_order_canceled:
    pattern: [
      { event_type: "ORDER_CANCELED",         event_subtitle: "Limit-Buy-Order storniert",    section_title: "Dokumente", document_title: "Löschbestätigung" },
      { event_type: "TRADE_CANCELED",         event_subtitle: "Verkauf-Abrechnung storniert", section_title: "Dokumente", document_title_regex: "Abrechnung \\d" },
      { event_type: "SAVINGS_PLAN_CANCELED",  event_subtitle: "Sparplan storniert",           section_title: "Dokumente", document_title: "Stornierungsbestätigung" },
    ]
    path: "Stocks/Order canceled/{iso_date_year}/"

# stocks Notice
  stock_notice1:
    pattern: [
      { event_type: "EXERCISE",     event_subtitle: "Ausübung",                                 section_title: "Dokumente", document_title: "Kundenanschreiben" },
      { event_type: "SHAREBOOKING", event_subtitle: "Split",                                    section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Fusion",                                   section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Verkauf",                                  section_title: "Dokumente", document_title_regex: "Ausführungsanzeige \\d" },
      { event_type: "SHAREBOOKING", event_subtitle: "Ausbuchung",                               section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "ISIN Wechsel",                             section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Entflechtung",                             section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Reverse Split",                            section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Titelumtausch",                            section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Umtausch/Bezug",                           section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Reorganisation",                           section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Stockdividende",                           section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Kapitalreduktion",                         section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Titelgleichstellung",                      section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Wertlose Ausbuchung",                      section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Effektive Auslieferung",                   section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Kapitalerhöhung gegen Bar",                section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING", event_subtitle: "Kapitalerhöhung aus Gesellschaftsmitteln", section_title: "Dokumente", document_title: "Ausführungsanzeige" },
    ]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

# split for better readability
  stock_notice2:
    pattern: [
      { event_type: "CORPORATE_ACTION",      event_subtitle: "Kapitalherabsetzung",                 section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_CANCELED", event_subtitle: "Entflechtung storniert",              section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING_CANCELED", event_subtitle: "Stockdividende storniert",            section_title: "Dokumente", document_title: "Ausführungsanzeige" },
      { event_type: "SHAREBOOKING_CANCELED", event_subtitle: "Kapitalerhöhung gegen Bar storniert", section_title: "Dokumente", document_title: "Ausführungsanzeige" },
    ]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

# split for better readability
  stock_notice3:
    pattern: [
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Fusion",                      section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Reinvestierung",              section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Umtausch/Bezug",              section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Zwangsübernahme",             section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "SHAREBOOKING_TRANSACTIONAL",   event_subtitle: "Kapitalreduktion",            section_title: "Dokumente", document_title: "Abrechnung" },
      # Capital increase in exchange for cash / Kapitalerhöhung gegen Bar
      { event_type: "INSTRUCTION_CORPORATE_ACTION", event_subtitle: "Weisung zur Aktiendividende", section_title: "Dokumente", document_title: "Kundenanschreiben" },
      { event_type: "INSTRUCTION_CORPORATE_ACTION", event_subtitle: "Weisung zur Aktiendividende", section_title: "Dokumente", document_title_regex: "Kundenanschreiben \\d" },
    ]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

# split for better readability
  stock_notice4:
    pattern: [
      { event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Spin-off",                           section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Reverse Split",                      section_title: "Dokumente", document_title: "Dokumente" },
      # Recognition of subscription rights / Einbuchung Bezugsrechte
      { event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Zwischenvertrieb von Wertpapieren",  section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Wechsel",                            section_title: "Dokumente", document_title: "Information" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Information",                        section_title: "Dokumente", document_title: "Information" },
      # Repurchase of interim securities / Rückkauf von Zwischentiteln
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Aufruf von Zwischenpapieren",        section_title: "Dokumente", document_title: "Kapitalmaßnahmen" },
    ]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

  stock_report:
    pattern: [
      { event_type: "QUARTERLY_REPORT", event_title_regex: "Q\\d/\\d+ Abschluss", section_title: "Dokumente", document_title: "Kontoauszug" },
      { event_type: "QUARTERLY_REPORT", event_title_regex: "Q\\d/\\d+ Abschluss", section_title: "Dokumente", document_title: "Depotauszug" },
      { event_type: "QUARTERLY_REPORT", event_title_regex: "Q\\d/\\d+ Abschluss", section_title: "Dokumente", document_title: "Cryptoauszug" },
    ]
    path: "Stocks/Report/{iso_date_year}/"
    filename: "{iso_date} {document_title} {event_title}"

# General Meetings
  stock_general_meetings:
    pattern: [
      { event_type: "GENERAL_MEETING",                                event_subtitle: "Hauptversammlung",                                 section_title: "Dokumente", document_title: "Hauptversammlung" },
      { event_type: "GENERAL_MEETING",                                event_subtitle: "Hauptversammlung",                                 section_title: "Dokumente", document_title_regex: "Hauptversammlung \\d" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Jährliche Hauptversammlung",                       section_title: "Dokumente", document_title: "Information" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Außerordentliche oder spezielle Hauptversammlung", section_title: "Dokumente", document_title: "Information" },
    ]
    path: "Stocks/General Meetings/{iso_date_year}/"

# Savings plan
  savings_plan:
    pattern: [
      { event_type: "SAVINGS_PLAN_INVOICE_CREATED", event_subtitle: "Sparplan ausgeführt", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
      { event_type: "SAVINGS_PLAN_EXECUTED",        event_subtitle: "Sparplan ausgeführt", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
    ]
    path: "Stocks/Savings plan/{iso_date_year}/"

# pre-determined tax base earning
  stock_pre_earning_tax:
    pattern: [
      { event_type: "PRE_DETERMINED_TAX_BASE_EARNING", event_subtitle: "Vorabpauschale", section_title: "Dokumente", document_title: "Vorabpauschale" },
    ]
    path: "Stocks/PreEarningTax/{iso_date_year}/"

# Dividends
  dividends_received:
    pattern: [
      { event_type: "CREDIT",                             event_subtitle: "Dividende",                   section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "CREDIT",                             event_subtitle: "Ausschüttung",                section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "ssp_corporate_action_invoice_cash",  event_subtitle: "Bardividende",                section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_invoice_cash",  event_subtitle: "Dividende Wahlweise",         section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_invoice_cash",  event_subtitle: "Bardividende korrigiert",     section_title: "Dokumente", document_title: "Dokumente" },
      { event_type: "ssp_corporate_action_invoice_cash",  event_subtitle: "Reinvestition der Dividende", section_title: "Dokumente", document_title: "Dokumente" },
    ]
    path: "Dividends/{iso_date_year}/"

# Dividends Corporate action election
  dividends_election:
    pattern: [
      { event_type: "INSTRUCTION_CORPORATE_ACTION",                   event_subtitle: "Weisung zur Aktiendividende",  section_title: "Dokumente", document_title: "Dividende Wahlweise" },
      { event_type: "ssp_dividend_option_customer_instruction",       event_subtitle: "Cash oder Aktie",              section_title: "Dokumente", document_title: "Dividende Wahlweise" },
      { event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Dividende Wahlweise",          section_title: "Dokumente", document_title: "Kapitalmaßnahmen" },
    ]
    path: "Dividendelection/{iso_date_year}/"

# bonds
  bond_repayment:
    pattern: [
      { event_type: "REPAYMENT", event_subtitle: "Tilgung", section_title: "Dokumente", document_title: "Abrechnung" },
    ]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Repayment {event_title}"

  bond_interest:
    pattern: [
      { event_type: "COUPON_PAYMENT", event_subtitle: "Coupon Zahlung", section_title: "Dokumente", document_title: "Abrechnung" },
    ]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Interest {event_title}"

# Saveback
  saveback_enabled:
    pattern: [
      { event_type: "benefits_saveback_execution", event_subtitle: "Saveback", section_title: "Dokumente", document_title: "Aktivierung" },
    ]
    path: "Saveback/Enabled/"

  saveback_executed:
    pattern: [
      { event_type: "benefits_saveback_execution", event_subtitle: "Saveback", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
    ]
    path: "Saveback/Report/"

  saveback_cost_report:
    pattern: [
      { event_type: "benefits_saveback_execution", event_subtitle: "Saveback", section_title: "Dokumente", document_title: "Kosteninformation" },
    ]
    path: "Saveback/Cost report/"

# Round up
  roundup_enabled:
    pattern: [
      # same files - multiple times at once
      { event_type: "benefits_spare_change_execution", event_subtitle: "Round up", section_title: "Dokumente", document_title: "Aktivierung" },
    ]
    path: "Roundup/Enabled/"

  roundup_executed:
    pattern: [
      { event_type: "benefits_spare_change_execution", event_subtitle: "Round up", section_title: "Dokumente", document_title: "Abrechnung Ausführung" },
    ]
    path: "Roundup/Report/{iso_date_year}/"

  roundup_cost_report:
    pattern: [
      { event_type: "benefits_spare_change_execution", event_subtitle: "Round up", section_title: "Dokumente", document_title: "Kosteninformation" },
    ]
    path: "Roundup/Cost report/{iso_date_year}/"

# account
  cash_interest:
    pattern: [
      { event_type: "INTEREST_PAYOUT",          event_title: "Zinsen", section_title: "Dokumente", document_title: "Abrechnung" },
      { event_type: "INTEREST_PAYOUT_CREATED",  event_title: "Zinsen", section_title: "Dokumente", document_title: "Abrechnung" },
    ]
    path: "Cash Interest/"
    filename: "{iso_date} Zinsabrechnung"

  cash_transfer_report:
    pattern: [
      { event_type: "INCOMING_TRANSFER", event_subtitle: "Erhalten", section_title: "Dokument", document_title: "Transaktionsbestätigung" },
    ]
    path: "Cash Report/"
    filename: "{iso_date}.{iso_time} {document_title}" # {event_title} = Personal name

# annual tax report for account
  account_tax_report:
    pattern: [
      { event_type: "TAX_REFUND",          event_subtitle: "Steuerbuchung",   event_title: "Steuerabrechnung",        section_title: "Dokumente", document_title: "Steuerabrechnung" },
      { event_type: "YEAR_END_TAX_REPORT", event_subtitle_regex: "Jahr \\d+", event_title: "Jährlicher Steuerreport", section_title: "Dokument",  document_title_regex: "Steuerreport \\d+" },
    ]
    path: "Tax/"
    filename: "{iso_date} {document_title}"

  account_tax_adjustment:
    pattern: [
      { event_type: "ssp_tax_correction_invoice", event_title: "Steuerkorrektur", section_title: "Dokumente", document_title: "Steuerabrechnung" },
    ]
    path: "Tax/"
    filename: "{iso_date} {event_title}"

# common informations
  notice_stocks:
    pattern: [
      { event_type: "CUSTOMER_CREATED",  event_subtitle: "Erhalten", section_title: "Dokumente",               document_title: "Basisinformationen über Wertpapiere" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Kundenvereinbarung" }, # same file as below
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Kundenvereinbarung" }, # same file as above
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Datenschutzinformationen" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Datenschutzinformationen" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Info zur Einlagensicherung" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Info zur Einlagensicherung" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Hinweise zu den Handelsplätzen" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Hinweise zu den Handelsplätzen" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Dokumente",               document_title: "Preis- und Leistungsverzeichnis" },
      { event_type: "DOCUMENTS_CHANGED", event_subtitle: "Geändert", section_title: "Aktualisierte Dokumente", document_title: "Preis- und Leistungsverzeichnis" },
    ]
    path: "Notice/"
    filename: "{iso_date} {document_title}"

  notice_crypto:
    pattern: [
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Kundenvereinbarung" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Customer Agreement" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Mistrade Regelungen" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Risikohinweise Crypto" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Crypto Verwahrbedingungen" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Vorvertragliche Informationen" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Widerrufsbelehrung Cryptoverwahrer" },
      { event_type: "DOCUMENTS_ACCEPTED", event_subtitle: "Angenommen", section_title: "Dokumente", document_title: "Datenschutzinformationen Cryptoverwahrer" },
    ]
    path: "Notice/"
    filename: "{iso_date} {document_title}"

  notice_warrant_and_etfs:
    pattern: [
      # when you buy a product for the first time, you will receive information documents about this product
      { event_type: "ORDER_EXECUTED",                           event_subtitle: "Kauforder",            section_title: "Dokumente", document_title: "Basisinformationsblatt" },
      { event_type: "GESH_CORPORATE_ACTION",                    event_subtitle: "Unternehmensmeldung",  section_title: "Dokumente", document_title: "Kundenanschreiben" },
      { event_type: "GESH_CORPORATE_ACTION_MULTIPLE_POSITIONS", event_subtitle: "Gesellschaftshinweis", section_title: "Dokumente", document_title_regex: "Kundenanschreiben \\d" },
    ]
    path: "Notice/"
    filename: "{iso_date} {document_title} - {event_title}"

  card_ordered:
    pattern: [
      { event_type: "card_order_billed", event_title: "Trade Republic Card",  section_title: "Dokumente", document_title: "Bestellung Trade Republic Karte" },
    ]
    path: "Notice/"
    filename: "{iso_date} {document_title}"

# The following events do not contain any documents, these nodes are skipped.
#
#  payment_inbound:
#    pattern: [
#      { event_type: "PAYMENT_INBOUND", event_subtitle: , section_title: , document_title: },
#      { event_type: "INCOMING_TRANSFER", event_subtitle: "Erhalten", section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  payment_outbout:
#    pattern: [
#      { event_type: "PAYMENT_OUTBOUND", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  card_refund:
#    pattern: [
#      { event_type: "card_refund", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  card_successful_transaction:
#    pattern: [
#      { event_type: "card_successful_transaction", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  card_successful_atm_withdrawal:
#    pattern: [
#      { event_type: "card_successful_atm_withdrawal", event_subtitle: , event_title: "Abhebung", section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#
#  card_failed_transaction:
#    pattern: [
#      { event_type: "card_failed_transaction", event_subtitle: "Abgebrochen", section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  card_successful_verification:
#    pattern: [
#      { event_type: "card_successful_verification", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  new_account_iban:
#    pattern: [
#      { event_type: "new_tr_iban", event_subtitle: , section_title: , document_title: },
#    ]
#    path: "Cash Report/"
#
#  stock_order_rejected:
#    pattern: [
#      { event_type: "ORDER_REJECTED", event_subtitle: "Kauforder abgelehnt", section_title: , document_title: },
#    ]
#    path: "Stocks/Order canceled/{iso_date_year}/"
#
#
#  stock_order_expired:
#    pattern: [
#      { event_type: "ORDER_EXPIRED", event_subtitle: "Kauforder abgelaufen", section_title: , document_title: },
#    ]
#    path: "Stocks/Order canceled/{iso_date_year}/"
#

Here is my suggestion for improving the points mentioned above.

diff --git a/pytr/file_destination_provider.py b/pytr/file_destination_provider.py
index 10e5287..e88249c 100644
--- a/pytr/file_destination_provider.py
+++ b/pytr/file_destination_provider.py
@@ -16,8 +16,9 @@ except ImportError:
     from yaml import Loader, Dumper

-ALL_CONFIG = "all"
+ALL_CONFIG = "default"
 UNKNOWN_CONFIG = "unknown"
+MULTIPLEMATCH_CONFIG = "multiple_match"

 TEMPLATE_FILE_NAME ="file_destination_config__template.yaml"

@@ -36,12 +37,17 @@ class DestinationConfig:

 class Pattern:
-    def __init__(self, event_type: str, event_subtitle: str, event_title: str,  section_title: str, document_title: str):
+    def __init__(self, event_type: str, event_subtitle: str, event_title: str,  section_title: str, document_title: str, event_type_regex: str, event_subtitle_regex: str, event_title_regex: str,  section_title_regex: str, document_title_regex: str):
         self.event_type = event_type
         self.event_subtitle = event_subtitle
         self.event_title = event_title
         self.section_title = section_title
         self.document_title = document_title
+        self.event_type_regex = event_type_regex
+        self.event_subtitle_regex = event_subtitle_regex
+        self.event_title_regex = event_title_regex
+        self.section_title_regex = section_title_regex
+        self.document_title_regex = document_title_regex

 class FileDestinationProvider:
@@ -51,7 +57,7 @@ class FileDestinationProvider:
         A provider for file path and file names based on the event type and other parameters.
         '''
         self._log = get_logger(__name__)
-        
+
         config_file_path = Path(DESTINATION_CONFIG_FILE)
         if config_file_path.is_file() == False:
             self.__create_default_config(config_file_path)
@@ -62,7 +68,7 @@ class FileDestinationProvider:
         self.__validate_config(destination_config)

         destinations = destination_config["destination"]
-        
+
         self._destination_configs: list[DestinationConfig] = []

         for config_name in destinations:
@@ -72,6 +78,9 @@ class FileDestinationProvider:
             elif config_name == UNKNOWN_CONFIG:
                 self._unknown_file_config = DestinationConfig(
                     UNKNOWN_CONFIG, destinations[UNKNOWN_CONFIG]["filename"], destinations[UNKNOWN_CONFIG]["path"])
+            elif config_name == MULTIPLEMATCH_CONFIG:
+                self._multiplematch_file_config = DestinationConfig(
+                    MULTIPLEMATCH_CONFIG, destinations[MULTIPLEMATCH_CONFIG]["filename"], destinations[MULTIPLEMATCH_CONFIG]["path"])
             else:
                 patterns = self.__extract_pattern(
                     destinations[config_name].get("pattern", None))
@@ -91,6 +100,20 @@ class FileDestinationProvider:
         variables (dict): The variables->value dict to be used in the file path and file name format.
         '''

+        # invalid characters dict,
+        # for cleaning up the node names
+        # in order to be able to use them as valid filenames.
+        replacements_dict = {
+            '"': '',
+            '?': '',
+            '<': '',
+            '>': '',
+            '*': '',
+            '|': '-',
+            '/': '-',
+            '\\': '-'
+        }
+
         matching_configs = self._destination_configs.copy()

         # Maybe this can be improved looks like a lot of code duplication ... on the other hand using a
@@ -98,27 +121,27 @@ class FileDestinationProvider:
         if event_type is not None:
             matching_configs = list(filter(lambda config: self.__is_matching_config(
                 config, "event_type", event_type), matching_configs))
-            variables["event_type"] = event_type
+            variables["event_type"] = event_type.translate(str.maketrans(replacements_dict)).strip()

         if event_title is not None:
             matching_configs = list(filter(lambda config: self.__is_matching_config(
                 config, "event_title", event_title), matching_configs))
-            variables["event_title"] = event_title
+            variables["event_title"] = event_title.translate(str.maketrans(replacements_dict)).strip()

         if event_subtitle is not None:
             matching_configs = list(filter(lambda config: self.__is_matching_config(
                 config, "event_subtitle", event_subtitle), matching_configs))
-            variables["event_subtitle"] = event_subtitle
+            variables["event_subtitle"] = event_subtitle.translate(str.maketrans(replacements_dict)).strip()

         if section_title is not None:
             matching_configs = list(filter(lambda config: self.__is_matching_config(
                 config, "section_title", section_title), matching_configs))
-            variables["section_title"] = section_title
+            variables["section_title"] = section_title.translate(str.maketrans(replacements_dict)).strip()

         if document_title is not None:
             matching_configs = list(filter(lambda config: self.__is_matching_config(
                 config, "document_title", document_title), matching_configs))
-            variables["document_title"] = document_title
+            variables["document_title"] = document_title.translate(str.maketrans(replacements_dict)).strip()

         if len(matching_configs) == 0:
             self._log.debug(
@@ -126,15 +149,21 @@ class FileDestinationProvider:
             return self.__create_file_path(self._unknown_file_config, variables)

         if len(matching_configs) > 1:
-            self._log.debug(f"Multiple Destination Patterns where found. Using 'unknown' config! Parameter: event_type:{event_type}, event_title:{event_title},event_subtitle:{event_subtitle},section_title:{section_title},document_title:{document_title}")
-            return self.__create_file_path(self._unknown_file_config, variables)
+            self._log.debug(f"Multiple Destination Patterns where found. Using 'multiple_match' config! Parameter: event_type:{event_type}, event_title:{event_title},event_subtitle:{event_subtitle},section_title:{section_title},document_title:{document_title}")
+            return self.__create_file_path(self._multiplematch_file_config, variables)

         return self.__create_file_path(matching_configs[0], variables)

     def __is_matching_config(self, config: DestinationConfig, key: str, value: str):
+        key_regex = key + '_regex'
         for pattern in config.pattern:
             attribute = getattr(pattern, key)
-            if attribute is None or re.match(attribute, value):
+            attribute_regex = getattr(pattern, key_regex)
+            if attribute is None and attribute_regex is None:
+                return True
+            if attribute and attribute == value:
+                return True
+            if attribute_regex and re.match(attribute_regex, value):
                 return True

         return False
@@ -156,7 +187,12 @@ class FileDestinationProvider:
                                     pattern.get("event_subtitle", None),
                                     pattern.get("event_title", None),
                                     pattern.get("section_title", None),
-                                    pattern.get("document_title", None)))
+                                    pattern.get("document_title", None),
+                                    pattern.get("event_type_regex", None),
+                                    pattern.get("event_subtitle_regex", None),
+                                    pattern.get("event_title_regex", None),
+                                    pattern.get("section_title_regex", None),
+                                    pattern.get("document_title_regex", None)))

         return patterns

I hope this helps, my tests were all successful.

BjBe82 commented 4 months ago

Thanks for the feedback so far. I am a little bit limited in time during the week but will take a look at it over the weekend.

BjBe82 commented 4 months ago

I tied to address all the issues mentioned so far except one thing, the “_regex”-postfix. I am not sure if i like it since from a user perspective It's easy to miss and also in code i am not as happy. Furthermore, I think the main issue is that we need to use "fullmatch" for the regex, and basically if regex is used it must match the full string. I changed this in the current version.

b166er commented 3 months ago

I totally agree with you, your new code change works perfectly for me! My “unknown” and “MultipleMatch” folders are empty, I'm happy. I would love to see your pull-request approved into the main-branch.

Katzmann1983 commented 3 months ago

Do we need the regex at all? I think the code would be easier to maintain without it. When playing with it myself, I just removed all these criteria and the outcome was the same

b166er commented 3 months ago

if i remove the nodes with the regex expressions, then the rule blocks become ambiguous because a missing node outputs a true https://github.com/BjBe82/pytr/blob/Issue-98-Customizable-destination-folder-structure/pytr/file_destination_provider.py#L149 and therefore many documents are stored in the MultiMatch folder.

Example: An event with the following values: event_type: “TRADE_INVOICE”, event_subtitle: “Limit buy order”, section_title: “Documents”, document_title: “Settlement”

Screenshot 2024-07-28 234528

main-loop in https://github.com/BjBe82/pytr/blob/Issue-98-Customizable-destination-folder-structure/pytr/file_destination_provider.py#L110

matching_configs = list(filter(lambda config: self.__is_matching_config(config, “event_type”, event_type), matching_configs))

iterates each block in the config, filters blocks, if a pattern was found in the block, in this example the first and the second block marked as 1 & 2 in red on the screenshot. for each block there is a sub-loop, for each pattern line, to find a specific node (e.g. event_type) in https://github.com/BjBe82/pytr/blob/Issue-98-Customizable-destination-folder-structure/pytr/file_destination_provider.py#L146

    def __is_matching_config(self, config: DestinationConfig, key: str, value: str):
        for pattern in config.pattern:
            attribute = getattr(pattern, key)
            # only checks whether the value of “event_type” matches the input, 
            # but not whether the entire pattern line matches all nodes.
            if attribute is None or re.fullmatch(attribute, value): 
                return True

        return False

each node is currently checked individually and not in connection with other nodes per pattern line, marked as blue numbers in the screenshot. with green square for match / return true. It can therefore happen that a pattern line and therefore a rule block is found although the individual nodes are distributed across the block and are not in a single pattern line. I think this point needs to be reworked so that the nodes are all compared together per pattern row.

my suggestion, we could remove pattern rows where the nodes do not match to avoid checking in the next node loop with pattern rows that already do not match.

diff --git a/pytr/file_destination_provider.py b/pytr/file_destination_provider.py
index 10e5287..e88249c 100644
--- a/pytr/file_destination_provider.py
+++ b/pytr/file_destination_provider.py
@@ -145,1 +145,2 @@ def __is_matching_config(self, config: DestinationConfig, key: str, value: str):
-        for pattern in config.pattern:
+        while len(config.pattern) > 0: # loop until patterns exists
+           pattern = config.pattern[0] # get first pattern line
            attribute = getattr(pattern, key)
            if attribute is None or re.fullmatch(attribute, value):
                return True
+           else: # if value not match
+              config.pattern.pop(0) # remove pattern line from the block to avoid checking for next value
        return False

I also think that the regex option would be a powerful function for selecting dynamic titles in the future. regex comparison doesn't actually take much performance. I am aware that not everyone can use regex. I think users without regex experience will continue to make requests via github-issues to find new documents with the rule config.

BjBe82 commented 3 months ago

I agree with @b166er that regex can be a powerful feature to have finer grained pattern matching. From an end user perspective who just wants to download the docs, I think he expects a yaml file which just works out of the box without ever touching it. Maybe we can provide multiple languages as examples (but this is nice to have and plain translation work).

Also, we will not get this to work without any regex, for example i have this: grafik The only parameter where this can be distinguished is the "document_title" which can be "Abrechnung", "Kosteninformation" and "Kosteninformation" appended with or without a number based on the fact how mayn transaction you have at the same day.

@b166er Ok i see the issue, but we cannot use config.pattern.pop(0). This will modify the loaded configuration and will cause issues for following calls. For example:

provider = FileDestinationProvider()
print(provider.get_file_path("TRADE_INVOICE", None, "Limit-Buy-Order", "Dokumente", "Kosteninformation", {"variable1": "value1"}))
print(provider.get_file_path("STOCK_PERK_REFUNDED", None, "Eingelöst", "Dokumente", "Kosteninformation", {"variable1": "value1"}))
print(provider.get_file_path("TRADE_INVOICE", None, "Limit-Buy-Order", "Dokumente", "Kosteninformation", {"variable1": "value1"}))
print(provider.get_file_path("STOCK_PERK_REFUNDED", None, "Eingelöst", "Dokumente", "Kosteninformation", {"variable1": "value1"}))

will print:

Stocks/Cost report/{iso_date_year}/{iso_date}.{iso_time} {event_title}
Stocks/Cost report/{iso_date_year}/{iso_date}.{iso_time} {event_title}
Unknown/Dokumente/{iso_date}.{iso_time} TRADE_INVOICE - Limit-Buy-Order - Kosteninformation - {event_title}
Unknown/Dokumente/{iso_date}.{iso_time} STOCK_PERK_REFUNDED - Eingelöst - Kosteninformation - {event_title}

We would need a deep copy of the config on each call to avoid this. Since i don't want to deeply copy this for each call, i changed the matching to check all pattern values and only accept full matches. The change is now in git.

Other changes: Patterns which are not found/need to be added or changed:

To be discussed points: (1) A pattern which can be observed is that if we have multiple trades per day, we will have documents with "document_title" that have a number appended. Maybe we should in general add (.\\d+) to all "document_title" entries. This would also resolve entries like this

      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",                section_title: "Dokumente", document_title: "Kosteninformation" }, # Aktiengeschenk
      { event_type: "STOCK_PERK_REFUNDED",  event_subtitle: "Eingelöst",                section_title: "Dokumente", document_title: "Kosteninformation \\d" }, # Aktiengeschenk

to a single entry.

(2) Another thing what i was thinking about is that we might need to add a version number to the yaml file if we want to detect deprecated yaml configurations. For example, if TR changes the texts of the event, event title, subtitle and co we might need to adapt the config. And this change should be reflected somehow and reported to the user which might still use an older version.

Katzmann1983 commented 3 months ago

I think it would be best to remove some details and make both the parser and the yaml file shorter and easier to maintain. Please have a look at this PR https://github.com/BjBe82/pytr/pull/2/files

BjBe82 commented 3 months ago

Fixed the conflicts and accepted your pull request with the following changes:

b166er commented 3 months ago

i have tested the code from the pull-request branch, works perfectly! well done. I would be very happy to find this change in the release as soon as possible. here are my configuration changes to avoid triggering the MultiMatch and Unknown rule: file_destination_config__template.yaml.patch

BjBe82 commented 3 months ago

Applied @b166er patch + appended all document titles with (.\\d+)? just to be safe if there are multiple occurrences of the same file. There might be cases where this never happens, but just to have a consistent pattern, I also applied it to them since there is no downside to it.

b166er commented 2 months ago

Here is my current configuration version.

  1. I have split up the patternlines for “General Meetings”-Documents, to give the file names better/more individual names.
  2. I have split the “notice_stocks” to better/more individualize the file names.
  3. i have migrated the updated “contract_documents” into its own separate block, so that the file names do not get a UUID, but nice names with “updated” as postfix.
  4. I created the "notice_option_contract" pattern line to a separate block, for better filenames from options trading.
  5. I removed some unnecessary fields with values that do not increase the uniqueness of pattern lines. I have removed spaces.
  6. fixed one unknow document: event_type: "ssp_corporate_action_informative_notification", event_subtitle: "Außerordentliche oder spezielle Hauptversammlung", document_title: "Kapitalmaßnahmen", event_title: "Du hast eine Kapitalmaßnahme erhalten"
destination:
  ################################################################################################
  ## Default and fallback patterns used for the destination of a downloaded document.
  ################################################################################################
  # valid for all blocks without explicit filename
  default:
    filename: "{iso_date}.{iso_time} {event_title}" # {event_title} = Wertpapier-/ETF-/Produkt-Name

  # if pattern not found, use this block
  unknown:
    path: "Unknown/{section_title}/"
    filename: "{iso_date}.{iso_time} {event_type} - {event_subtitle} - {document_title} - {event_title}"

  # if pattern found multiple times, use this block
  multiple_match:
    path: "MultipleMatch/{section_title}/"
    filename: "{iso_date}.{iso_time} {event_type} - {event_subtitle} - {document_title} - {event_title}"
  ################################################################################################

  ################################################################################################
  ## Specific patterns for the destination of a downloaded document.
  ################################################################################################
  # stocks
  stock_order_settlement:
    pattern: [
        {event_type: "ORDER_EXECUTED",       document_title: "Abrechnung(.\\d+)?"}, # mit limit verkauft
        {event_type: "TRADE_INVOICE",        document_title: "Abrechnung(.\\d+)?"}, # mit limit gekauft
        {event_type: "STOCK_PERK_REFUNDED",  document_title: "Abrechnung(.\\d+)?"}, # Aktiengeschenk
        {event_type: "SHAREBOOKING",         document_title: "Abrechnung(.\\d+)?"}, # Kapitalmassnahme
      ]
    path: "Stocks/Settlement/{iso_date_year}/"

  stock_order_cost_report:
    pattern: [
        {event_type: "ORDER_CREATED",        document_title: "Kosteninformation(.\\d+)?"}, # limit erstellt
        {event_type: "ORDER_EXECUTED",       document_title: "Kosteninformation(.\\d+)?"},
        {event_type: "TRADE_INVOICE",        document_title: "Kosteninformation(.\\d+)?"},
        {event_type: "STOCK_PERK_REFUNDED",  document_title: "Kosteninformation(.\\d+)?"}, # Aktiengeschenk
        {event_type: "EX_POST_COST_REPORT"},
      ]
    path: "Stocks/Cost report/{iso_date_year}/"

  stock_order_created:
    pattern: [
        {event_type: "ORDER_CREATED",  document_title: "Auftragsbestätigung(.\\d+)?"},
        {event_type: "ORDER_EXECUTED", document_title: "Auftragsbestätigung(.\\d+)?"},
        {event_type: "TRADE_INVOICE",  document_title: "Auftragsbestätigung(.\\d+)?"},
      ]
    path: "Stocks/Order created/{iso_date_year}/"

  stock_order_canceled:
    pattern: [{event_type: "TRADE_CANCELED"},{event_type: "ORDER_CANCELED"},]
    path: "Stocks/Order canceled/{iso_date_year}/"

  order_expired:
    pattern: [{event_type: "ORDER_EXPIRED"}]
    path: "Stocks/Order canceled/{iso_date_year}/"
    filename: "{iso_date} {document_title}"

  # Kapitalmaßnahmen
  stock_notice_1:
    pattern: [{event_type: "EXERCISE"}, {event_type: "SHAREBOOKING", document_title: "Ausführungsanzeige(.\\d+)?"}]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

  # split for better readability
  stock_notice_2:
    pattern: [{event_type: "CORPORATE_ACTION"}, {event_type: "SHAREBOOKING_CANCELED"}]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

  # split for better readability
  stock_notice_3:
    pattern: [
        {event_type: "SHAREBOOKING_TRANSACTIONAL"},
        {event_type: "INSTRUCTION_CORPORATE_ACTION", document_title: "Kundenanschreiben(.\\d+)?"}, #Kapitalerhöhung gegen Bar
      ]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} {event_title}"

  # split for better readability
  stock_notice_4:
    pattern: [
        {event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Spin-off"},
        {event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Reverse Split"},
        {event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Aktiendividende"},
        {event_type: "ssp_corporate_action_invoice_shares",            event_subtitle: "Zwischenvertrieb von Wertpapieren"},
        {event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Wechsel"},
        {event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Information"},
        {event_type: "ssp_corporate_action_informative_notification",  event_subtitle: "Aufruf von Zwischenpapieren"},
      ]
    path: "Stocks/Notice/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle} - {event_title}"

  stock_report:
    pattern: [
        {event_type: "QUARTERLY_REPORT", document_title: "Kontoauszug(.\\d+)?"},
        {event_type: "QUARTERLY_REPORT", document_title: "Depotauszug(.\\d+)?"},
        {event_type: "QUARTERLY_REPORT", document_title: "Cryptoauszug(.\\d+)?"},
      ]
    path: "Stocks/Report/{iso_date_year}/"
    filename: "{iso_date} {document_title} {event_title}"

  # General Meetings
  stock_general_meetings:
    pattern: [
        {event_type: "GENERAL_MEETING", document_title: "Hauptversammlung"},
        {event_type: "ssp_corporate_action_informative_notification", event_subtitle: "Jährliche Hauptversammlung"},
      ]
    path: "Stocks/General Meetings/{iso_date_year}/"

  stock_general_meetings_multiple_files:
    pattern: [{event_type: "GENERAL_MEETING", document_title: "Hauptversammlung \\d+"}]
    path: "Stocks/General Meetings/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_title} - {document_title}"

  stock_special_meetings:
    pattern: [{event_type: "ssp_corporate_action_informative_notification", event_subtitle: "Außerordentliche oder spezielle Hauptversammlung"}]
    path: "Stocks/General Meetings/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} {event_subtitle}"

  # Savings plan
  savings_plan:
    pattern: [{event_type: "SAVINGS_PLAN_INVOICE_CREATED"}, {event_type: "SAVINGS_PLAN_EXECUTED"}, {event_type: "SAVINGS_PLAN_CANCELED"}]
    path: "Stocks/Savings plan/{iso_date_year}/"

  # pre-determined tax base earning
  stock_pre_earning_tax:
    pattern: [{event_type: "PRE_DETERMINED_TAX_BASE_EARNING"}]
    path: "Stocks/PreEarningTax/{iso_date_year}/"

  # Dividends
  dividends_received:
    pattern: [
        {event_type: "CREDIT", event_subtitle: "Dividende"},
        {event_type: "CREDIT", event_subtitle: "Ausschüttung"},
        {event_type: "CREDIT_CANCELED"},
        {event_type: "ssp_corporate_action_invoice_cash"},
      ]
    path: "Dividenden/{iso_date_year}/"

  # Dividends Corporate action election
  dividends_election:
    pattern: [
        {event_type: "INSTRUCTION_CORPORATE_ACTION",                  document_title: "Dividende Wahlweise(.\\d+)?"},
        {event_type: "ssp_dividend_option_customer_instruction",      event_subtitle: "Cash oder Aktie"},
        {event_type: "ssp_corporate_action_informative_notification", event_subtitle: "Dividende Wahlweise"},
      ]
    path: "Dividendelection/{iso_date_year}/"

  # bonds
  bond_repayment:
    pattern: [{event_type: "REPAYMENT"}]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Repayment {event_title}"

  bond_interest:
    pattern: [{event_type: "COUPON_PAYMENT"}]
    path: "Bonds/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Interest {event_title}"

  # Saveback
  saveback_enabled:
    pattern: [{event_type: "benefits_saveback_execution", document_title: "Enabled(.\\d+)?"}]
    path: "Saveback/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Enabled {event_title}" 

  saveback_executed:
    pattern: [{event_type: "benefits_saveback_execution", document_title: "Abrechnung Ausführung(.\\d+)?"}]
    path: "Saveback/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Report {event_title}" 

  saveback_cost_report:
    pattern: [{event_type: "benefits_saveback_execution", document_title: "Kosteninformation(.\\d+)?"}]
    path: "Saveback/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Cost report {event_title}" 

  # Round up
  roundup_enabled:
    pattern: [{event_type: "benefits_spare_change_execution", document_title: "Enabled(.\\d+)?"}] # same files - multiple times at once
    path: "Roundup/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Enabled {event_title}" 

  roundup_executed:
    pattern: [{event_type: "benefits_spare_change_execution", document_title: "Abrechnung Ausführung(.\\d+)?"}]
    path: "Roundup/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Report {event_title}" 

  roundup_cost_report:
    pattern: [{event_type: "benefits_spare_change_execution", document_title: "Kosteninformation(.\\d+)?"}]
    path: "Roundup/{iso_date_year}/"
    filename: "{iso_date}.{iso_time} Cost report {event_title}" 

  # account
  cash_interest:
    pattern: [{event_type: "INTEREST_PAYOUT"}, {event_type: "INTEREST_PAYOUT_CREATED"}]
    path: "Cash Interest/"
    filename: "{iso_date} Report"

  cash_transfer_report:
    pattern: [{event_type: "INCOMING_TRANSFER"}]
    path: "Cash Report/"
    filename: "{iso_date}.{iso_time} {document_title}" # {event_title} = Personal name

  # annual tax report for account
  account_tax_report:
    pattern: [{event_type: "TAX_REFUND"},{event_type: "TAX_ENGINE_ANNUAL_REPORT"},{event_type: "YEAR_END_TAX_REPORT"}]
    path: "Tax/"
    filename: "{iso_date} {document_title}"

  account_tax_adjustment:
    pattern: [{event_type: "ssp_tax_correction_invoice"}, {event_type: "TAX_CORRECTION"}]
    path: "Tax/"
    filename: "{iso_date} {event_title}"

  # common informations
  notice_stocks:
    pattern: [{event_type: "ORDER_CREATED", document_title: "Basisinformationsblatt(.\\d+)?"}]
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {document_title} - {event_title}"

  notice_stocks2:
    pattern: [
        {event_type: "TRADE_INVOICE",         document_title: "Basisinformationsblatt(.\\d+)?"}, 
        {event_type: "GESH_CORPORATE_ACTION", event_subtitle: "Unternehmensmeldung"}
      ]
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {event_subtitle} - {event_title}"

  notice_stocks3:
    pattern: [{event_type: "CUSTOMER_CREATED"}]
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {document_title}"

  notice_option_contract:
    pattern: [{event_type: "ORDER_EXECUTED", document_title: "Basisinformationsblatt(.\\d+)?"}]
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {document_title} Option"

  notice_multiple_documents:
    pattern: [{event_type: "GESH_CORPORATE_ACTION_MULTIPLE_POSITIONS"}] #event_subtitle: Gesellschaftshinweis
    path: "Notice/{iso_date_year}/"
    filename: "{iso_date} {event_subtitle} - {event_title} - {document_title}"

  contract_documents:
    pattern: [
        {event_type: "card_order_billed"}, # Bestellung Trade Republic Karte
        {event_type: "DOCUMENTS_CREATED"}, # Basisinformationen über Wertpapiere
        {event_type: "DOCUMENTS_ACCEPTED"}, # Rechtliche Dokumente: Kundenvereinbarung / Vorvertragliche Informationen / Datenschutzinformationen* / Widerrufsbelehrung* / *Crypto* / Risikohinweise
        {event_type: "DOCUMENTS_CHANGED", section_title: "Dokumente"} # Rechtliche Dokumente: Kundenvereinbarung
      ]
    path: "Contract/"
    filename: "{iso_date} {event_title} - {document_title}"

  contract_documents_updated:
    pattern: [{event_type: "DOCUMENTS_CHANGED", section_title: "Aktualisierte Dokumente"}] # aktualisierte Rechtliche Dokumente: Kundenvereinbarung
    path: "Contract/"
    filename: "{iso_date} {event_title} - {document_title} updated"

here is the result.

in folder /Stocks/General Meetings/XXX/

before:

20xx-yy-zz.aa-bb Company AAA.pdf
20xx-yy-zz.aa-bb Company BBB.pdf
20xx-yy-zz.aa-bb Company CCC (5956d0ed-072c-48d0-8482-1fcd727df063).pdf
20xx-yy-zz.aa-bb Company CCC.pdf
20xx-yy-zz.aa-bb Company DDD.pdf
20xx-yy-zz.aa-bb Company EEE.pdf
20xx-yy-zz.aa-bb Company FFF (05cb8c83-7226-49fa-b2cd-b9ab11172158).pdf
20xx-yy-zz.aa-bb Company FFF.pdf
20xx-yy-zz.aa-bb Company GGG (d5a19969-e29f-43c9-aee8-756a27b7f27d).pdf
20xx-yy-zz.aa-bb Company GGG.pdf
20xx-yy-zz.aa-bb Company HHH.pdf
20xx-yy-zz.aa-bb Company CCC.pdf
20xx-yy-zz.aa-bb Company EEE (67812f93-1de7-4a42-9f1b-7f24e750b9b1).pdf
20xx-yy-zz.aa-bb Company EEE.pdf
20xx-yy-zz.aa-bb Company GGG.pdf

after:

20xx-yy-zz.aa-bb Company AAA.pdf
20xx-yy-zz.aa-bb Company BBB.pdf
20xx-yy-zz.aa-bb Company CCC - Hauptversammlung 1.pdf
20xx-yy-zz.aa-bb Company CCC - Hauptversammlung 2.pdf
20xx-yy-zz.aa-bb Company DDD.pdf
20xx-yy-zz.aa-bb Company EEE.pdf
20xx-yy-zz.aa-bb Company FFF - Hauptversammlung 1.pdf
20xx-yy-zz.aa-bb Company FFF - Hauptversammlung 2.pdf
20xx-yy-zz.aa-bb Company GGG - Hauptversammlung 1.pdf
20xx-yy-zz.aa-bb Company GGG - Hauptversammlung 2.pdf
20xx-yy-zz.aa-bb Company HHH.pdf
20xx-yy-zz.aa-bb Company CCC.pdf
20xx-yy-zz.aa-bb Company EEE - Hauptversammlung 1.pdf
20xx-yy-zz.aa-bb Company EEE - Hauptversammlung 2.pdf
20xx-yy-zz.aa-bb Company GGG.pdf

in folder /Contract/

before:

20xx-yy-zz Kundenvereinbarung (aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa).pdf
20xx-yy-zz Preis- und Leistungsverzeichnis (aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa).pdf
20xx-yy-zz Preis- und Leistungsverzeichnis.pdf
20xx-yy-zz Info zur Einlagensicherung (bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb).pdf
20xx-yy-zz Info zur Einlagensicherung.pdf
20xx-yy-zz Datenschutzinformationen (cccccccc-cccc-cccc-cccc-cccccccccccc).pdf
20xx-yy-zz Datenschutzinformationen.pdf
20xx-yy-zz Hinweise zu den Handelsplätzen (cccccccc-cccc-cccc-cccc-cccccccccccc).pdf
20xx-yy-zz Kundenvereinbarung (cccccccc-cccc-cccc-cccc-cccccccccccc).pdf
20xx-yy-zz Kundenvereinbarung.pdf
20xx-yy-zz Hinweise zu den Handelsplätzen (dddddddd-dddd-dddd-dddd-dddddddddddd).pdf
20xx-yy-zz Hinweise zu den Handelsplätzen.pdf

after:

202x-yy-zz Rechtliche Dokumente - Kundenvereinbarung.pdf
202x-yy-zz Rechtliche Dokumente - Preis- und Leistungsverzeichnis aktualisiert.pdf
202x-yy-zz Rechtliche Dokumente - Preis- und Leistungsverzeichnis.pdf
202x-yy-zz Rechtliche Dokumente - Info zur Einlagensicherung aktualisiert.pdf
202x-yy-zz Rechtliche Dokumente - Info zur Einlagensicherung.pdf
202x-yy-zz Rechtliche Dokumente - Datenschutzinformationen aktualisiert.pdf
202x-yy-zz Rechtliche Dokumente - Datenschutzinformationen.pdf
202x-yy-zz Rechtliche Dokumente - Hinweise zu den Handelsplätzen.pdf
202x-yy-zz Rechtliche Dokumente - Kundenvereinbarung aktualisiert.pdf
202x-yy-zz Rechtliche Dokumente - Kundenvereinbarung.pdf
202x-yy-zz Rechtliche Dokumente - Hinweise zu den Handelsplätzen aktualisiert.pdf
202x-yy-zz Rechtliche Dokumente - Hinweise zu den Handelsplätzen.pdf

in folder /Notice/ for general company announcements

before:

20xx-yy-zz Basisinformationen über Wertpapiere.pdf
20xx-yy-zz Basisinformationsblatt.pdf
20xx-yy-zz Basisinformationsblatt.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben (eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee).pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben (ffffffff-ffff-ffff-ffff-ffffffffffff).pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben 1.pdf
20xx-yy-zz Kundenanschreiben 2.pdf
20xx-yy-zz Kundenanschreiben 1.pdf
20xx-yy-zz Kundenanschreiben 2.pdf
20xx-yy-zz Kundenanschreiben 1.pdf
20xx-yy-zz Kundenanschreiben 2.pdf
20xx-yy-zz Kundenanschreiben 3.pdf
20xx-yy-zz Kundenanschreiben 4.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben.pdf
20xx-yy-zz Kundenanschreiben 1.pdf
20xx-yy-zz Kundenanschreiben 2.pdf
20xx-yy-zz Kundenanschreiben 3.pdf
20xx-yy-zz Kundenanschreiben 4.pdf
20xx-yy-zz Basisinformationsblatt.pdf

after:

202x-yy-zz Basisinformationen über Wertpapiere.pdf
202x-yy-zz Basisinformationsblatt Option.pdf
202x-yy-zz Basisinformationsblatt Option.pdf
202x-yy-zz Unternehmensmeldung - Company AAA.pdf
202x-yy-zz Unternehmensmeldung - Company BBB.pdf
202x-yy-zz Unternehmensmeldung - Company BBB.pdf
202x-yy-zz Unternehmensmeldung - Company CCC.pdf
202x-yy-zz Unternehmensmeldung - Company DDD.pdf
202x-yy-zz Unternehmensmeldung - Company CCC.pdf
202x-yy-zz Unternehmensmeldung - Company CCC.pdf
202x-yy-zz Unternehmensmeldung - Company DDD.pdf
202x-yy-zz Unternehmensmeldung - Company EEE.pdf
202x-yy-zz Unternehmensmeldung - Company FFF.pdf
202x-yy-zz Unternehmensmeldung - Company BBB.pdf
202x-yy-zz Unternehmensmeldung - Company CCC.pdf
202x-yy-zz Unternehmensmeldung - Company GGG.pdf
202x-yy-zz Unternehmensmeldung - Company BBB.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 1.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 2.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 1.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 2.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 1.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 2.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 3.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 4.pdf
202x-yy-zz Unternehmensmeldung - Company HHH.pdf
202x-yy-zz Unternehmensmeldung - Company AAA.pdf
202x-yy-zz Unternehmensmeldung - Company III.pdf
202x-yy-zz Unternehmensmeldung - Company JJJ.pdf
202x-yy-zz Unternehmensmeldung - Company KKK.pdf
202x-yy-zz Unternehmensmeldung - Company LLL.pdf
202x-yy-zz Unternehmensmeldung - Company JJJ.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 1.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 2.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 3.pdf
202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 4.pdf
202x-yy-zz Basisinformationsblatt - Company BBB.pdf

I have created another Feature Request Issue #114 to further improve the file name, before: 202x-yy-zz Gesellschaftshinweis - Mehrere Investments - Kundenanschreiben 1.pdf. after: 202x-yy-zz Gesellschaftshinweis - Company ZZZ - Kundenanschreiben 1.pdf as well as other file names where the company name is not directly contained in the event block of the API response.

BjBe82 commented 2 months ago

Just observed https://github.com/pytr-org/pytr/pull/112 we might lack an Event type "PAYMENT_INBOUND_GOOGLE_PAY". So i just transferred some € via credit card and Google Pay. I added "PAYMENT_INBOUND_GOOGLE_PAY"/"PAYMENT_INBOUND_CREDIT_CARD" and also tested it.

There might still be Apple Pay, but I don't have this to test it.

Katzmann1983 commented 2 months ago

Sorry was in summer vacation. Let me do a check this week and then merge into the new version 0.3

BjBe82 commented 1 week ago

Are we still interested in this, then i would resolve the conflicts again? If not, i would close the pull request.