isaacharrisholt / quiffen

Quiffen is a Python package for parsing QIF (Quicken Interchange Format) files.
MIT License
34 stars 30 forks source link

How to add support for custom tag? #16

Closed kapitan-iglu closed 2 years ago

kapitan-iglu commented 2 years ago

I'm trying to use this module to create QIF files that can be imported to KMyMoney. I has to use QIF file format because of split transactions (other formats like CSV nor OFS doesn't supports splits). But in QIF there is no unique identifier for each transaction, so KMyMoney implemented custom tag # to overcome this drawback. I want to use it to be able to not export already imported transactions to KMyMoney.

I assume that quiffen does not support this tag yet, so my question is:

How to add support for custom tag?

Maybe in some universal way like Transaction.add_tag({'#': value})?

amaten69 commented 2 years ago

while i agree this is a total hackjob what i did, till this feature is added, is i created the tags as children of category and did a string replace.

Since the qif format for tags is: LCategory1:SubCategory/Tag1:Tag2

just do (only putting the important parts of the code) tag_marker = "TAG_MARKER" cat1 = quiffen.Category('Category1') cat2 = quiffen.Category('SubCategory') tag1 = quiffen.Category(f''{tag_marker}Tag1') tag2 = quiffen.Category(''Tag2') cat1.add_child(cat2) cat2.add_child(tag1) tag1.add_child(tag2) tr = quiffen.Transaction(date=date,amount=amount,category=tag2) qif.to_qif().replace(f':{tag_marker}','/')

or to make it a little less bulky you can make the tag part a def tag_marker = "TAG_MARKER" cat1 = quiffen.Category('Category1') cat2 = quiffen.Category('SubCategory') cat1.add_child(cat2) tags = ["tag1","tag2"] tr = quiffen.Transaction(date=date,amount=amount,category=add_tags(cat2,tags,tag_marker)) qif.to_qif().replace(f':{tag_marker}','/')

def add_tags(category,tags,tag_marker):
    initial = True
    for tag in tags:
        if initial:
            this_tag = quiffen.Category(tag_marker+tag)
            category.add_child(this_tag)
            initial = False
        else:
            this_tag = quiffen.Category(tag)
            last_tag.add_child(this_tag)
        last_tag = this_tag

    return this_tag
amaten69 commented 2 years ago

(or really what i did is build categories and tags into a function that accecpts lists since hierarchy is kinda bulky in general)

categories = ["Shopping","Electronics"]
tags = ["tag1","tag2","tag3"]
tag_marker = "TAG_MARKER"

tr = quiffen.Transaction(date=date,
                                amount=amount,
                                category=build_categories_and_tags(
                                            categories=categories,
                                            tag_marker=tag_marker,
                                            tags=tags
                                            )
                                )
print(qif.to_qif().replace(f':{tag_marker}','/'))

def build_categories_and_tags(categories=[],tag_marker="",tags=[]):
    initial_category = True
    last_category = None
    last_tag = None
    for category in categories:
        if initial_category:
            this_category = quiffen.Category(category)
            initial_category = False
        else:
            this_category = quiffen.Category(category)
            last_category.add_child(this_category)
        last_category = this_category

    initial_tag = True
    for tag in tags:
        if initial_tag:
            this_tag = quiffen.Category(tag_marker+tag)
            if last_category is not None:
                last_category.add_child(this_tag)
            initial_tag = False
        else:
            this_tag = quiffen.Category(tag)
            last_tag.add_child(this_tag)
        last_tag = this_tag

    if last_tag is not None:
        return this_tag
    else:
        return this_category
isaacharrisholt commented 2 years ago

There should be a much easier way of implementing this feature into the code using class variables and the __dict__ property of the class instances.

amaten69 commented 2 years ago

There should be a much easier way of implementing this feature into the code using class variables and the __dict__ property of the class instances.

im not exactly sure what you mean. this is of course a workaround solution to not have to modify the module itself (pretty dirty one really).

if youre talking about a better workaround using class dict objects im really not totally sure how one would do that (im not even real sure for which piece i mentioned here were talking about to use that).

however yeah this doesnt seem like a super complex feature to add. looks well setup to add a method or a transaction class input. its just enough of a core change its probably better you do it to keep with how'd you would want it done.

It also looks like / its used in investment transactions as the transfer account, so youll have to account for that too.

rkaramc commented 2 years ago

I'm trying to use this module to create QIF files that can be imported to KMyMoney. I has to use QIF file format because of split transactions (other formats like CSV nor OFS doesn't supports splits). But in QIF there is no unique identifier for each transaction, so KMyMoney implemented custom tag # to overcome this drawback. I want to use it to be able to not export already imported transactions to KMyMoney.

... to be able to not export[import???] already imported transactions ...

@kapitan-iglu Re:need for an ID in the qif file for KMyMoney, you may be able to use check_number(tagged N) field to uniquely identify a transaction. If you can use N, you'd avoid having to create a new tag.

The check number is typically used to match/reconcile the bank statements with transactions in accounting software. Seems to me that is your intent here?

edit: assuming you haven't found a solution/implemented one already!! :-)

edit: This issue "resonates" with me a lot! Especially as I am just coming off a 2-day effort to automate converting my bank's statement PDFs (password-protected PDF rcvd by email attachment) to TXT to QIF imported into YNAB4! AND then had to manually reconcile some transactions because the bank did not have useful check_numbers/identifiers. Of course, having to review each transaction opened my eyes to the number of food delivery purchases in my account ;-) !!!!

@isaacharrisholt Thanks for publishing this package!! Saved me a lot of time!

isaacharrisholt commented 2 years ago

@kapitan-iglu @amaten69 @rkaramc I'm currently working on v2 of Quiffen, and I can confirm that it will support this functionality. The current syntax is looking to be something like the following:

from quiffen import Transaction

Transaction.add_custom_field(
    line_code='#',
    attr='unique_id',
    field_type=str,
)

# Parse your Qif here...

# Then you can access the the unique ID of your transactions
print(t.unique_id)

This will also use the Pydantic library to coerce the field value into the type specified, so something like the following is possible:

from datetime import datetime
from decimal import Decimal

from quiffen import Transaction

Transaction.add_custom_field(
    line_code='DT',  # Custom fields support multi-character line codes
    attr='timestamp',
    field_type=datetime,
)
Transaction.add_custom_field(
    line_code='UT',
    attr='unix_timestamp',
    field_type=Decimal,
)

# Parse file...

print(type(t.timestamp) == datetime)  # prints True
print(type(t.unix_timestamp) == Decimal)  # prints True
kapitan-iglu commented 2 years ago

@rkaramc No, I haven't implemented new custom record. Finally I reverted to raw generation of QIF file in my app to be able to include custom tag #. The content of N tag is in KMyMoney stored as checkNumber per each split inside one transaction. So each split may have different N tag(?) Also, KMM assumes this field as integer and generates (sometimes) it's value automatically when manually adding transaction (increments last used value). So there is a chance for collision. I do not know how checkNumber should really work, this is only my observation.

On transaction level there is bankId field, which:

@isaacharrisholt Looks good! That will allow new use cases for your lib (like adding/converting records in existing file, storage of custom data in "hidden" form and so on...).

isaacharrisholt commented 2 years ago

Added in #36 as part of Quiffen v2