search

akopulko / ffiiitc

FireFly III Transactions Classifier
MIT License
37 stars 9 forks source link

Potential Enhancement Idea(s) #14

Open vorwd opened 9 months ago

vorwd commented 9 months ago

Firstly, great tool you've implemented here.. I was worried I'd have to go through all my transactions to classify them manually -- this tool has thankfully provided me with some relief, so thank you for that!

Testing the tool last night, I noticed a fair number of incorrect classifications, which, from what I can tell are due to the descriptions being fairly ambiguous...

E.g. "PP*9658CODE 4029357733 AUS" was classified as "grocery" but none of the grocery stores I've shopped at even have a remotely similar description to this.
E.g. "INTEREST CHARGE" being classified as "grocery", which again, doesn't align with my grocery store descriptions.

I don't fully understand the statistical theory used in this tool, as I am not at that level, but also can't understand how it found a correlation between these based on previous transactions of stores with "Coles" and "Woolworths" in the description.

Is there a potential to include a "level of certainty" threshold that would prevent the tool from adding the category if it fell below a level of certainty in its matching? Setting this as a environment variable would be ideal, if possible. Percentage or numerical scale?

-- Further to this, an option to tag the transactions that were categorized by the tool. Being able to identify them and review if needing an edit, due to the mismatching that was discovered would be ideal. This would be an added tag, in addition to the tagging completed by the ffiii importing tool already, thus resulting in 2+ tags. E.g. tagging them with "ffiiitc categorised on date @ time"

Then, if the tool runs against a transaction that falls below the "level of certainty" value configured, it would add a tag as well, for easy identification of needing manual intervention. E.g. tagging them with "ffiiitc categorisation aborted on date @ time"

If the complexity to implement the "certainty threshold" was not possible, the ability to still tag them as being categorised by ffiiitc would be ideal for easy review and management.

Supporting details: 58 pages of transactions (sets of 50) used for training.

Batch of "Grocery" entries that were self-categorised by me and in the set of training transactions.

DESCRIPTION | GROUP | DETAILED WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx8907 Value Date: 28/07/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 11/08/2021 | FOOD AND DINING | GROCERY COLES ONLINE HAWTHORN EAST AUS Card xx7785 Value Date: 14/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 14/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 15/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1149 MARRICKVILLE NS AUS Tap and Pay xx9245 Value Date: 17/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx4424 Value Date: 18/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx4424 Value Date: 18/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 22/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 23/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 27/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1149 MARRICKVILLE NS AUS Tap and Pay xx9245 Value Date: 29/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 27/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1149 MARRICKVILLE NS AUS Tap and Pay xx9245 Value Date: 28/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Tap and Pay xx9245 Value Date: 31/08/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 05/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 04/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 07/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7777 Value Date: 11/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7777 Value Date: 13/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7777 Value Date: 16/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 17/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 19/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 17/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 22/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7777 Value Date: 23/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 24/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 27/09/2021 | FOOD AND DINING | GROCERY THEOS CECINAS HURLST HURLSTONE PAR NS AUS Card xx7785 Value Date: 27/09/2021 | FOOD AND DINING | GROCERY THEOS CECINAS HURLST HURLSTONE PAR NS AUS Card xx7777 Value Date: 28/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1149 MARRICKVILLE NS AUS Card xx7777 Value Date: 28/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 30/09/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7777 Value Date: 01/10/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1149 MARRICKVILLE NS AUS Card xx7785 Value Date: 04/10/2021 | FOOD AND DINING | GROCERY THEOS CECINAS HURLST HURLSTONE PAR NS AUS Card xx7785 Value Date: 05/10/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 09/10/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7777 Value Date: 12/10/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 14/10/2021 | FOOD AND DINING | GROCERY COLES 0791 ASHFIELD AU AUS Card xx7785 Value Date: 19/10/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx6424 Value Date: 21/10/2021 | FOOD AND DINING | GROCERY WOOLWORTHS 1034 CANTERBURY NS AUS Card xx7785 Value Date: 21/10/2021 | FOOD AND DINING | GROCERY

Jazzmax commented 9 months ago

I like the idea of adding tags but I would not limit to just identify the categoriser itself. It actually can extend categorisation by adding meaningful tags. For example, category: Enjoyment, can have also have more details in tags, e.g. Dining, Concert, or Alcohol...