bvanelli / actualpy

Python API implementation for Actual server - reference https://actualbudget.org/
https://actualpy.readthedocs.io/en/latest/
28 stars 7 forks source link

reconcile_transaction() somehow breaks the "rules" functionality on actual #68

Closed tcpr1 closed 3 weeks ago

tcpr1 commented 1 month ago

Checks

Reproducible example

from actual import Actual
from actual.queries import get_accounts, get_transactions, reconcile_transaction

csv_data = [["Date","Payee","Notes","Category","Amount","Cleared", "imported_ID"]]

with Actual(base_url=URL_ACTUAL, password=PASSWORD_ACTUAL, file=FILE_ACTUAL) as actual:

accounts = get_accounts(actual.session)
for account in accounts:

     for row in csv_data:
        # here, we define the basic information from the file
        date, payee, notes, category, amount, cleared, imported_ID = (
            datetime.strptime(row[0], "%Y-%m-%d").date(),  # transform to date
            row[1],
            row[2],
            row[3],
            decimal.Decimal(row[4]),
            row[5] == "Cleared",
            row[6],
        )
        t = reconcile_transaction(
            actual_session,
            date,
            account,
            payee,
            notes,
            category,
            amount,
            imported_ID,
            cleared=cleared,
            imported_payee=payee,
            already_matched=added_transactions,
        )
        added_transactions.append(t)
actual.commit()

Log output

No response

Issue description

I am using reconcile_transaction() to import bank data into Actual. Everything seems to be working fine, but the "rules" funcionality to rename payees do not work.

One thing I also noticed is when I import .ofx or .csv inside Actual, the payee name gets bolded, but it is not the case for the payees names of imported transactions through this API.

Expected behavior

After actual.commit() the newly created transactions should be renamed based on Actual rules automatically.

Installed versions

bvanelli commented 1 month ago

I understand your use case, maybe adding a run_rules on commit would be enough, but it's not optimal because this API tries to mimic adding entries via frontend, and if you add a transaction via frontend, no rule will be run (as far as I could test).

Wouldn't something like this work for you?

from actual import Actual
from datetime import datetime
import decimal
from actual.queries import get_accounts, reconcile_transaction, get_ruleset

csv_data = [["Date", "Payee", "Notes", "Category", "Amount", "Cleared", "imported_ID"]]

URL_ACTUAL = "http://localhost:5006"
PASSWORD_ACTUAL = "mypass"
FILE_ACTUAL = "CSV Importer"

with Actual(base_url=URL_ACTUAL, password=PASSWORD_ACTUAL, file=FILE_ACTUAL) as actual:
    accounts = get_accounts(actual.session)
    ruleset = get_ruleset(actual.session)
    added_transactions = []
    for account in accounts:
        for row in csv_data:
            # here, we define the basic information from the file
            date, payee, notes, category, amount, cleared, imported_ID = (
                datetime.strptime(row[0], "%Y-%m-%d").date(),  # transform to date
                row[1],
                row[2],
                row[3],
                decimal.Decimal(row[4]),
                row[5] == "Cleared",
                row[6],
            )
            t = reconcile_transaction(
                actual.session,
                date,
                account,
                payee,
                notes,
                category,
                amount,
                imported_ID,
                cleared=cleared,
                imported_payee=payee,
                already_matched=added_transactions,
            )
            actual.session.flush()  # flush to load the ids and relationships
            ruleset.run(t)  # run the rule here
            added_transactions.append(t)
    actual.commit()
tcpr1 commented 1 month ago

It worked now! Appreciate your help!

tcpr1 commented 1 month ago

Caught a new bug, now it's when running ruleset.run(t). The code example is basically the same as you sent above. The error happens when my budget has rules with "imported payee - any option - name" (similar to issue #67) or has transactions automatically added by schedules rules.

This raised due to scheduled transactions. Ir only happens then the date from a transaction "t" is equal a trasaction added by a schedule. When I delete the schedule, it stops the error:

AttributeError: 'NoneType' object has no attribute 'date'

Traceback: File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling result = func() ^^^^^^ File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 590, in code_to_exec exec(code, module.dict) File "/app/streamlit_app.py", line 91, in pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date) File "/app/functions.py", line 256, in pluggy_sync data_to_actual(csv_data, actual.session, account) File "/app/functions.py", line 163, in data_to_actual ruleset.run(t) # run the rule here ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 555, in run the order 'pre' -> None -> 'post'. You can provide a value to run only a certain stage of rules.""" ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 544, in _run for t in transaction: File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 495, in run def run(self, transaction: Transactions) -> bool: ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 490, in evaluate def evaluate(self, transaction: Transactions) -> bool: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 490, in def evaluate(self, transaction: Transactions) -> bool: ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 302, in run attr = get_attribute_by_table_name(Transactions.tablename, self.field) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/rules.py", line 185, in condition_evaluation

https://github.com/actualbudget/actual/blob/98a7aac73667241da350169e55edd2fc16a6687f/packages/loot-core/src/server/accounts/rules.ts#L302-L304

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/site-packages/actual/schedules.py", line 166, in is_approx before = self.before(date) ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/schedules.py", line 233, in before return self.do_skip_weekend(dt_start, before_datetime).date() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This raised due to " imported payee - any option - name " rule. Deleting the rule, stops the error:

ValidationError: 1 validation error for list[function-after[check_operation_type(), function-after[convert_value(), Condition]]] 0.type Input should be 'date', 'id', 'string', 'number' or 'boolean' [type=enum, input_value='imported_payee', input_type=str] For further information visit https://errors.pydantic.dev/2.9/v/enum

Traceback: File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling result = func() ^^^^^^ File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 590, in code_to_exec exec(code, module.dict) File "/app/streamlit_app.py", line 91, in pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date) File "/app/functions.py", line 256, in pluggy_sync data_to_actual(csv_data, actual.session, account) File "/app/functions.py", line 136, in data_to_actual ruleset = get_ruleset(actual_session) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/actual/queries.py", line 604, in get_ruleset :param s: session from Actual local database. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydantic/type_adapter.py", line 135, in wrapped return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pydantic/type_adapter.py", line 384, in validate_json return self.validator.validate_json(data, strict=strict, context=context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

bvanelli commented 1 month ago

Do your schedule happens to have an end date? Could you share how it is defined?

tcpr1 commented 1 month ago

Sure, here it is:

image

image

bvanelli commented 1 month ago

I generated a new release, could you update and check if it still fails?

tcpr1 commented 1 month ago

It failed 😔 However, there is improvement! It did not failed now because of the "imported payee" rule. It only fails because of a scheduled rule.

Oh, while typing here I was testing to find exactly what was causing the error and I found it! It's this date rule inside the schedule: image

It was set to "before". Changing to "after" or unselecting it stops the error.

The error output:

> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> Cell In[4], [line 15](vscode-notebook-cell:?execution_count=4&line=15)
>      [13](vscode-notebook-cell:?execution_count=4&line=13) start_date = "2024-09-08"
>      [14](vscode-notebook-cell:?execution_count=4&line=14) end_date = "2024-09-10"
> ---> [15](vscode-notebook-cell:?execution_count=4&line=15) pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date)
> 
> File [z:\docker\code-server\config\actual-pluggy-py\functions.py:257](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:257), in pluggy_sync(URL_ACTUAL, PASSWORD_ACTUAL, FILE_ACTUAL, start_date, end_date)
>     [255](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:255)             if pluggy_status: # if pluggy connection failed, do nothing
>     [256](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:256)                 print(f"Starting Actual reconciliation for {accName}")
> --> [257](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:257)                 data_to_actual(csv_data, actual.session, account)
>     [259](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:259) actual.commit()
> 
> File [z:\docker\code-server\config\actual-pluggy-py\functions.py:164](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:164), in data_to_actual(csv_data, actual_session, account)
>     [162](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:162) actual_session.flush()  # flush to load the ids and relationships
>     [163](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:163) # print(t)
> --> [164](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:164) ruleset.run(t)  # run the rule here
>     [165](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:165) added_transactions.append(t)
>     [167](file:///Z:/docker/code-server/config/actual-pluggy-py/functions.py:167) if t.changed():
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:594](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:594), in RuleSet.run(self, transaction, stage)
>     [592](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:592) if stage == "all":
>     [593](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:593)     self._run(transaction, "pre")
> --> [594](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:594)     self._run(transaction, None)
>     [595](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:595)     self._run(transaction, "post")
>     [596](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:596) else:
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:583](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:583), in RuleSet._run(self, transaction, stage)
>     [581](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:581)         rule.run(t)
>     [582](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:582) else:
> --> [583](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:583)     rule.run(transaction)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:534](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:534), in Rule.run(self, transaction)
>     [531](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:531) def run(self, transaction: Transactions) -> bool:
>     [532](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:532)     """Runs the rule on the transaction, calling evaluate, and if the return is `True` then running each of
>     [533](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:533)     the actions."""
> --> [534](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:534)     if condition_met := self.evaluate(transaction):
>     [535](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:535)         splits = self.set_split_amount(transaction)
>     [536](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:536)         if splits:
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529), in Rule.evaluate(self, transaction)
>     [527](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:527) """Evaluates the rule on the transaction, without applying any action."""
>     [528](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:528) op = any if self.operation == "or" else all
> --> [529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529) return op(c.run(transaction) for c in self.conditions)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529), in <genexpr>(.0)
>     [527](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:527) """Evaluates the rule on the transaction, without applying any action."""
>     [528](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:528) op = any if self.operation == "or" else all
> --> [529](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:529) return op(c.run(transaction) for c in self.conditions)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:322](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:322), in Condition.run(self, transaction)
>     [320](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:320) true_value = get_value(getattr(transaction, attr), self.type)
>     [321](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:321) self_value = self.get_value()
> --> [322](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:322) return condition_evaluation(self.op, true_value, self_value, self.options)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\rules.py:200](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:200), in condition_evaluation(op, true_value, self_value, options)
>     [198](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:198)     interval = datetime.timedelta(days=2)
>     [199](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:199)     if isinstance(self_value, Schedule):
> --> [200](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:200)         return self_value.is_approx(true_value, interval)
>     [201](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:201) else:
>     [202](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:202)     # Actual uses 7.5% of the value as threshold
>     [203](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:203)     # https://github.com/actualbudget/actual/blob/243703b2f70532ec1acbd3088dda879b5d07a5b3/packages/loot-core/src/shared/rules.ts#L261-L263
>     [204](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/rules.py:204)     interval = round(abs(self_value) * 0.075, 2)
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\schedules.py:166](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:166), in Schedule.is_approx(self, date, interval)
>     [164](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:164) if date < self.start or (self.end_mode == EndMode.ON_DATE and self.end_date < date):
>     [165](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:165)     return False
> --> [166](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:166) before = self.before(date)
>     [167](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:167) after = self.xafter(date, 1)
>     [168](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:168) if before and (before - interval <= date <= before + interval):
> 
> File [c:\git\actualpy\.conda\Lib\site-packages\actual\schedules.py:233](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:233), in Schedule.before(self, date)
>     [231](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:231) if not before_datetime:
>     [232](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:232)     return None
> --> [233](file:///C:/git/actualpy/.conda/Lib/site-packages/actual/schedules.py:233) return self.do_skip_weekend(dt_start, before_datetime).date()
bvanelli commented 1 month ago

Hello @tcpr1, thanks for your last message, I could finally reproduce the reason why it was crashing. Turns out I was comparing the wrong cutoff date when resolving the before weekend mode.

It should be fixed in the soon to be released version 0.5.1.

tcpr1 commented 1 month ago

No more errors!

bvanelli commented 3 weeks ago

Hey @tcpr1 , I'll then close this issue with https://github.com/bvanelli/actualpy/pull/85 after including more documentation and one example using your use case as a base.