Import does not seem to work when using script_utils.ingest with hooks

[jord1e]

I currently have a ingest-config.py file:

#!/usr/bin/python3

CONFIG = [
    myimporter.Importer()
]

# This line also does not seem to be working
extract.HEADER = ';; -*- mode: org; mode: beancount; coding: utf-8; -*-\n'

def process_extracted_entries(extracted_entries_list, ledger_entries):
  # do things here
  return extracted_entries_list

scripts_utils.ingest(CONFIG, hooks=[process_extracted_entries])

In my main file i have the following options set:

1970-01-01 custom "fava-option" "import-config" "ingest-config.py"
1970-01-01 custom "fava-option" "import-dirs" "ingest"

When i have the script_utils.ingest method in my config i get the following error when starting Fava:

Running Fava on http://localhost:5000
usage: ingest-config.py [-h] [--version] [--downloads DIR-OR-FILE]
                        {identify,extract,file} ...
ingest-config.py: error: argument command: invalid choice: 'all.beancount' (choose from 'identify', 'extract', 'file'
)

But when i run bean-extract ingest-config.py ingest > tmp.beancount everything works fine. Removing the script_utils.ingest will ignore the hooks that i have defined to clean up my payees and add some metadata. Is there any way i can solve this?

Also:

When i run bean-extract the following gets printed on the first line, is there any way to remove this? Fava gives me errors (which seems strange to me because when i paste the same contents in my main file i do not get any errors, maybe it's a bug):

;; -*- mode: beancount -*-
**** path\to\my\csv

Fava version: 1.17 (Installed using pip on Debian WSL) Browser version: Latest Stable Firefox (83.0 64-bit) Beancount version with which bean-extract works: 2.3.3 (git:36eca6bb; 2020-10-27) (Installed using pip on Windows)

[yagebu]

When i have the script_utils.ingest method in my config i get the following error when starting Fava:

Calling script_utils.ingest turns the whole python file into a script which is expected to be run directly (see https://github.com/beancount/beancount/blob/12e7c3d31d675341f31781c0b801c968ab3e625e/beancount/ingest/scripts_utils.py#L45). So that will conflict with Fava which tries to import this file. You should wrap the call to script_utils.ingest in a if __name__ == "__main__" block so that it isn't called when Fava imports it.

Fava will only see the importers as specified in CONFIG. To apply some function to your imported entries, you should call that function in your importers extract method instead or use some helper like apply_hooks from smart_importer to patch your importer class (https://github.com/beancount/smart_importer/blob/bb896bd437faec0a632e1372f3426b93c11810a7/smart_importer/hooks.py#L27)

[jord1e]

I actually figured it out using Beancount plugins (beancount.fix_payees). I am still wondering why i am getting invalid token errors on the included file though.

[tarioch]

Related to this, is there a way to disable the default hooks?

Unfortunately beancount uses the standard duplicate detector if you don't specify any hooks. This unfortunately marks entries as duplicates although they aren't.

I'm using the apply_hooks from smart importer, so I would not want any hooks from beancount to run.

[yagebu]

Related to this, is there a way to disable the default hooks?

There's not, no. Do you want to open a new issue for that?

[tarioch]

Yes, see #1197