Encoding Issue?

[MDKaiser]

First time using the importer, so this may be user error or specific to my system...

The csv downloaded from DKB is encoded with ISO 8859-1, so I configured the import.config as: file_encoding='ISO-8859-1',

My .beancount file is encoded with utf-8, defined by its first line: ;; -*- mode: beancount; coding: utf-8; fill-column: 400; -*-

When running the importer with bean-extract path_to_config path_to_csv >> path_to_beancount, the content of my csv is parsed and appended, however there seems to be some sort of encoding mismatch, see attached screenshot. Visual Studio Code guesses that the generated encoding is UTF-16 LE (opening with that encoding, the space and line breaks appear correct, however Umlaute are incorrect, so something is off there as well)

[siddhantgoel]

Are you running the importer on a Windows system or a POSIX one?

[MDKaiser]

Windows 10.

[siddhantgoel]

Another question: if you don't pipe the result to path_to_beancount and just print it to the console, do those characters still show up?

[MDKaiser]

Interesting: So far I've used "powershell" (default in Visual Studio Code) to call on bean-extract. This produces the symbol-issue shown above. When piping into the console, everything appears normal however.

After your comment, I've also tried running bean-extract in a Windows command prompt and piping into the .beancount file: This works (almost)! Umlaute are still messed up (like mentioned in the last sentence of my first comment).

[siddhantgoel]

Hey, I'm not sure if we need to keep this issue around? To me this sounds like something with the development environment. Especially because the methods inside this package are returning plain Python data structures. The actual job of piping the output is done inside bean-extract so Beancount's mailing list might be more helpful in this regard.

[siddhantgoel]

I'll mark this as closed for now.

[asteriuscz]

Just for the other who need to find a way on Win10. bean-extract config.py transaction.csv | Out-File -Encoding utf8 -Append ledger.beancount

will do the work for you