astanin / python-tabulate

Pretty-print tabular data in Python, a library and a command-line utility. Repository migrated from bitbucket.org/astanin/python-tabulate.
https://pypi.org/project/tabulate/
MIT License
2.08k stars 162 forks source link

UnicodeEncodeError: 'charmap' codec can't encode characters #327

Open charliemb2 opened 2 months ago

charliemb2 commented 2 months ago

Many table formats (tablefmt=) experience the following error when redirecting outputs to STDOUT by using the > character on the command line.

Traceback (most recent call last):
  File "c:\Py.cmb\tables\table_try.py", line 66, in <module>
    print(tabulate(list_o_lists,
  File "C:\Users\Charlie\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-92: character maps to <undefined>

The default table format works, as do some simple ones like pipe, outline. But at least the following exhibit the same or similar error, rounded_outline, simple_outline, fancy_outline, and heavy_outline. There may be others.

Probably related to these failing table formats is that \n newline characters within cells don't format properly. However, the documentation hints at this stating: Most table formats support multiline cell text (text containing newline characters). The newline characters are honored as line break characters. "Most" being the operative word. However, I mention it because it may be possible to hit two birds with one stone.

Environment

Windows 10 Pro Python 3.10 Windows Powershell

Expected behavior:

I'd expect tabulate to at least fall back to a simpler perhaps default table even if this this isn't planned on being fixed. However ideally, this could be fixed and if it can be it should be.

Reproducing the error

To reproduce the error, run this code while redirecting output to STDOUT. I used the command line with e.g., >output.txt.

from tabulate import tabulate

list_o_lists: list = []
# list_o_lists.append(["Outside of Roth accounts _____", ])
list_o_lists.append(["US Small Value", "AVUV", 51683, 23929, -27754])
list_o_lists.append(["Intl Small Value", "AVDV", 25490, 0, -25490])
list_o_lists.append(["Emerging", "DFAI", 20673, 0, -20673])   

header = ["Asset", "Symbol", "Ideal Target", "Existing", "Imbalance +/-"]

print(tabulate(list_o_lists,
               headers=header, 
            #    tablefmt="rounded_outline", 
               tablefmt="simple_outline",
               stralign="right",
               numalign="right", 
               floatfmt=",.0f",  # But you can also pass an f-string from your python code.
               intfmt=",",
               # None of the below work.
            #    maxcolwidths=191
            #    maxcolwidths=[16, None, None, None, None] 
            #    maxcolwidths=[16, 16, 16, 16, 16] 
            #    maxcolwidths=[None, 16, 16, 16, 16] 
              )
      )
charliemb2 commented 2 months ago

redirect_to_stdout_for_error.zip