UnicodeEncodeError

[cody-mar10]

1. Installation: mamba create -n genomad -c condo-forge -c bioconda genome
2. Relevant versions:
   • genomad: 1.5.0
   • click: 8.1.3
   • rich: 13.3.2
   • rich-click: 1.6.1
3. Sys info: Linux ubuntu 18 x86_64

There seems to be an encoding issue with the rich console output. I have encountered the following error when running the following commands:

• genomad end-to-end -h
• genomad download-database . after the tarball is downloaded
• PYTHONIOENCODING="utf-8" genomad download-database . error suggests I add an env var to specify encoding, but that did not work

Error stack:

Traceback (most recent call last):
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/modules/download.py", line 93, in main
    console.log(f"Database extracted to [green]{database_path}[/green].")
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/utils.py", line 99, in log
    self.regular_console.log(*args, **kwargs)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 1940, in log
    with self:
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 864, in __exit__
    self._exit_buffer()
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 822, in _exit_buffer
    self._check_buffer()
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 2060, in _check_buffer
    self.file.write(text)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u280f' in position 280: ordinal not in range(256)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/storage1/data14/miniconda3/envs/genomad/bin/genomad", line 10, in <module>
    sys.exit(cli())
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich_click/rich_group.py", line 21, in main
    rv = super().main(*args, standalone_mode=False, **kwargs)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/cli.py", line 335, in download_database
    genomad.download.main(destination, keep, verbose)
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/genomad/modules/download.py", line 91, in main
    with console.status(f"Extracting the database to [green]{database_path}[/green]."):
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/status.py", line 106, in __exit__
    self.stop()
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/status.py", line 91, in stop
    self._live.stop()
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/live.py", line 147, in stop
    with self.console:
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 864, in __exit__
    self._exit_buffer()
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 822, in _exit_buffer
    self._check_buffer()
  File "/storage1/data14/miniconda3/envs/genomad/lib/python3.10/site-packages/rich/console.py", line 2060, in _check_buffer
    self.file.write(text)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u280f' in position 280: ordinal not in range(256)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

[apcamargo]

Hi @cody-mar10

Thanks for the detailed report.

Right now I'm away with no access to my computer, so I can't investigate the issue. Some thoughts:

• You can download the database here: https://portal.nersc.gov/genomad/__data__/genomad_db_v1.2.tar.gz
• Try to run the end-to-end command with the --quiet flag. Maybe you won't have the issue if the characters are not drawn to the terminal.
• Did you try using another terminal application? Does the one you use allow you to change the encoding?

[cody-mar10]

Using a different terminal solved the issue. Whenever connecting to my group's linux server using the default MacOS terminal, my LANG environment variable became set to en_US. Using a different terminal (VS Code terminal), LANG="en_US.UTF-8, which had no issues, and I just set that as a permanent environment variable. Thanks for the suggestion

[apcamargo]

No problem! Good to hear you solved the issue :)