utf-8 decoding error on installing a new package

[makemeunsee]

Description

I've been getting utf-8 decoding errors when running aurman -S some_package (heimdall for instance).

Stack:

2019-01-18 14:29:48,843 - main - main - ERROR - Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/aurman/main.py", line 1312, in main process(argv[1:]) File "/usr/lib/python3.7/site-packages/aurman/main.py", line 865, in process upstream_system = System(System.get_repo_packages()) File "/usr/lib/python3.7/site-packages/aurman/classes.py", line 1342, in get_repo_packages return Package.get_packages_from_expac("-S", [], PossibleTypes.REPO_PACKAGE) File "/usr/lib/python3.7/site-packages/aurman/classes.py", line 325, in get_packages_from_expac repos = Package.get_known_repos() File "/usr/lib/python3.7/site-packages/aurman/classes.py", line 302, in get_known_repos return [db.name for db in PacmanConfig(conf="/etc/pacman.conf").initialize_alpm().get_syncdbs()] File "/usr/lib/python3.7/site-packages/aurman/classes.py", line 302, in return [db.name for db in PacmanConfig(conf="/etc/pacman.conf").initialize_alpm().get_syncdbs()] UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 1-2: invalid continuation byte

The UnicodeDecodeError varies with each execution; sometimes it's:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 4: invalid start byte and the byte offset varies too.

Expected Behavior

aurman should continue and not crash. I cant prove the content it's decoding is correct, but I've replaced:

@staticmethod
def get_known_repos() -> List[str]:
        [...]
        return [db.name for db in PacmanConfig(conf="/etc/pacman.conf").initialize_alpm().get_syncdbs()]

with

@staticmethod
def get_known_repos() -> List[str]:
        [...]
        pcCfg = PacmanConfig(conf="/etc/pacman.conf")
        alpm = pcCfg.initialize_alpm()
        dbs = alpm.get_syncdbs()
        return [db.name for db in dbs]

and the errors disappear and I can install my package. I've checked issue #88 but my locale is correct and the error is different in the first place (decoding using utf-8, not ascii).

note as well that pacman doesn't even blink an eye, although I guess it's decoding the same data somehow.

Current Behavior

see description above

Possible Solution

?? seems idiotic to split the code line into 3 and magically the error disappears, there has to be a better fix. But it is a workaround at least...

Version of aurman you are using

$ aurman -V :: 2.18.r6.g85d854a-1

Content of aurman config

it's empty

Steps to Reproduce

unsure how to reproduce it, it's just been consistently happening on my setup. I can try to provide as much details on it as possible although I dont know what parts are relevant.

Read the README

I've read the README, paying attention to the fact that no further public development is happening; I'm creating this issue so maybe someone else benefits from me looking into it a little, and maybe devises a better fix, or maybe points me to something wrong on my setup.

Running linux distribution

$ uname -a Linux hostname_xyz 4.20.0-arch1-1-ARCH #1 SMP PREEMPT Mon Dec 24 03:00:40 UTC 2018 x86_64 GNU/Linux

[Reisenfag]

It's broken after last update pyalpm package.

[2019-01-18 15:36] [ALPM] upgraded pyalpm (0.8.4-2 -> 0.8.5-1)

[jelly]

Please provide your pacman.conf, this seems to be a pyalpm issue.

[eli-schwartz]

pacconf --raw > /tmp/config

https://paste.xinu.at/4Rtf5V/

Broken:

$ python -c 'from pycman.config import PacmanConfig; syncdbs = PacmanConfig(conf="/tmp/config").initialize_alpm().get_syncdbs(); print([db.name for db in syncdbs])'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <listcomp>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte

Works:

$ python -c 'from pycman.config import PacmanConfig; alpm = PacmanConfig(conf="/tmp/config").initialize_alpm(); print([db.name for db in alpm.get_syncdbs()])'
['eschwartz', 'custom', 'testing', 'core', 'extra', 'community-testing', 'community', 'multilib-testing', 'multilib', 'xyne-x86_64', 'quarry']

[polygamma]

Interesting problem, thanks for the great issue by the way @makemeunsee . But I am glad that it seems to be no aurman problem ;)

Anyway: If needed I'd surely push a new aurman version containing the workaround. But let's see if @jelly can figure out what's wrong.

[Reisenfag]

Pyalpm updated, and now problem solved.

[redfast00]

Can confirm that this is no longer an issue: had to run aurman -Syu twice: the first time it updated pyalpm and then crashed, the second time it ran without issues.

[Reisenfag]

First start after updating pyalpm or was it updated in the process?

[redfast00]

It was updated in the process.

I think this issue can be closed now?

[Reisenfag]

Yes. Launched many times, works fine.

[makemeunsee]

Hi everyone, Thanks for complementary analysis! This is working fine as well for me, with the latest pyalpm installed. Glad it wasnt aurman failing. Closing the issue!