Base64 attributes caused the update process to crash.

[lfield]

Our top-level BDII update process failed with the following error:

Traceback (most recent call last):
  File "/usr/sbin/bdii-update", line 1031, in <module>
    main(config, log)
  File "/usr/sbin/bdii-update", line 622, in main
    new_ldif = fix(new_dns, new_ldif)
  File "/usr/sbin/bdii-update", line 517, in fix
    entry = convert_entry(ldif[dns[dn][0]:dns[dn][1]])
  File "/usr/sbin/bdii-update", line 358, in convert_entry
    value = base64.b64decode(line[index + 2:].strip()).decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 2: ordinal not in range(128)

The issue was due to a site publishing the latitude and longitude in the wrong format. e.g. 50°04'07.5"N resulting in the following attributes.

GLUE2LocationLatitude:: NTDCsDA0JzA3LjUiTg==
GLUE2LocationLongitude:: MTnCsDU0JzMyLjUiRQ==

[adriansev]

on centos 7 the upgrade from python-2.7.5-88 to -89 (and 90) breaks this part of code. (so i suspect soon to be a flood of messages as it breaks the ehi nagios probes)

[andrea-manzi]

Hello, @lfield thanks for reporting this! we should better handle the decoding in that part of the code ( using utf-8 and maybe ignoring the decoding errors) @adriansev we checked that python update and we could not find how it could break the code there. Can you confirm that using the latest BDII from EPEL and python-2.7.5-88 you don't have issues, and only when upgrading python you have errors? thanks Andrea

[adriansev]

Hi @andrea-manzi well, i would like to not break again my arc-ce .. but after a full update the steps to make it works again were this: with python 2.7.5-90 : not working downgrade python 2.7.5-90 to -89 : still not working downgrade to -88 : still not working downgrade bdii : problem solved (with python 2.7.5-88) so, the the only working bdii was 5.2.25-2 with python 2.7.5-88, bdii 5.2.26-1 was not working no matter the python version

[lfield]

@adriansev could it be that the site fixed the issue in the meantime while you were investigating and it just happened to coincide with your downgrade?

[adriansev]

@lfield erm, sorry, i do not understand... i updated my arc-ce, then nagios test started to fail. i pinpointed in the logs the breakage of bdii-update script then downgraded versions as above until the nagios probes worked again.. (this was ~21:00 EET last night)

[lfield]

During my investigation I contacted the site who was publishing the base64 encoded latitude and longitude. They stopped publishing these values and it started working for me without any downgrades. I was just wondering if your downgrades coincided with the site intervention and hence it looked liked it was the downgrade which fixed the issue but actually it was the site. The change happened between 13:40 and 13:50 on 04 Jan.

[andrea-manzi]

an update on this:

The issue you found is related to the version of BDII released to EPEL7, which is different from the one tagged here and released to UMD.

The package on EPEL7 includes a patch for py3 ( which has then been merged to this repo too), but it caused the issue you reported when running with py2

I have asked Matthias to remove that patch from EPEL7 for now.

the next version of BDII will be py3 only so the code that is failing now will be fine ( as decode/encode functions by default are using utf-8 on py3)

[andrea-manzi]

@adriansev i don't know if you can do this test but it would be great if you could try the BDII package available in UMD ( https://repository.egi.eu/sw/production/umd/4/centos7/x86_64/updates/) . The same code will be soon available in EPEL7 testing.

[brucellino]

@lfield I suspect this issue is obsolete - can we close it?

[lfield]

Fine for me.

[brucellino]

Closed obsolete