iati-data-access / data-backend

GNU Affero General Public License v3.0
2 stars 0 forks source link

Look into removing `re.match` by having `iatiflattener` produce the required column #14

Open simon-20 opened 1 year ago

simon-20 commented 1 year ago

See: https://github.com/iati-data-access/data-backend/blob/main/iatidatacube/import_data.py#L273

    ro_row = row['reporting_org#en']
    if ro_row not in reporting_orgs:
        ro_row_ref = get_groups_or_none(re.match(r"(.*) \[(.*)\]",
                ro_row), 1)

Currently iati_flattener provides a column with content such as African Development Bank [XM-DAC-46002] which has to be split by regex.

From @markbrough: "It would probably be useful if iati-flattener would add a column reporting_org_ref which would e.g. just contain XM-DAC-46002 rather than African Development Bank [XM-DAC-46002] - I would imagine this line adds some unnecessary overhead."