ONSdigital / sdg-build-original

Python package to run data build for SDG indicators
MIT License
0 stars 7 forks source link

Order meta output in indicator ID order #23

Open shaneporter opened 6 years ago

shaneporter commented 6 years ago

The order of indicators specified in meta/all.json is alphanumerically sorted, whereas it should be ordered numberically. This impacts on the sort order of the indicators on the SDGs website (ONSdigital/sdg-indicators#2829)

dougmet commented 6 years ago

It shouldn't be too hard to apply a sort here. Python understands semantic versioning pretty well. So for example you can do:

from distutils.version import LooseVersion
LooseVersion('1.10.1') > LooseVersion('1.5.a')
> True

We can use this to impose a sort order on the list. This ended up being a bit fiddly because sort doesn't like the result of LooseVersion (it compares ints to strings). So I hacked in a hex conversion, which I don't like.

allmeta = [{'id': '1.10.a'},
           {'id': '1.5.1'},
           {'id': '2.5.1'},
           {'id': '1.10.a'},
           {'id':'1.b.2'}]

allmeta.sort(key=lambda x: [int(str(y), 16) for y in LooseVersion(x['id']).version ])

But that puts 1.b before 1.10. It might be better to have a more robust sorter.

dougmet commented 6 years ago

Could fix by having a more complicated key function. Great first issue for @shenavall !

dougmet commented 6 years ago

Arguably no less hacky:

def version_sorter(x):
    v = LooseVersion(x['id']).version
    return [ 999 + int(y,16) if isinstance(y, str) else y  for y in v]

allmeta.sort(key=version_sorter)
dougmet commented 6 years ago

Might be easier to write a simple bubble sorter and use the LooseVersion comparison which appears to work between individual versions. Those are my ideas!