malariagen / ag1000g-phase3-data-paper

Other
1 stars 2 forks source link

bug in calculation of location-colours #75

Open leehart opened 2 years ago

leehart commented 2 years ago

WRT https://github.com/malariagen/ag1000g-phase3-data-paper/blob/master/notebooks/popstructure_calc-location-colours.ipynb

Calculation of scaled_lat and scaled_long in latlong_to_rgb_hex_via_lab() assume lat and long are int, but may be pd.core.series.Series.

lat and long are int in spot checks, so the diagnostics miss this.

Results in miscalculation of colour space, when lat and long are pd.core.series.Series, not always noticeable.

leehart commented 2 years ago

There also seems to be something else wrong with the calculation of scaled_lat and scaled_long, resulting in values outside the expected range of 0 to 1.

    # Scale the lat-long 0 to 1, relative to the specified min and max
    lat_extent = max_lat - min_lat
    long_extent = max_long - min_long
    scaled_lat = (lat + abs(min_lat)) / lat_extent
    scaled_long = (long + abs(min_long)) / long_extent
    # lat, lng: -1.919 18.934
    # min_lat, max_lat, min_lng, max_lng: -11.579 3.334 14.435 29.187
    # lat_extent, lng_extent: 14.913 14.752
    # scaled_lat, scaled_lng: 0.6477569905451619 2.261998373101952
leehart commented 2 years ago

Looks like the calculation should be corrected to something like:

    # Scale the lat-long 0 to 1, relative to the specified min and max
    lat_extent = max_lat - min_lat
    long_extent = max_long - min_long
    scaled_lat = (lat - min_lat) / lat_extent
    scaled_long = (lng - min_long) / long_extent
lat, lng: -1.919 18.934
min_lat, max_lat, min_lng, max_lng: -11.579 3.334 14.435 29.187
lat_extent, lng_extent: 14.913 14.752
scaled_lat, scaled_lng: 0.6477569905451619 0.3049755965292842