CliDyn / climsight

prototype of a system that provide local climate information
BSD 3-Clause "New" or "Revised" License
27 stars 9 forks source link

A couple of population improvements #27

Open koldunovn opened 7 months ago

koldunovn commented 7 months ago

In the current main, there is a warning:

DtypeWarning: Columns (2) have mixed types. Specify dtype option on import or set low_memory=False.
  pop_dat = pd.read_csv(pop_path)

Can we specify dtype when open the data? Probably will five us some speedup.

Here https://github.com/koldunovn/climsight/blob/e4bf6d07f9f7b0cfbb60d9c123bc33b6665e22de/climsight.py#L498 We load population data once again. Instead I would load it once and pass to function as argument, it will save us one load.

Here https://github.com/koldunovn/climsight/blob/e4bf6d07f9f7b0cfbb60d9c123bc33b6665e22de/climsight.py#L791 we might check if data are not None and save a function call.

AntoniaJost commented 7 months ago

Totally agree that loading the data twice is not the efficiency we are looking for. I will fix this soon.

AntoniaJost commented 7 months ago

With respect to the warning: I am aware of it and have tried many ways to remove it but didn't manage to do so. Would appreciate some help from you @koldunovn .

AntoniaJost commented 6 months ago

Thoughts on new / different data sets

Gridded population of the world (GPW):

WorldPop:

Global Human Settlement Layer: