Additional Data Findings

[adriana-madi]

• [ ] population density API
• [ ] altitude information (?)

Working together with @OlympiaG

[OlympiaG]

• Additional Dataset for Water Consumption per country : Water_Usage.xlsx

• Additional Dataset for Urbanization Rate per country : WUP2018-F09-Urbanization_Rate.xls

Several Datasets can be found here.

[VasLem]

Great findings, is it possible to harvest the several datasets from the webpage? I could not find a way to load them using csv

[OlympiaG]

Unfortunately, you have to pay for a professional account in order to have unlimited access to the datasets. I didn't realize that because I didn't try to download any dataset. I saw that it provides the opportunity of exportation, so I supposed that you can do it. However, each dataset is retrieved from another original website which is mentioned. So if we are interested in one specific topic we could search the original website.

[VasLem]

@bajo1207 given the input from Olympia, I believe that we have the world development indicators already loaded, right? Can you check? @OlympiaG is there another dataset that is being referenced as 'Original Data'?

[VasLem]

@adriana-madi and @OlympiaG let me know if you need help with the API

[VasLem]

• Additional Dataset for Water Consumption per country : Water_Usage.xlsx
• Additional Dataset for Urbanization Rate per country : WUP2018-F09-Urbanization_Rate.xls

Several Datasets can be found here.

Also @OlympiaG what is the source of the first excel? Looks nice

[bajo1207]

@bajo1207 given the input from Olympia, I believe that we have the world development indicators already loaded, right? Can you check? @OlympiaG is there another dataset that is being referenced as 'Original Data'?

The hdro2019.json dataset contains a lot of different development indicators.

[OlympiaG]

• This website has retrieved the datasets from other original databases (other websites) and therefore each dataset has its original data source. Because it contains a large number of datasets, it would be more efficient to find the kind of data that we would be interested in and make a search in the original source.

• The source of the first excel from here. There is no way to export the data from this website so I made a copy-paste in an excel file.

• Regarding the API, we were going to use an open database that includes all the cities around the world but there is a problem with the API and it only retrieves 10.000 while the number of records is around 3.000.000. So probably we will need to use another dataset. I had some ideas about this, but I just realized that the website is not working

• I also created 2 jupyter notebooks for the geospatial clustering of the cities based on the coordinates but I am not sure how useful this is.

[VasLem]

Regarding the first point, can you check if there is something other than worldbank related to water security? Because we have already gotten data from that source, so any other sources could become useful.

Regarding your third point, probably the results are paged, that's why you see them limited.

[OlympiaG]

So:

• I found this page which is a database on Water supply. By searching specifically here we can get some information regarding the safety, service, and facility of the water for each country. You can also download the .csv files (I downloaded them but I cannot upload them here in the comment). However, it might be a problem that the data is from 2017. In my opinion, it wouldn't because probably the situation for the majority of the countries has not changed until today.

• Another source is the AQUASTAT database which provides global information on Water and Agriculture for a broad period (1960 - 2020) but you can select a specific time period or the latest one. It contains datasets and values regarding several water issues and provides .csv files (again I cannot upload them here). However, some datasets include missing values but not all of them.

• And something not so relevant with the databases but I also found this website which has several projects on water issues. For example this. It may give us some ideas for the data or the project. Although I am not sure about this, it is just a suggestion.

[VasLem]

Great findings! We can use them if the current datasets used prove to be insufficient.

[bajo1207]

I've opened a issue for the aquastat dataset #34. Can this issue be closed?

[OlympiaG]

Yes, I don't think that we need it anymore.