search

urbanbigdatacentre / ideamaps-models

Models of deprivation sub-domains for the IDEAMAPS data ecosystem project. This repo contains the source code used to run models and the model outputs. It also contains logic to upload model outputs to the IDEAMAPS platform.
MIT License
3 stars 0 forks source link

Create New Reference Dataset with high/medium/low categories #103

Open qszhao opened 1 month ago

qszhao commented 1 month ago

We will finalise the validation datasets:

  1. Adenike/Alex’s human-generated training datasets - only for Lagos, very limited for Kano. We need to find out where are the source for Monika's Nairobi data.
  2. PAR data validation for SDS/ISL/overall deprivation. We will need to create this data based on the PAR results and combine it with the master datasets @manuellacomerio will create.
andymithamclarke commented 1 month ago

This is an ongoing process to determine how and whether to use the validation data created by @Adenikemie and @AlexandraMiddleton as part of the creation of new training data.

Progress so far is that a meeting was held with the above + @manuellacomerio - outcome: reporting to @qszhao all of the workflows that they have done.

Next steps - a subsequent meeting to determine information about the grid that was used and to see whether the grid matches.

qszhao commented 1 month ago

@manuellacomerio will convert the SDS/ISL output grid for Lagos and Kano from the output files and try to match it with Adenike/Alex's validation data. If the grid matches, the data will be joined. She will also try to match it with Nairobi data Monika supplied.

Then PAR data will be joined to the grid. @andymithamclarke if you can help @manuellacomerio to identify the PAR results that will be great. Now it is a bit confused which one we should use.

andymithamclarke commented 1 month ago

Closing this as complete - see update in #101

qszhao commented 1 month ago

Reopen the issue - we will need to generate the new high/medium/low categories for morphological informality based on the reference data in Nairobi and Lagos, plus the PAR results for SDS and ISL.

diegopajarito commented 4 weeks ago

Reference dataset with L/M/H is ready for Nairobi. This has been shared over CRIB and GDrive.

Reference dataset with L/M/H are planned for Friday 18. @SebastianHafner @manuellacomerio will update on the progress

gielinkg commented 3 weeks ago

@SebastianHafner to upload reference to GDrive and CRIB for Lagos. @SebastianHafner to close this issue once all the data is uploaded to the appropriate locations.