DigitalCommons / mykomap-monolith

A web application for mapping initiatives in the Solidarity Economy
0 stars 0 forks source link

[CWM] Add India Data to CWM #54

Closed lin-d-hop closed 23 hours ago

lin-d-hop commented 1 week ago

Description

Add the India data to the CWM. This spreadsheet lists all the datasets and their source files

This will involve:

Acceptance Criteria

ColmDC commented 4 days ago

See https://github.com/DigitalCommons/cwm-test-data/issues/1

wu-lee commented 2 days ago

I think all the datasets are now normalised via the datafactory (open-data repo) and deployed on the relevant URL https://dev.data.solidarityeconomy.coop/<dataset tag>/. In general I'll be using the development data versions.

Next step is to perform quick and dirty concatenation without any deduplication of organisations, as discussed. Only three of the new datasets have domains and so can be deduplicated. This data can then be deployed and checked on the CWM map.

After that I can look at a more scalable "merging" of the datasets using deduplication by .coop and potentially other attributes. If I can see a quick way to do that I will.

wu-lee commented 2 days ago

Ok, I've done a slap-dash "merge" of the new datasets CUK, USFWC, CMC and IFFCO, without any de-duplication (although IFFCO, the massive one, has no domain info), converted the data to myko-data and deployed it on dev-2. Et voila: https://dev.maps.coop/cwm/?datasetId=delhi

The script and the intermediate data will get pushed to demo-merge-map-data although I'm likely to review it and possibly tinker with it tomorrow.

The myko-data version is on the delhi-demo branch of the cwm-test-data repository.

ColmDC commented 23 hours ago

All the required datasets are in. Any deficiencies and co-op-co-op deduplication process can have a seperate issue.