linking Kartoza data schemas with CCCS database

[cccs-ip]

Kartoza wrote an script to load shapefile data into postgres. I would like to get these data loaded into the CCCS database in a meaningful way so that the qgis files they are creating can access these data (once qgis server is integrated with Django).

Please help us to run this script and import data. Please let us know if there are conflicting table names, database settings, etc. We want to ensure that all teams share a common database organization structure ASAP so that we don't risk continuing to develop using paths / references / etc. that would need to change when we bring everything together under production.

Please note that we also need to do this for 'Abadi'.

[cccs-ip]

Please update me on the status of this request. I remain confused by the need (and benefit) of renaming shapefiles as appears to be occurring with the sql importer. Specifically, I am worried that the MTB and MBD filenames will conflict. I would like to verify that independent schemas are being created appropriately, and that we have a systematic approach to creating schemas that will allow us to scale out easily as we work in other areas / countries / world regions.

[NyakudyaA]

Renaming helps in keeping with consistent naming convention. In the MTB and MBD folders they have shapefiles that have the same names and they are only being differentiated by the prefix mtb or mdb. So by renaming them we ensure that we have a single layer that is representative of both shapefiles. The records from either the shapefiles ie MTB are loaded first and then the MBD are then appended to the same layer ensuring that no data is lost. As they are loaded into the table the features get a unique id such that they will never be any conflict in filename. Viewing the shapefile admin_area-l4_idn_mbd and admin_area-l4_idn_mtb.shp and admin_area-l4 from the database will show you that they are identical.

[cccs-ip]

Thanks, Admire, for the clear explanation. It's good to know that these data are not lost.

With regard to our naming convention, CCCS decided on the current shapefile naming structure to allow us to keep all files in relatively 'flat' directories, and also to allow us to quickly identify our target files according to name. Thus, if I want to see admin polygons for the district level for MTB, I can simply find admin_area-L4_idn_mtb.shp. For villages, it would be admin_area-L5_idn_mtb.shp.

It's terrific that the database allows for hierarchical naming schemas that can help to maintain consistency and referential integrity. Paul is currently reviewing your work and will get back to you if there are any questions in terms of how the schemas are defined and organized.

[NyakudyaA]

We could change the loading script to add an extra column in the table that differentiates the data source ie for any layer that comes from mtb it will be distinct in the database. This will still allow us to have one layer for the same shapefile. This will allow you to do

select * from admin_area_l4 where source = mtb;

Source would be a column that describes whether the records comes from mtb or mdb

[cccs-ip]

Thanks for the suggestion. Such a column would definitely be helpful.

I am also thinking about the the difference in terms of what we use as a file naming system for shapefiles that we keep in a folder directory (such as in 'sync') versus the sorts of names that we need / want to present in a map. For example, the name 'admin-area-L5' is terrific in a file directory where I might have "villages" as the fifth administration level within a country, but something else at a corresponding admin level (possibly something like "hamlets").

Might it make sense to create something like a a 'global naming conversions' table that would allow us to store files with some 'technical' name, but would allow us to present them as a different 'common name' for our map keys (something that could also allow us to present translations, such as 'desa' and 'village') ?