CSOIreland / PxStat

Data Dissemination Management System for publishing Statistics in Open Data formats.
MIT License
34 stars 7 forks source link

[ENHANCEMENT] Add GeoJSON entity to PxStat #423

Closed damienderoistecso closed 3 years ago

damienderoistecso commented 3 years ago

Build GeoJOSN entity in PxStat with ability to add, update, edit and delete GeoJSON objects.

Provide RESTfull API to read GeoJSON data from repository for use in tables to display geospatial data on a map

Update build create/update to select the appropriate maps and insert the RESTfull API endpoint into PX files

neilfromcork commented 3 years ago

APIs created

neilfromcork commented 3 years ago

API amendments completed:

neilfromcork commented 3 years ago

ReadDataByMap - null output amended - rolled back

neilfromcork commented 3 years ago

Server changes complete

neilfromcork commented 3 years ago

Validation amendment complete for Build Update

damienderoistecso commented 3 years ago

Client side changes complete

liamie commented 3 years ago

@neilfromcork If a px file has two or more map entities for the same classification it passes validation for create, update and import As discussed there should only be one map allowed per classification in a valid file and so should be picked up in validation.

Note: Only the first map entry seems to be used/retained in the create/update tools but for clarity I think it should still be part of the validation.

liamie commented 3 years ago

@damienderoistecso text on import error message goes out of bounds when map file does not exist. image

may just be unbroken text line too long rather than map specific

neilfromcork commented 3 years ago

Validation of PX file extended to allow maximum only one map per classification

damienderoistecso commented 3 years ago

@damienderoistecso text on import error message goes out of bounds when map file does not exist. image

may just be unbroken text line too long rather than map specific

@lorenzobruni I think we can solve this issue by adding the bootstrap class "overflow-auto" to the <li> when we list errors from the api.ajax.jsonrpc function. When I tested in using the inspector it outputs like this which is much better and allows the user to properly read the error with the help of a scroll bar when needed

image image

liamie commented 3 years ago

@damienderoistecso @neilfromcork Currently the add map validation in System > Maps passes if the features contain a properties tag it doesn't confirm that there are actually named properties in that tag so if any/all features have empty properties the dropdowns for assigning properties are empty. As we have no means of manually assigning a property validation should probably ensure we have at least one property for the dropdowns. However if properties are listed in the first feature but not in some of the subsequent features the drop down is populated but the map wont load just hangs on the loading bars

"type": "Feature","properties": { },

image

liamie commented 3 years ago

Validation of PX file extended to allow maximum only one map per classification

This seems resolved, now correctly reporting error with inconsistent map data

damienderoistecso commented 3 years ago

@damienderoistecso @neilfromcork Currently the add map validation in System > Maps passes if the features contain a properties tag it doesn't confirm that there are actually named properties in that tag so if any/all features have empty properties the dropdowns for assigning properties are empty. As we have no means of manually assigning a property validation should probably ensure we have at least one property for the dropdowns. However if properties are listed in the first feature but not in some of the subsequent features the drop down is populated but the map wont load just hangs on the loading bars

"type": "Feature","properties": { },

image

Client updated to validate properties during "Set Properties" step

@liamie ready for testing

liamie commented 3 years ago

@damienderoistecso "Set Properties" button not disabled after first press so everytime click "Set Properties" an additional english field is added. image

Similar occurs for multi language image

neilfromcork commented 3 years ago

"Currently the add map validation in System > Maps passes if the features contain a properties tag it doesn't confirm that there are actually named properties in that tag so if any/all features have empty properties the dropdowns for assigning properties are empty. As we have no means of manually assigning a property validation should probably ensure we have at least one property for the dropdowns. However if properties are listed in the first feature but not in some of the subsequent features the drop down is populated but the map wont load just hangs on the loading bars"

From a map validation point of view, this is correct. GeoMap allows anything in the properties object, hence we to validate this as a GeoJson file, we don't concern ourselves with the contents of the properties object.

damienderoistecso commented 3 years ago

@damienderoistecso "Set Properties" button not disabled after first press so everytime click "Set Properties" an additional english field is added. image

Similar occurs for multi language image

Fixed in dev @liamie ready for testing

liamie commented 3 years ago

@damienderoistecso "Set Properties" button not disabled after first press so everytime click "Set Properties" an additional english field is added. image Similar occurs for multi language image

Fixed in dev @liamie ready for testing

This aspect is now corrected, the Set Properties button cannot be activated multiple times now.

liamie commented 3 years ago

Possible conflict between validation rules image this error is flagged in create but if the same map link as used in the create/import file is readded then it will pass through all other validation for import and publishing stages.

damienderoistecso commented 3 years ago

Fixed in dev @liamie ready for test

liamie commented 3 years ago

Possible conflict between validation rules image this error is flagged in create but if the same map link as used in the create/import file is readded then it will pass through all other validation for import and publishing stages.

This issue appears to be resolved now, no longer producing error so no need to keep readding the map again

JohnPCarroll commented 3 years ago

The Build update fails when map included. Table used E2003. This is a single language table. When you add a map with duel languages the following error message appears image

The build create works correctly

In both cases the Elimination value was applied

neilfromcork commented 3 years ago

Elimination issue fixed in dev - please retest @JohnPCarroll

liamie commented 3 years ago

@damienderoistecso in system> Maps the table count and attached tables for geomaps seems to have been reset to 0 for all and no longer incrementing or listing for new table attachments. image some of these have tables attached should not be all 0 also modal page when link attached no longer has any tables for any entry image

JohnPCarroll commented 3 years ago

Map Widgets not working. Table E2003. This table has a status of "Work in Progress". Nothing is shown when the downloaded widget is used Table E2003JC. This table has a status of "Live". The title is shown but no map when the downloaded widget is used.

damienderoistecso commented 3 years ago

@damienderoistecso in system> Maps the table count and attached tables for geomaps seems to have been reset to 0 for all and no longer incrementing or listing for new table attachments. image some of these have tables attached should not be all 0 also modal page when link attached no longer has any tables for any entry image

@neilfromcork Can you look at this. "TableCount" always returns zero from "PxStat.Data.GeoMap_API.ReadCollection"

neilfromcork commented 3 years ago

@damienderoistecso - That should be ok now

JohnPCarroll commented 3 years ago

Map not displayed. Table E2003JCNULL. This table contains no values for counties Kilkenny and Leitrim.

damienderoistecso commented 3 years ago

Map Widgets not working. Table E2003. This table has a status of "Work in Progress". Nothing is shown when the downloaded widget is used Table E2003JC. This table has a status of "Live". The title is shown but no map when the downloaded widget is used.

@JohnPCarroll fixed in dev, ready for test, update both widget code and client code to test

JohnPCarroll commented 3 years ago

PNG issue. When you download a map the png output is not very clear as the colors in the map match the background color ( white). Example taken from E2003JCZERO ( work in progress )

Selection

image

PNG Output

image

Note: Can the legend be also be added so that there is some frame of reference as to what the colours represent figure wise. (LM)

JohnPCarroll commented 3 years ago

Map Widget not working for Table E2003. This table has a status of "Work in Progress".

Map Widgets working for Table E2003JC. This table has a status of "Live". The only issue I have is that you do not know what the map details refer to eg. Time period, Statistical indicator.

damienderoistecso commented 3 years ago

Map not displayed. Table E2003JCNULL. This table contains no values for counties Kilkenny and Leitrim.

@JohnPCarroll This issue should be fixed. Update widget code

damienderoistecso commented 3 years ago

@damienderoistecso - That should be ok now

@neilfromcork Might still be issue with "PxStat.Data.Matrix_API.ReadByGeoMap", always returns null

JohnPCarroll commented 3 years ago

Map displayed for live Table E2003JCNULL. This table contains no values for counties Kilkenny and Leitrim.

liamie commented 3 years ago

@damienderoistecso - That should be ok now

@neilfromcork Might still be issue with "PxStat.Data.Matrix_API.ReadByGeoMap", always returns null @damienderoistecso @neilfromcork Maybe related to this but currently maps do not seem to have deletion protection if attached to an existing file so if the front end delete button is purposefully/accidently made activate the map entry can be deleted as if it had 0 tables attached. This results in an oops! message on the tables pages that had these now deleted maps attached. image

neilfromcork commented 3 years ago

@liamie - bug found and fixed related to this, can you check this issue again? Thanks

damienderoistecso commented 3 years ago

E2003

@JohnPCarroll Can you test this now please. Update both client and widget folder from SVN

liamie commented 3 years ago

@damienderoistecso - That should be ok now

@neilfromcork Might still be issue with "PxStat.Data.Matrix_API.ReadByGeoMap", always returns null @damienderoistecso @neilfromcork Maybe related to this but currently maps do not seem to have deletion protection if attached to an existing file so if the front end delete button is purposefully/accidently made activate the map entry can be deleted as if it had 0 tables attached. This results in an oops! message on the tables pages that had these now deleted maps attached. image

This appears to be resolved now. Table names are appearing and deletion with existing connection is protected now

JohnPCarroll commented 3 years ago

Map Widget working for tables with a status of "Work in Progress". For Example E2003JCMAPTEST