search

OpenDataNode / ckanext-odn-ic2pc-sync

CKAN Extenstion for synchronization of catalog records from internal catalog to public (external) catalog
GNU Affero General Public License v3.0
0 stars 0 forks source link

Synchronization from internal to public catalog not working #19

Closed jakubklimek closed 9 years ago

jakubklimek commented 9 years ago

Initially, there was an issue regarding diacritics according to @Jan-Marcek however, there are problems even with datasets that are in english - see the screenshots below. In internal catalog, everything is filled out, in public catalog there is only a name. Internal catalog: image Public catalog: image

Jan-Marcek commented 9 years ago

@jakubklimek please, describe how you create dataset and resource. It is done by some script? dpu?

Jan-Marcek commented 9 years ago

the situation when it happens 

#!/bin/bash

ret=`curl --insecure https://odn-ci-test1.odn.in.eea.sk/internalcatalog/api/action/package_create -H "Authorization:3a6066d4-2d73-4490-a99a-bdf099f3267c"  -d '{"owner_org":"comsode-cuni","schema":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories-vocabulary.trig","license_link":"https://creativecommons.org/licenses/by/4.0/","temporal_end":"2013-12-31+01:00","notes":"This dataset contains numbers of inhabitants in districts of Czech Republic in age categories splited in 5year intervals with a distinguished category 0 years. The last age category is 90-94 years. All older categories are aggregated to 95 years and more. The dataset distinguishes males and females.","metadata_modified":"2015-07-21+02:00","publisher_name":"Opendata.cz","ruian_code":1,"metadata_created":"2015-02-26+01:00","resources":[],"temporal_start":"2007-01-01+01:00","title":"Number of inhabitants in districts of Czech Republic in 5year age categories","frequency":"R/P1Y","tags":[{"name":"5year age groups"},{"name":"population"},{"name":"demography"}],"publisher_uri":"http://opendata.cz","license_id":"other-open","spatial_uri":"http://ruian.linked.opendata.cz/resource/staty/1","name":"czso-demography-in-regions-czech-republic-age-categories","maintainer_email":"datasets@opendata.cz","theme":"http://eurovoc.europa.eu/385 http://eurovoc.europa.eu/5860 ","ruian_type":"ST"}'`

id=`echo $ret | grep -oP ' \"id\": \"\K\w+-\w+-\w+-\w+-\w+' | head -1`

curl --insecure https://odn-ci-test1.odn.in.eea.sk/internalcatalog/api/action/package_update -H "Authorization:3a6066d4-2d73-4490-a99a-bdf099f3267c"  -d '{"owner_org":"comsode-cuni","schema":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories-vocabulary.trig","license_link":"https://creativecommons.org/licenses/by/4.0/","temporal_end":"2013-12-31+01:00","notes":"This dataset contains numbers of inhabitants in districts of Czech Republic in age categories splited in 5year intervals with a distinguished category 0 years. The last age category is 90-94 years. All older categories are aggregated to 95 years and more. The dataset distinguishes males and females.","metadata_modified":"2015-07-21+02:00","publisher_name":"Opendata.cz","ruian_code":1,"metadata_created":"2015-02-26+01:00","resources":[{"name":"SPARQL Endpoint","format":"api/sparql","resource_type":"api","id":"d1b6c5b3-ff2a-4591-bed6-595403fa3426","url":"http://linked.opendata.cz/sparql"},{"format":"example/turtle","resource_type":"file","name":"Example resource in Turtle","mimetype":"text/turtle","id":"c82f915b-910d-4b2d-9fb2-d0d0fa6cd76f","url":"http://linked.opendata.cz/sparql?query=DESCRIBE+%3Chttp://data.czso.cz/resource/observation/demography-in-regions-czech-republic-age-categories/CZ0100/2007-12-31/F/35%3E&default-graph-uri=http%3A%2F%2Fdata.czso.cz%2Fresource%2Fdataset%2Fdemography-in-regions-czech-republic-age-categories&output=text%2Fturtle"},{"format":"HTML","resource_type":"file","name":"Example resource","mimetype":"text/html","id":"f8d26c4a-8ea5-47e2-ad9f-3472f284e413","url":"http://data.czso.cz/resource/observation/demography-in-regions-czech-republic-age-categories/CZ0100/2007-12-31/F/35"},{"license_link":"https://creativecommons.org/licenses/by/4.0/","created":"2015-02-26+01:00","format":"application/trig","resource_type":"file","description":"This dataset contains numbers of inhabitants in districts of Czech Republic in age categories splited in 5year intervals with a distinguished category 0 years. The last age category is 90-94 years. All older categories are aggregated to 95 years and more. The dataset distinguishes males and females.","url":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories.trig","describedByType":"text/turtle","name":"Number of inhabitants in districts of Czech Republic in 5year age categories","describedBy":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories-vocabulary.trig","mimetype":"application/trig","id":"d38f3fe2-9ef2-4207-aa20-a36177a2898a","last_modified":"2015-07-21+02:00","distro_url":"http://data.czso.cz/resource/dataset/demography-in-regions-czech-republic-age-categories/distribution"}],"temporal_start":"2007-01-01+01:00","title":"Number of inhabitants in districts of Czech Republic in 5year age categories","frequency":"R/P1Y","tags":[{"name":"5year age groups"},{"name":"population"},{"name":"demography"}],"publisher_uri":"http://opendata.cz","license_id":"other-open","spatial_uri":"http://ruian.linked.opendata.cz/resource/staty/1","id": "'"$id"'","maintainer_email":"datasets@opendata.cz","theme":"http://eurovoc.europa.eu/385 http://eurovoc.europa.eu/5860 ","ruian_type":"ST"}'

I observed when command down is run instead of second one (package_update) then it works correctly. The difference is that JSON's attribute and value "id" is replaced by "name".

curl --insecure https://odn-ci-test1.odn.in.eea.sk/internalcatalog/api/action/package_update -H "Authorization:3a6066d4-2d73-4490-a99a-bdf099f3267c"  -d '{"owner_org":"comsode-cuni","schema":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories-vocabulary.trig","license_link":"https://creativecommons.org/licenses/by/4.0/","temporal_end":"2013-12-31+01:00","notes":"This dataset contains numbers of inhabitants in districts of Czech Republic in age categories splited in 5year intervals with a distinguished category 0 years. The last age category is 90-94 years. All older categories are aggregated to 95 years and more. The dataset distinguishes males and females.","metadata_modified":"2015-07-21+02:00","publisher_name":"Opendata.cz","ruian_code":1,"metadata_created":"2015-02-26+01:00","resources":[{"name":"SPARQL Endpoint","format":"api/sparql","resource_type":"api","id":"d1b6c5b3-ff2a-4591-bed6-595403fa3426","url":"http://linked.opendata.cz/sparql"},{"format":"example/turtle","resource_type":"file","name":"Example resource in Turtle","mimetype":"text/turtle","id":"c82f915b-910d-4b2d-9fb2-d0d0fa6cd76f","url":"http://linked.opendata.cz/sparql?query=DESCRIBE+%3Chttp://data.czso.cz/resource/observation/demography-in-regions-czech-republic-age-categories/CZ0100/2007-12-31/F/35%3E&default-graph-uri=http%3A%2F%2Fdata.czso.cz%2Fresource%2Fdataset%2Fdemography-in-regions-czech-republic-age-categories&output=text%2Fturtle"},{"format":"HTML","resource_type":"file","name":"Example resource","mimetype":"text/html","id":"f8d26c4a-8ea5-47e2-ad9f-3472f284e413","url":"http://data.czso.cz/resource/observation/demography-in-regions-czech-republic-age-categories/CZ0100/2007-12-31/F/35"},{"license_link":"https://creativecommons.org/licenses/by/4.0/","created":"2015-02-26+01:00","format":"application/trig","resource_type":"file","description":"This dataset contains numbers of inhabitants in districts of Czech Republic in age categories splited in 5year intervals with a distinguished category 0 years. The last age category is 90-94 years. All older categories are aggregated to 95 years and more. The dataset distinguishes males and females.","url":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories.trig","describedByType":"text/turtle","name":"Number of inhabitants in districts of Czech Republic in 5year age categories","describedBy":"http://linked.opendata.cz/dumps/czso-demography-in-regions-czech-republic-age-categories-vocabulary.trig","mimetype":"application/trig","id":"d38f3fe2-9ef2-4207-aa20-a36177a2898a","last_modified":"2015-07-21+02:00","distro_url":"http://data.czso.cz/resource/dataset/demography-in-regions-czech-republic-age-categories/distribution"}],"temporal_start":"2007-01-01+01:00","title":"Number of inhabitants in districts of Czech Republic in 5year age categories","frequency":"R/P1Y","tags":[{"name":"5year age groups"},{"name":"population"},{"name":"demography"}],"publisher_uri":"http://opendata.cz","license_id":"other-open","spatial_uri":"http://ruian.linked.opendata.cz/resource/staty/1","name":"czso-demography-in-regions-czech-republic-age-categories","maintainer_email":"datasets@opendata.cz","theme":"http://eurovoc.europa.eu/385 http://eurovoc.europa.eu/5860 ","ruian_type":"ST"}'