Closed valeriodeluca closed 7 months ago
I have migrated an instance and now face this issue, too. Creating widgets for the same datasets in a map is working fine. Dashboards fail with aboves error message though.
@dsuren1 can you please take a look?
I was looking at the responses of the csw call from the dashboard. On stable.demo it shows lots of <dct:references scheme="WWW:DOWNLOAD-1.0-http--download">
entries for a record.
On our migrated instance it does not show those. Do we need to update our metadata?
@gannebamm There is a management command: python manage.py addmissinglinks
. The metadataxsl says this:
In order to make it possibile for a resource to have its metadata exported as ISO+XSL, it needs to have a new entry in the
base_link
table. The implementation will create the needed links every time a new resource is added, but we need to post-process the existing resources and add the missing links.In order to add the missing links, you need to enter the geonode directory and issue the command:
python manage.py addmissinglinks
For each resource having a link for the ISO metadata, a new link for the ISO+XSL will be created.
@gannebamm The warning popup is shown when there is an exception, could you please check the network tab for /ows
calls (on selecting a layer) for response with exception information and report back, as I'm unable to reproduce this error on our stable.demo env for above mentioned layer
I do not see any error messages in the network tab.
We do use a new pycsw version. Does the dashboard determine resources using the CSW endpoint and Dublin Core output format? Here is a (GET) example output from our instance: https://geonode-demo.thuenen.de/catalogue/csw?REQUEST=GetRecords&SERVICE=CSW&version=2.0.2&resultType=results&ElementSetName=full&typenames=csw:Record
and from stable.demo: https://stable.demo.geonode.org/catalogue/csw?REQUEST=GetRecords&SERVICE=CSW&version=2.0.2&resultType=results&ElementSetName=full&typenames=csw:Record
The resultset is different, and the links are not listed in our instance.
Further testing revealed that I can add a WFS as a resource provider for the dashboard application. If I use our demo instances' GeoServer WFS endpoint, the widgets can be loaded!
For me, the issue seems to be due to our pycsw version and output and not bound to GeoNode´s code.
@ridoo I tried the specified management command but the output still does not contain dct:references scheme="WWW:DOWNLOAD-1.0-http--download">
entries.
@ridoo I tried the specified management command but the output still does not contain dct:references scheme="WWW:DOWNLOAD-1.0-http--download"> entries.
Ah .. now I see, what you mean. Our instance requests http://172.18.0.1:8001/catalogue/csw?service=CSW&version=2.0.2&request=
and responds with:
<!-- pycsw 3.0.dev0 -->
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:gml32="http://www.opengis.net/gml/3.2" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
<csw:SearchStatus timestamp="2024-03-19T10:34:54Z"/>
<csw:SearchResults numberOfRecordsMatched="1" numberOfRecordsReturned="0" nextRecord="1" recordSchema="http://www.opengis.net/cat/csw/2.0.2" elementSet="full"/>
</csw:GetRecordsResponse>
While the demo instance requests https://stable.demo.geonode.org/catalogue/csw?service=CSW&version=2.0.2&request=
and responds with:
<!-- pycsw 2.6.1 -->
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
<csw:SearchStatus timestamp="2024-03-19T10:34:21Z"/>
<csw:SearchResults numberOfRecordsMatched="3942" numberOfRecordsReturned="10" nextRecord="11" recordSchema="http://www.opengis.net/cat/csw/2.0.2" elementSet="full">
<csw:Record>
<dc:identifier>0a39ea0a-54db-11eb-9de0-0242ac150009</dc:identifier>
<dc:title>single_point</dc:title>
<dc:type>dataset</dc:type>
<dc:subject>features</dc:subject>
<dc:subject>single_point</dc:subject>
<dc:format>vector</dc:format>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Asingle_point&format=image%2Fjpeg&height=550&width=550&srs=EPSG%3A4326&bbox=9.9999%2C9.9999%2C10.0001%2C10.0001</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Asingle_point&format=application%2Fpdf&height=550&width=550&srs=EPSG%3A4326&bbox=9.9999%2C9.9999%2C10.0001%2C10.0001</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Asingle_point&format=image%2Fpng&height=550&width=550&srs=EPSG%3A4326&bbox=9.9999%2C9.9999%2C10.0001%2C10.0001</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Asingle_point&outputFormat=SHAPE-ZIP&srs=EPSG%3A4326&format_options=charset%3AUTF-8</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Asingle_point&outputFormat=gml2&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Asingle_point&outputFormat=text%2Fxml%3B+subtype%3Dgml%2F3.1.1&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Asingle_point&outputFormat=csv&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Asingle_point&outputFormat=excel&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Asingle_point&outputFormat=json&srs=EPSG%3A4326&srsName=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:LINK-1.0-http--link">https://stable.demo.geonode.org/catalogue/#/dataset/8</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://stable.demo.geonode.org/geoserver/ows?service=WMS&request=GetLegendGraphic&format=image/png&WIDTH=20&HEIGHT=20&LAYER=geonode:single_point&STYLE=single_point&version=1.3.0&sld_version=1.1.0&legend_options=fontAntiAliasing:true;fontSize:12;forceLabels:on</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">https://master.demo.geonode.org/uploaded/thumbs/layer-0a39ea0a-54db-11eb-9de0-0242ac150009-thumb.png?v=856f53ff</dct:references>
<dct:references scheme="OGC:WMS">https://stable.demo.geonode.org/geoserver/ows</dct:references>
<dct:references scheme="OGC:WFS">https://stable.demo.geonode.org/geoserver/ows</dct:references>
<dct:modified>2021-01-12T13:35:31Z</dct:modified>
<dct:abstract>No abstract provided</dct:abstract>
<dc:date>2021-01-12T13:35:31Z</dc:date>
<dc:creator>admin</dc:creator>
<dc:language>eng</dc:language>
<dct:alternative>geonode:single_point</dct:alternative>
<dct:spatial scheme="http://www.opengis.net/def/crs">EPSG:4326</dct:spatial>
<ows:BoundingBox crs="urn:x-ogc:def:crs:EPSG:6.11:4326" dimensions="2">
<ows:LowerCorner>9.9999 9.9999</ows:LowerCorner>
<ows:UpperCorner>10.0001 10.0001</ows:UpperCorner>
</ows:BoundingBox>
</csw:Record>
<csw:Record>
<dc:identifier>686164e0-54f0-11eb-a517-0242ac150009</dc:identifier>
<dc:title>Metop1_AVHRR_OrbitFootprint_PlateCarree_20201</dc:title>
<dc:type>dataset</dc:type>
<dc:subject>Metop1_AVHRR_OrbitFootprint_PlateCarree_20201102005203_20201102023103</dc:subject>
<dc:subject>features</dc:subject>
<dc:format>vector</dc:format>
<!-- cut for brevity -->
</csw:GetRecordsResponse>
Both send requests without a request
value.
It is actually a POST request @ridoo with this payload:
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmi="http://www.isotc211.org/2005/gmi" xmlns:ows="http://www.opengis.net/ows" service="CSW" version="2.0.2" resultType="results" startPosition="11" maxRecords="10"> <csw:Query typeNames="csw:Record"> <csw:ElementSetName>full</csw:ElementSetName> <csw:Constraint version="1.1.0"> <ogc:Filter> <ogc:Or>
<ogc:PropertyIsEqualTo>
<ogc:PropertyName>dc:type</ogc:PropertyName>
<ogc:Literal>dataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
<ogc:PropertyIsEqualTo>
<ogc:PropertyName>dc:type</ogc:PropertyName>
<ogc:Literal>http://purl.org/dc/dcmitype/Dataset</ogc:Literal>
</ogc:PropertyIsEqualTo>
</ogc:Or> </ogc:Filter> </csw:Constraint> </csw:Query> </csw:GetRecords>
:see_no_evil: thanks, now we are on the same page. I also did not copy the very first response, which looks on our instance as such:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 3.0.dev0 -->
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:gml32="http://www.opengis.net/gml/3.2" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
<csw:SearchStatus timestamp="2024-03-19T11:10:04Z"/>
<csw:SearchResults numberOfRecordsMatched="1" numberOfRecordsReturned="1" nextRecord="0" recordSchema="http://www.opengis.net/cat/csw/2.0.2" elementSet="full">
<csw:Record>
<dc:identifier>e242083c-a840-4a84-a55a-9b04af6798a3</dc:identifier>
<dc:title>admin_boundary_level1_divisions</dc:title>
<dc:type>dataset</dc:type>
<dc:subject></dc:subject>
<dc:format>vector</dc:format>
<dct:modified>2024-03-19T09:41:27Z</dct:modified>
<dct:abstract></dct:abstract>
<dc:date>2024-03-19T09:41:27Z</dc:date>
<dc:creator>admin</dc:creator>
<dc:language>eng</dc:language>
<dct:alternative>geonode:admin_boundary_level1_divisions</dct:alternative>
<dct:spatial scheme="http://www.opengis.net/def/crs">EPSG:4326</dct:spatial>
<ows:BoundingBox crs="urn:x-ogc:def:crs:EPSG:6.11:4326" dimensions="2">
<ows:LowerCorner>20.746230800000006 88.00862798610592</ows:LowerCorner>
<ows:UpperCorner>26.634513009669096 92.6802573821795</ows:UpperCorner>
</ows:BoundingBox>
</csw:Record>
</csw:SearchResults>
</csw:GetRecordsResponse>
Freshly imported dataset which also misses the <dct:references scheme="WWW:DOWNLOAD-1.0-http--download">
entries. Switching back to pycsw 2.6
does not seem to have an effect, the entries are still missing:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.6.0 -->
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
<csw:SearchStatus timestamp="2024-03-19T14:32:19Z"/>
<csw:SearchResults numberOfRecordsMatched="1" numberOfRecordsReturned="1" nextRecord="0" recordSchema="http://www.opengis.net/cat/csw/2.0.2" elementSet="full">
<csw:Record>
<dc:identifier>e242083c-a840-4a84-a55a-9b04af6798a3</dc:identifier>
<dc:title>admin_boundary_level1_divisions</dc:title>
<dc:type>dataset</dc:type>
<dc:subject></dc:subject>
<dc:format>vector</dc:format>
<dct:modified>2024-03-19T09:41:27Z</dct:modified>
<dct:abstract></dct:abstract>
<dc:date>2024-03-19T09:41:27Z</dc:date>
<dc:creator>admin</dc:creator>
<dc:language>eng</dc:language>
<dct:alternative>geonode:admin_boundary_level1_divisions</dct:alternative>
<dct:spatial scheme="http://www.opengis.net/def/crs">EPSG:4326</dct:spatial>
<ows:BoundingBox crs="urn:x-ogc:def:crs:EPSG:6.11:4326" dimensions="2">
<ows:LowerCorner>20.746230800000006 88.00862798610592</ows:LowerCorner>
<ows:UpperCorner>26.634513009669096 92.6802573821795</ows:UpperCorner>
</ows:BoundingBox>
</csw:Record>
</csw:SearchResults>
</csw:GetRecordsResponse>
@gannebamm not sure when this came in, but the pycsw-mapping was wrong. The pycsw_local_mappings.py
contained this entry:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 3.0.dev0 -->
<csw:GetRecordsResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:gml32="http://www.opengis.net/gml/3.2" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
<csw:SearchStatus timestamp="2024-03-19T15:07:06Z"/>
<csw:SearchResults numberOfRecordsMatched="1" numberOfRecordsReturned="1" nextRecord="0" recordSchema="http://www.opengis.net/cat/csw/2.0.2" elementSet="full">
<csw:Record>
<dc:identifier>e242083c-a840-4a84-a55a-9b04af6798a3</dc:identifier>
<dc:title>admin_boundary_level1_divisions</dc:title>
<dc:type>dataset</dc:type>
<dc:subject></dc:subject>
<dc:format>vector</dc:format>
<dct:references scheme="WWW:LINK-1.0-http--link">http://172.18.0.1:8001/catalogue/#/None/1</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Aadmin_boundary_level1_divisions&format=application%2Fpdf&height=550&width=550&srs=EPSG%3A4326&bbox=88.0086280%2C20.7462308%2C92.6802574%2C26.6345130</dct:references>
<dct:references scheme="OGC:WMS">http://172.18.0.1/geoserver/ows</dct:references>
<dct:references scheme="OGC:WFS">http://172.18.0.1/geoserver/ows</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Aadmin_boundary_level1_divisions&format=image%2Fpng&height=550&width=550&srs=EPSG%3A4326&bbox=88.0086280%2C20.7462308%2C92.6802574%2C26.6345130</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Aadmin_boundary_level1_divisions&outputFormat=SHAPE-ZIP&srs=EPSG%3A4326&format_options=charset%3AUTF-8</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Aadmin_boundary_level1_divisions&outputFormat=gml2&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Aadmin_boundary_level1_divisions&outputFormat=text%2Fxml%3B+subtype%3Dgml%2F3.1.1&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Aadmin_boundary_level1_divisions&outputFormat=csv&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Aadmin_boundary_level1_divisions&outputFormat=excel&srs=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=geonode%3Aadmin_boundary_level1_divisions&outputFormat=json&srs=EPSG%3A4326&srsName=EPSG%3A4326</dct:references>
<dct:references scheme="WWW:LINK-1.0-http--link">http://172.18.0.1:8001/catalogue/#/dataset/1</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WMS&request=GetMap&layers=geonode%3Aadmin_boundary_level1_divisions&format=image%2Fjpeg&height=550&width=550&srs=EPSG%3A4326&bbox=88.0086280%2C20.7462308%2C92.6802574%2C26.6345130</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1/geoserver/ows?service=WMS&request=GetLegendGraphic&format=image/png&WIDTH=20&HEIGHT=20&LAYER=geonode:admin_boundary_level1_divisions&STYLE=admin_boundary_level1_divisions&version=1.3.0&sld_version=1.1.0&legend_options=fontAntiAliasing:true;fontSize:12;forceLabels:on</dct:references>
<dct:references scheme="WWW:DOWNLOAD-1.0-http--download">http://172.18.0.1:8001/uploaded/thumbs/dataset-e242083c-a840-4a84-a55a-9b04af6798a3-thumb-0bb36af9-fd28-4fa7-bb82-82ccda7755a7.jpg</dct:references>
<dct:modified>2024-03-19T09:41:27Z</dct:modified>
<dct:abstract></dct:abstract>
<dc:date>2024-03-19T09:41:27Z</dc:date>
<dc:creator>admin</dc:creator>
<dc:language>eng</dc:language>
<dct:alternative>geonode:admin_boundary_level1_divisions</dct:alternative>
<dct:spatial scheme="http://www.opengis.net/def/crs">EPSG:4326</dct:spatial>
<ows:BoundingBox crs="urn:x-ogc:def:crs:EPSG:6.11:4326" dimensions="2">
<ows:LowerCorner>20.746230800000006 88.00862798610592</ows:LowerCorner>
<ows:UpperCorner>26.634513009669096 92.6802573821795</ows:UpperCorner>
</ows:BoundingBox>
</csw:Record>
</csw:SearchResults>
</csw:GetRecordsResponse>
I will fix it on our repo and close this issue then.
@giohappy sorry for the noise.
@valeriodeluca I closed this ticket because I found the issue we had on our setup. However, this does not mean your issue has the same root cause. Sorry, for that. Please feel free to re-open the ticket.
Hi eb and Happy New Year. When in a Dashboard I try to create a chart widget from a vector layer I get this error:
The vector layer is a very popular layer and can be downloaded from here: https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/50m/cultural/ne_50m_admin_0_countries.zip
When I try to create the chart widget in a Map instead of a Dashboard, using the same ne_50m_admin_0_countries layer, I have no problem and I can create the chart correctly. It seems that the chart widget creation problem is only when I'm in a Dashboard. Thanks, Happy 2023