GeoNetwork Harvesting backend: CSV download failed links

[manilly]

Hi, when will the GeoNetwork-based backend will be used for official Harvestings? Is there a estimated start?

I'm still missing the feature where I can download the list of data-metadata records, which failed to be properly connected to a download and/or to a view service (like in old/current harvest console). Will this be available?

[MatAlps]

Hi Manilly,

Mathieu here for LU (not part of helpdesk), concerning

I'm still missing the feature where I can download the list of data-metadata records, which failed to be properly connected to a download and/or to a view service (like in old/current harvest console). Will this be available?

In case that can help you, as workaround we intercept the back-end request to get the table of results as geojson, then use an online tool to convert geojson to csv. That way in a few minutes you can compute the table.

[jescriu]

Dear @manilly,

when will the GeoNetwork-based backend will be used for official Harvestings? Is there a estimated start?

As we reported in the 72nd MIG-T Meeting last week, the plan is to use the revamped INSPIRE Geoportal backend based on GeoNetwork (GN) for the next M&R round. The rollout of the revamped INSPIRE Geoportal (not only including the new GN backend but also the new geoportal user interface) is on-going. Howerver, when undergoing the necessary security assessment, several issues were identified that needs to be fixed before going online with the new system. GeoCat is working on it.

I'm still missing the feature where I can download the list of data-metadata records, which failed to be properly connected to a download and/or to a view service (like in old/current harvest console). Will this be available?

Again, as reported in the past MIG-T meeting, the revamped INSPIRE Geoportal will miss some of the functionalities that were present in the classical INSPIRE Geoportal. Since the new code of the new Geoportal components is open source, the community should be able to test, analyse and potentially propose corrections, changes and enhancements, as part of the joint effort to improve the system step-by-step. That is the reason why some of the functionalities were not considered as a priority in the first delivery of the system.

During the last 'GeoNetwork and INSPIRE' event we had in Brussels last week (on 24th Nov 2022), back-to-back with the MIG and MIG-T, we asked GeoCat to provide access to a simple tool they developed (and were using themselves for testing purposes) which analyse the accessibility of services for a single data set metadata record. They answer that this tool would be also available for the community in their code repository.

Once available, we will consider the feasibility to improve the user experience by iterating its usage for a collection of data set metadata records.

[jescriu]

Dear @MatAlps,

In case that can help you, as workaround we intercept the back-end request to get the table of results as geojson, then use an online tool to convert geojson to csv. That way in a few minutes you can compute the table

Interested in knowing more about your trick, to check if this could be useful for the rest of the community.

[MatAlps]

Dear @jescriu ,

In chrome I looked at developper tools -> network & got the back-end request loading the table of results. From there I could change some query parameters and get the whole list in geojson.

I have not written down the url, so I can't share it now. I will do it once it's back online

[manilly]

Thanks, that sounds good. Looking forward to get more details. @jescriu is the GeoNetwork backend down at the moment? (https://jrc-geoportal-internal.geocat.live/geonetwork)

[jescriu]

Dear @manilly,

Yes, the server is temporarily inaccessible. We are working to solve it with our contractor, with which we had a technical meeting this afternoon.

This includes an upgrade of the server to increase its capacity and performance (as we reported in our last MIG-T Meeting).

We hope the server will be accessible again by tomorrow. We will keep you informed.

Jordi

[MatAlps]

Dear @MatAlps,

In case that can help you, as workaround we intercept the back-end request to get the table of results as geojson, then use an online tool to convert geojson to csv. That way in a few minutes you can compute the table

Interested in knowing more about your trick, to check if this could be useful for the rest of the community.

Hi @jescriu , @manilly ,

I mixed up the operation I use here https://inspire-geoportal.ec.europa.eu/results.html?country=be&view=details&theme=none to get the results in an excel format.

Today I have tried the same on the new harvester. It's a bit more complex, as there is a payload that your need to send along (I use postman). Then there is an issue for authorization (expected) that is blocking.

So sorry, can't help on that :/

So that feature to be able to download the results in CSV format, with columns about availability in view & download, becomes a MUST for me IMHO (already shared in the survey).

[manilly]

@jescriu any updates on these improvements. For us it is essential to have such a functionality (see mockup below)

[jrc-inspire]

Dear @manilly,

As reported in the last 75 MIG-T meeting, a new procurement has been launched to provide specific improvements for the revamped INSPIRE Geoportal based on GeoNetwork, including this feature.

Please, see this presentation for more details.

We will keep you posted.

[jescriu]

Dear @manilly, As it was reported in previous INSPIRE MIG-T meetings the mentioned procurement (aimed to tackle this need, between others) got deserted. However, in the context of the INSPIRE Monitoring 2023 we developed a set of INSPIRE Monitoring and Reporting tools for evaluating the accessibility and conformity of INSPIRE resources. In the case of accessibility, we provided for each endpoint a log with the dataset identifiers that failed the accessibility test, indicating also the reason: logs for Viewable datasets / Log for Downloadable datasets. Hope this is useful while the issues of GeoNetwork for evaluating the accessibility are mitigated as part of #184.

[jescriu]

Dear @manilly, @MatAlps,

Meanwhile it is clarified the link-cheking approach in GeoNetwork, you can use the accessibility tools that JRC provided and used for the calculation of the related indicators in the 2023 Monitoring process: https://github.com/INSPIRE-MIF/mr-tools/tree/main/dataset-accessibility/link-checker_script

The log files obtained as part of the outputs might serve to similarly satisfy your needs (related CSV).