DCAT harvester: resource not added for a dataset

AntoineAugusti commented 2 years ago

Detailed Description

The Clermont Metropole portal is harvested on data.gouv.fr using DCAT. This remote dataset is missing a resource on data.gouv.fr.

The only resource is listed as "ArcGIS Hub Dataset" with a link to the remote dataset and not a direct link to the resource (a GTFS/zip file).

Context

When a portal is harvested, I want to have correct data about resources, so that my platform is trusted.

Possible Implementation

No idea sorry.

Your Environment

udata 3.2.1 for data.gouv.fr

abulte commented 2 years ago

For reference https://opendata.clermontmetropole.eu/data.json

abulte commented 2 years ago

Seems like it's due to the way the DCAT source is built:

     {
        "@type":"dcat:Dataset",
        "identifier":"https://opendata.clermontmetropole.eu/datasets/1e0aad2349db4571b3c91087e58367e8",
        "license":"http://opendatacommons.org/licenses/odbl/summary",
        "landingPage":"https://opendata.clermontmetropole.eu/datasets/1e0aad2349db4571b3c91087e58367e8",
        "title":"Horaires théoriques du réseau de transport public du Syndicat Mixte des Transports en commun de l'agglomération Clermontoise Automne 2021",
        "description":"<div>Données descriptives de l’offre de transport (lignes, arrêts, \nhoraires) du réseau de transport en commun mis en œuvre par T2C pour la \npériode du 30/08/2021 au 03/07/2022. <br /></div><div>Les données mises à disposition consistent en l’ensemble des éléments issus du guide horaire du réseau. <br /></div><div><br /></div><div>Description\n géographique : périmètre des transports urbains de l’agglomération \nclermontoise et des communes de Mur sur Allier et Pérignat ès Allier.</div><div><br /></div><div>Description technique : les données sont publiées au format GTFS. Pour plus d’informations sur ce format : <a href='https://developers.google.com/transit/gtfs/reference' target='_blank' rel='nofollow ugc noopener noreferrer'>https://developers.google.com/transit/gtfs/reference </a></div><div><br /></div><div>Exclusions\n : les données fournies portent sur les services réguliers hors lignes \nspécifiques (lignes scolaires) et hors lignes de TAD Zonaux.</div><div><br /></div><div style='text-align:center;'></div><div><br /></div>",
        "keyword":[
           "temps théoriques",
           "horaires aux arrêts",
           "lignes de transport urbain",
           "bus",
           "tramway",
           "Syndicat Mixte des Transports en Commun de l’Agglomération Clermontoise",
           "mobilité",
           "TCU",
           "GTFS",
           "T2C",
           "SMTC"
        ],
        "issued":"2021-09-14T12:43:12.000Z",
        "modified":"2021-10-19T07:19:05.000Z",
        "publisher":{
           "name":"SMTC-T2C"
        },
        "contactPoint":{
           "@type":"vcard:Contact",
           "fn":"cam_opendata"
        },
        "accessLevel":"public",
        "distribution":[
           {
              "@type":"dcat:Distribution",
              "title":"ArcGIS Hub Dataset",
              "format":"Web Page",
              "mediaType":"text/html",
              "accessURL":"https://opendata.clermontmetropole.eu/datasets/1e0aad2349db4571b3c91087e58367e8"
           },
           {
              "@type":"dcat:Distribution",
              "title":"ArcGIS GeoService",
              "format":"ArcGIS GeoServices REST API",
              "mediaType":"application/json"
           }
        ]
     },

Unless I'm mistaken, there's nothing in here that can point us to a direct download link.

abulte commented 2 years ago

I closed this fast to get it out of the way, but I'm still open for discussion if needed :-)

AntoineAugusti commented 2 years ago

Sounds good. I looked at the JSON catalog and arrived at the same conclusion.

I'll get in touch with this portal team to try to get this fixed.

Wondering how I can get access to the catalog URL next time if you can share.

On Tue, 26 Oct 2021, 17:15 Alexandre Bulté, @.***> wrote:

I closed this fast to get it out of the way, but I'm still open for discussion if needed :-)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/opendatateam/udata/issues/2670#issuecomment-952042916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACIGHOMEDQEYLNZJTBUDQ3UI3HZXANCNFSM5GXDRLVQ .

opendatateam / udata