search

outbreak-info / outbreak.info-resources

A curated repository of metadata of resources on COVID-19 and SARS-CoV-2
MIT License
0 stars 4 forks source link

Datasets added manually don't appear to be indexed by google #147

Open gtsueng opened 4 years ago

gtsueng commented 4 years ago

The sitemaps.xml file in google console for the Biothings DDE site does not appear to be updated so google does not seem to be crawling the datasets added manually via the guide.

Automating the update of the sitemaps xml file in the google console, may be a good first issue for a new research programmer. Will leave it up to Jerry to decide if who to assign it to.

namespacestd0 commented 4 years ago

The sitemap is dynamically generated here: https://discovery.biothings.io/sitemap/dataset.xml

namespacestd0 commented 3 years ago

This issue seems to be external to the sitemap handler: https://github.com/biothings/discovery-app/blob/671024a6d4cfeaaf8ecb079504ae4d50d089a331/discovery/sitemap.py

flaneuse commented 3 years ago

@gtsueng can you check if the crawling is working properly? It seems like there are a good number of indexed datasets from the DDE but I'm not sure if it's everything.