search

ioos / catalog

IOOS Catalog general repo for documentation and issues
https://ioos.github.io/catalog/
MIT License
2 stars 6 forks source link

CKAN has 35 results, CSW has 26 for the same query #27

Closed rsignell-usgs closed 7 years ago

rsignell-usgs commented 7 years ago

I get 35 results with CKAN: https://data.ioos.us/dataset?q=ROMS&res_format=WMS

I get 26 results with CSW: http://nbviewer.jupyter.org/gist/rsignell-usgs/48014c3c0617c47e0457dbc6898856cb?flush_cache=True (see numberOfRecordsMatched in cell [3] output)

Why?

lukecampbell commented 7 years ago

This is related to https://github.com/ioos/catalog-ckan/issues/143 Over time the search database, solr, becomes out of sync with the actual relational database (PostGIS) and needs to be synchronized manually, which is on my to-do list in the near term. I hope to get to it today.

lukecampbell commented 7 years ago

When you search in CKAN it's querying solr. When you query in CSW, it looks at the database.

mwengren commented 7 years ago

@rsignell-usgs is this still an issue at this point? Do the counts still not match?

There will probably always be slight inconsistency between CKAN/Solr search results and pycsw because they are totally separate.

But we do need to better automate the update of Solr search index ioos/catalog-ckan#143.

rsignell-usgs commented 7 years ago

Today I get 36 results from CKAN, 28 results from CSW.

mwengren commented 7 years ago

@rsignell-usgs I think there may always be differences in the search results of the CKAN Solr index and the pycsw db, but are you in favor of doing further investigation of this? Or can we close the issue? Can you do a quick test of your notebook count again today?

Looks like we're up to 38 datasets in CKAN currently.

mwengren commented 7 years ago

I'm going to close this one as it's unlikely we'll make a change to resolve this.