search

NCEAS / metacatui

MetacatUI: A client-side web interface for DataONE data repositories
https://nceas.github.io/metacatui
Apache License 2.0
42 stars 26 forks source link

Update solr query to filter out portal objects from /summary view #1282

Closed rushirajnenuji closed 4 years ago

rushirajnenuji commented 4 years ago

If we visit https://arcticdata.io/catalog - we get the DataCatalog page which displays the total count of publicly available datasets. Right now the view shows 5787 total datasets. If we visit the summary view https://arcticdata.io/catalog/profile we have 5790 total datasets. After inspecting the solr queries, the difference in the totals seems to be from the three additional portal objects that we have on urn:node:arctic. The same thing applies to urn:node:KNB.

We can solve this easily by updating the queries for /sumary view to eliminate portal objects from the total counts.

laurenwalker commented 4 years ago

Ah, good catch

rushirajnenuji commented 4 years ago

Hi @laurenwalker - the way we’ve done this for a DC view is to set a filter object to exclude these formatIds on the searchModel. But for the /profile view, we do not have a searchModel, we directly generate queries in StatsModel. So, I’ve appended these filters to those queries. I’ve verified the numbers and they look okay, but I was wondering if this approach sounds correct.

It would be great if you could review the following commit. Thanks! https://github.com/NCEAS/metacatui/commit/fb2e3809acace551dfdd4474844219f00f4af266

amoeba commented 4 years ago

Nice catch and debugging, @rushirajnenuji. I'm not overly-familiar with how we're exposing portals to users but I'm wondering if we may also want to set up our Solr instances to hide Portal objects by default from all searches so explicit filtering isn't necessary?

rushirajnenuji commented 4 years ago

Hey @amoeba - thank you for taking a look at this. Interesting to know that we can hide a specific type of object from searches. We might have more than one use case, but one I can think of is the collectionQuery attribute that @gothub got indexed within solr for Portal objects. Both our DataONE services depend on this query attribute to generate aggregated metrics.

Example: query

laurenwalker commented 4 years ago

@rushirajnenuji - That commit looks good to me. Thanks!

rushirajnenuji commented 4 years ago

Thank you, Lauren. I'm closing this issue.