Determine options to switch search engine

bcgov / aries-vcr

Hyperledger Aries Verifiable Credential Registry (VCR) is a set of application level software components designed to accelerate the adoption of trustworthy entity to entity communications.

Apache License 2.0

78 stars 67 forks source link

Determine options to switch search engine #761

Open esune opened 9 months ago

esune commented 9 months ago

Our Solr implementation is performing poorly due to difficulty in optimizing the search indexes as well as not being well-suited to live in OpenShift. We should determine a viable option for a new search engine that we could switch the project to.

WadeBarnes commented 9 months ago

We should also mention Solr is not well supported by our current (Haystack based) toolset. Supported backends are listed here. The toolset "seems" to support Solr 7.x now. However we're using an older version of the tools with Solr 8.11.2. Solr version 9.x is the current major release with plans for 10.x. All versions of Solr <8.11 are EOL. The tools seem to have always been one version behind on their support.

WadeBarnes commented 8 months ago

Since we've gotten the unbound queries somewhat under control (by blocking them), performance has been fantastic. We still have to find a search engine with better support.

swcurran commented 7 months ago

From ZenHub issue:

See https://github.com/bcgov/orgbook-configurations/pull/137 for background information.

Possible options:

OpenSearch: https://opensearch.org
ElastcSearch: https://www.elastic.co/elasticsearch/

Acceptance Criteria:

Determine best replacement option for the search engine
Assess and plan required code changes inAries VCR to support the new service

WadeBarnes commented 7 months ago

Performance continues to be fantastic since we've blocked the unbound queries through expression filtering.

swcurran commented 7 months ago

So just close this issue and we can search for this if needed?

WadeBarnes commented 7 months ago

I think we still want to look at other options, since the support for Solr, at least with the tech stack we're using, is not great. The fact that things are working makes this far less urgent though.

swcurran commented 7 months ago

I know that BC Registries is using Solr elsewhere, and it is really their project, their call. Enterprise Apps is not going to be making such a change, or spending time on this.

WadeBarnes commented 7 months ago

So we leave it here and open for them.

WadeBarnes commented 7 months ago

I've taken it off our board.