search

alphagov / datagovuk_find

Beta version of Find Data
12 stars 9 forks source link

[EPIC] Use Solr for search in DGU Find #1293

Open kentsanggds opened 3 months ago

kentsanggds commented 3 months ago

Add Solr Gem to find as a feature so can switch between Opensearch and Solr

### Search results page
- [x] https://github.com/alphagov/datagovuk_find/issues/1318
- [ ] https://github.com/alphagov/datagovuk_find/pull/1337
- [ ] https://github.com/alphagov/datagovuk_find/issues/1335
- [x] Don’t clear search keywords when filters are applied
- [ ] https://github.com/alphagov/datagovuk_find/pull/1344
- [ ] https://github.com/alphagov/datagovuk_find/issues/1336
- [ ] https://github.com/alphagov/datagovuk_find/issues/1354
- [ ] https://github.com/alphagov/datagovuk_find/issues/1339
- [ ] https://github.com/alphagov/datagovuk_find/issues/1340
- [x] Fix removing filters
- [ ] https://github.com/alphagov/datagovuk_find/issues/1348
- [ ] Too many results returned - deleted datasets? Filter for state: "active"
- [ ] ? Sanitise input, use strong params
- [ ] ? Escape special Solr characters in query
### Individual dataset page
- [x] https://github.com/alphagov/datagovuk_find/issues/1316
- [ ] https://github.com/alphagov/datagovuk_find/pull/1359
- [ ] https://github.com/alphagov/datagovuk_find/issues/1327
- [ ] https://github.com/alphagov/datagovuk_find/issues/1317
- [ ] Handle not found datasets e.g. /3c0d0074-05a3-491a-a144-89ea54654d47/aberdeen-city
- [ ] (NOT NOW) Formats should be derived e.g. /562eb38e-387a-440d-bc61-c7cb7e927fba/coastal-physiographic-features-barrier-beach (https://github.com/ckan/ckan/pull/1350)
- [ ] Issue rendering /dataset/solr/41600666-30dc-4243-9b64-4be3e8f5c65f/spend-over-f25-000-in-2gether-nhs-foundation-trust
- [ ] Legacy datasets - probably don't need to worry
- [ ] (NOT NOW) Tests are slow and difficult to extend. WIP: https://github.com/alphagov/datagovuk_find/compare/main-solr...solr-use-factories-for-testing
### Testing, pre-launch tasks
- [ ] Check Solr performance and carry out load testing
- [ ] Manually test popular URLs https://lookerstudio.google.com/u/0/reporting/1dLIysu5Ie131ZL3gGT7zyWBhffNgTSX2/page/cdu
- [ ] Comms, scheduled maintenance?
kentsanggds commented 2 months ago

We had a chat about how to proceed with progressing and reviewing the work in smaller chunks.

Proposal is to create another branch main-solr from which other solr update branches will be fed into. This will ensure that smaller PRs can be reviewed and merged into it without affecting any deployments to production.

The integration cluster can be updated to point to commits on this branch specifically when there is a need to deploy things for testing on EKS.

As there is generally very little development on the Find application I don't forsee any issues around the branch drifting from the main branch.