search

Altinn / digdir-slack-bot

MIT License
2 stars 0 forks source link

[scraper] Migrate synonyms and curations from current version of a collection #60

Closed bdb-dd closed 9 months ago

bdb-dd commented 11 months ago

Description

The typesense scraper handles indexing all content on docs.altinn.studio. It creates a new collection, keeping the current collection active for handling requests during indexing. When the indexing job is complete, the collection alias is updated to point to the new collection.

However, existing synonyms and curations are not automatically migrated.

Let's add a function to the scraper that reads all current synonyms and curations and uploads them to the new collection, before updating the alias.

Additional Information

No response

Tasks

No response

Acceptance Criterias

No response

bdb-dd commented 9 months ago

On hold until ColBERT reranking alternative has been evaluated.

bdb-dd commented 9 months ago

ColBERT reranking has proven to be a good alternative, especially now that we have efficient GPU hosting.