Altinn / digdir-slack-bot

MIT License
2 stars 0 forks source link

[polish] Setup scheduled jobs for scraping and search phrase generation #70

Open bdb-dd opened 11 months ago

bdb-dd commented 11 months ago

Description

The freetext index in Typesense should be updated automatically.

Execution alternatives include triggering based on changes to altinn-studio-docs repo, daily schedule. Consider starting with a full rebuild vs incremental sync for search-phrase generation

Use checksums to avoid regenerating vector embeddings when content has not changed.

No response

### Tasks
- [ ] Determine which task runner would be best for each job
- [ ] Create task definition
- [ ] Local test
- [ ] Cloud test