Build re-indexing of search into CI/CD process

[btholt]

As of right now, our docs are not re-index every time we ship new documentation so every time a user searches, they're using an outdated index that doesn't match current documentation. We should rebuild the index every single time new markdown gets committed.

[andinux]

The current search engine has been manually updated with an updated database based on current documentation. To automatically rebuld the index on every commit to docs, we should configure the new nodes on AWS:

• add the documentation.sqlite database
• create a edge function to run the following query SELECT url, replace(snippet(documentation, -1, '<b>', '</b>', '...', 10), x'0A', ' ') as 'snippet' FROM documentation WHERE content MATCH ? ORDER BY rank LIMIT 256;
• check CORS
• use non-linearizable connection to the db
• create a CI/CD process to update the documentation.sqlite and to replace it on these nodes

[marcobambini]

We need a GitHub action that can perform automatically the following steps after each commit:

1. Compile docbuilder from https://github.com/sqlitecloud/tools/tree/main/docbuilder
2. Execute docbuilder <input_docs_path> <output_db_path> where:
   • <input_docs_path> is the local path to docs directory, and
   • <output_db_path> is the full path to the output documentation.sqlite database to be created.
3. Once the process completes the documentation.sqlite database must be UPLOADED WITH REPLACE into the doc-search cluster at https://dashboard.sqlitecloud.io/projects/cdrehk1lik/nodes

[Gioee]

@marcobambini @btholt @andinux can I close this issue? I think we're done with https://github.com/sqlitecloud/docsearch-action