search

louis030195 / ega

3 stars 0 forks source link

Scrap history & index it #4

Closed louis030195 closed 2 years ago

louis030195 commented 2 years ago

image

louis030195 commented 2 years ago

Every day (google cloud function pubsub, distributed sharding?):

  1. Query bigquery
  2. scrap all unique URL
  3. write html to bigquery (same table or new one)
  4. another function build embeddings & indexes on trigger

Then build web app / google assistant extension to search memory by voice