search

hoarder-app / hoarder

A self-hostable bookmark-everything app (links, notes and images) with AI-based automatic tagging and full text search
https://hoarder.app
GNU Affero General Public License v3.0
6.63k stars 240 forks source link

chore: Add a new sqlite based queue package #308

Closed MohamedBassem closed 4 months ago

MohamedBassem commented 4 months ago

chore: Add a new sqlite based queue package

cloudflare-workers-and-pages[bot] commented 4 months ago

Deploying hoarder-landing with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1d458d5
Status: ✅  Deploy successful!
Preview URL: https://34cafb6a.hoarder-landing.pages.dev
Branch Preview URL: https://sapling-pr-archive-mohamedba.hoarder-landing.pages.dev

View logs

cloudflare-workers-and-pages[bot] commented 4 months ago

Deploying hoarder-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1d458d5
Status: ✅  Deploy successful!
Preview URL: https://e398f664.hoarder.pages.dev
Branch Preview URL: https://sapling-pr-archive-mohamedba.hoarder.pages.dev

View logs

MohamedBassem commented 4 months ago

why it is a separate db though?

Mostly for isolation purposes. I didn't want to expose hoarder to too much knowledge about what happens in the queue layer. The queues are mostly separate state that's not related to the data stored in hoarder, so I didn't see a reason to tie them to the main database. They are also kinda ephemeral. I'm planning to eventually extract it into a separate package, and having that package own the database makes things simpler.

I would love to be able to click on it (or create a filter) to see all the bookmarks that are pending/had issues/whatever, so you can reschedule them.

The state is already reflected in the database (bookmarks.taggingStatus and bookmarkLinks.crawlStatus), so you can already do this without introspecting the queue status.