bitmagnet-io / bitmagnet

A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
https://bitmagnet.io/
MIT License
2.07k stars 80 forks source link

A new front-end interface powered by Bitmagnet #275

Closed journey-ad closed 1 week ago

journey-ad commented 1 week ago

Please goto #276

Hi everyone,

I made a new web UI using Next.js and NextUI. The repo is here: https://github.com/journey-ad/Bitmagnet-Next-Web. Maybe someone will find it interesting.

It query the DB directly because the original Bitmagnet graphql API didn't meet my needs. I needed accurate total counts for pagination, but The graphql API returned inaccurate numbers. Also, I needed hash queries for torrent info.

The graphql API was slow (over 1 minute for 1.8 million torrents). So, I used the pg_trgm extension and added indexes to the torrents.name and torrent_files.path columns:

CREATE EXTENSION pg_trgm; -- Enable pg_trgm extension

-- Create indexes on `torrents.name` and `torrent_files.path`
CREATE INDEX idx_torrents_name_1 ON torrents USING gin (name gin_trgm_ops);
CREATE INDEX idx_torrent_files_path_1 ON torrent_files USING gin (path gin_trgm_ops);

This reduced the response time to a few hundred milliseconds ~ a few seconds, which should be good enough for personal use.

I am not a professional backend developer and don't have extensive PostgreSQL knowledge, just sharing some recent work. Hope it helps someone.

mgdigital commented 1 week ago

Hi, thanks for sharing - your frontend looks great and I hope people can start building apps and alternate frontends.

Can I give you some feedback:

I needed accurate total counts for pagination, but The graphql API returned inaccurate numbers

Please see here for the reasoning behind this: https://bitmagnet.io/faq.html#why-doesnt-bitmagnet-show-me-exactly-how-many-torrents-it-has-indexed - you can get accurate counts by setting the aggregationBudget very high in the graphql query, but bear in mind this will be slow when there are many rows to count. People may commonly have up to 20 million torrents indexed, and returning accurate counts just isn't feasible performance-wise for this size of index.

The graphql API was slow (over 1 minute for 1.8 million torrents)

Can you confirm which query was slow? Did you set an appropriate limit?

So, I used the pg_trgm extension and added indexes to the torrents.name and torrent_files.path columns

The tsv field on the torrent_contents table is intended for full text search, it already has an index.

Finally, I'd encourage you to remove the copyrighted content from your screenshots and replace with some public domain works to avoid unwanted attention.