search

pobrn / mktorrent

A simple command line utility to create BitTorrent metainfo files
Other
465 stars 73 forks source link

mktorrent seems to be able to create torrent files with more bencoded items than clients can handle #34

Open colindean opened 4 years ago

colindean commented 4 years ago

I created using mktorrent a very large torrent file from approximately 5 TB of videos, some ~200 files ranging from a few GB each up to around 75 GB. The torrent file itself is nearly 100 MB. The folks to whom I've distributed the torrent file report that they've not been able to get their clients to read the torrent file.

biglybt:

biglybt error

qbittorrent:

qbittorrent error

I'm having trouble getting it to work in Transmission. My only Transmission instance is the web interface and uploading it just… crashes the web interface. Latest versions, all.

I see two resolutions for this on the mktorrent side:

  1. Emit a warning when some threshold of number of bencoded items or depth of dictionary is met
  2. Do nothing and let others find this bug report, and realize that they need to break the single large torrent file into a few smaller files!
colindean commented 4 years ago

Related: https://github.com/qbittorrent/qBittorrent/pull/8455/files#diff-9aac8235f4555bbff82d790aa3d186d0R107

colindean commented 4 years ago

This was fixed in pmktorrent v1.3: https://github.com/xxkfqz/pmktorrent/commit/12b357f4bb7967bd52bb5fd8149ccf22feb2b57e

ruthtern commented 4 years ago
  1. The torrent will be smaller if the piece size is larger
  2. If clients have a limit - the 2MB limit in rutorrent is well-known - that's their problem
  3. 1TB is probably pushing the limit for most clients. 5TB is asking for trouble
FranciscoPombal commented 4 years ago

This is tracked in qBittorrent here:

https://github.com/qbittorrent/qBittorrent/issues/10913 https://github.com/qbittorrent/qBittorrent/issues/12029

libtorrent 2.0 will make the torrent loading limits even more flexible and configurable, btw.

ruthtern commented 4 years ago

There will always be limits, if only to avoid resource exhaustion Even so, it has always been best practice to choose a larger piece size for larger torrent data sizes. A .torrent file stores a 20-byte hash for each piece. The size of this part of the torrent file is (20 * number of pieces). A .torrent for serving 5TB of data does not need to be 100MB. This is caused by setting the piece size to 1MB. If the piece size is 64MB, the torrent will have about 80000 pieces and will be about 1.6MB

A non-technical point: bittorrent is a sharing system. Very few people are going to dedicate terabytes of disk space to seed a single torrent