Predefined directories: Allowing bittorrent to easily be adopted with other applications

[trymeouteh]

Bittorrent allows you to download files by choosing where they are stored on the device which is great. However many apps that work with large files could benefit by using bittorrent without adding a bittorrent client into the app. The problem with using bittorrent client with downloading files for an app is that many apps need the files to be downloaded to a specific directory on the device so the app knows were the file is to work with it and it is not likely the user will know this directory off hand, and the directory maybe hidden, making it harder to save the file to for non-tech savvy users.

If the app is installed as a portable app, the directory maybe different than using the installer. The directory will also vary on what operating system the user is using. This can be resolved with a simple text file stored in a root directory such as /etc/bittorrent-apps/. The text files can store the file usecase with a file path to store and seed the download.

Let me explain one example were Bittorrent could be used to explain this protocol feature request...

Lets say Docker images and Docker containers were released as torrents. When you install Docker CLI on your computer, it will add a bittorrent predefined directory file named docker.path and set the paths for docker images to the directory were docker images are stored and set the paths for docker containers to the directory were docker containers are stored. After installing the Docker CLI, your bittorrent predefined path file will look lile this...

/etc/bittorrent-apps/docker.path

[Docker] # The application tag name that is added to the torrent in the torrent client for organising your torrents

images=/var/lib/docker/images/ # Docker images directory
containers=/var/lib/docker/containers/ # Docker containers directory

When the torrent is added to a torrent client, it will automatically add an application tag (In the example above, it will be tagged under "Docker") for being able to sort hundreds of torrents from several different applications in your torrent client.

Now when you download and open a torrent file that has a predefined path set as docker/images or use a magnet link with a predefined path set to docker/images, your bittorrent client will download and seed the docker images to /var/lib/docker/images/.

I can also think of other usecases for this feature for other apps such as...

• LineageOS updates
• GrapheneOS updates
• Ollama Models

If the directory the application uses for the files has read/write resitrictions, the application can create another directory which can be used that the user has permission to use and the application will use two directories for managing the files, one directory for direct downloads, another directory for bittorrent downloads which will be setup by the application, not bittorrent or the bittorrent client itself.

By adding this predefined path into the bittorrent protocol as a standard, the bittorrent clients can use this to add several features to the clients such as...

• Unable to delete torrents in the bittorrent client that have a predefined directory path and instead it is up to the application to delete the torrent. The torrent client will detect if the file still exists and if it is deleted, it will stop seeding and not have the file listed in the bittorrent client
• Unable to move torrents in the bittorrent client that have a predefined directory path.
• (Advanced Users Only) Add/edit/remove predefined directories on the device by editing the predefined directory files in the bittorrent client.

[the8472]

The docker example doesn't quite work because the on-disk directory structure differs depending on what storage driver you use. E.g. btrfs and the overlay2 storage use different directories. And usually the docker daemon runs as root. You probably don't want a torrent client to have write access to that. Additionally bittorrent doesn't know which file permissions should be set.

A better approach would probably be some sort of tagging or other metadata (similar to mime types) in a torrent that would tell the torrent client to pass the downloaded data to some other software once the download is finished.

That other software can then import the files and if necessary convert them into a preferred structure.

[trymeouteh]

A better approach would probably be some sort of tagging or other metadata (similar to mime types) in a torrent that would tell the torrent client to pass the downloaded data to some other software once the download is finished.

That other software can then import the files and if necessary convert them into a preferred structure.

Passing the downloaded data to another directory once the download is done is not a bad idea.

Would this still allow the torrent client to seed the file after downloading the file without duplicating the file?

The docker example doesn't quite work because the on-disk directory structure differs depending on what storage driver you use. E.g. btrfs and the overlay2 storage use different directories.

This should not matter since docker will inform the torrent client what directory to use by what storage driver docker detects.

[the8472]

Would this still allow the torrent client to seed the file after downloading the file without duplicating the file?

If you have a filesystem that supports reflinks then the other application can just make reflink copies and each can keep its own "copy" while only paying the storage cost once.

Alternatively a torrent client can be implemented to scan a directory tree for files that match things it wants to seed instead of looking at fixed directories. On HDDs this might be painful but on SSDs it should only take a few minutes to tentatively build an index (the total time to fully check everything would depend on how large the set of seeded data is).

[stokebreakup]

If we agree with the observations above, maybe here is a draft proposal based on the "bep_0009" which defines the format of magnet URI format. We have a magnet link with a metadata called path that is responsible for having a predefined path. By clicking on the magnetic link, the user downloads the file to the predefined path. You could have v1 and v2 of the magnet link format: v1: magnet:?xt=urn:btih:<info-hash>&dn=<name>&tr=<tracker-url>&x.pe=<peer-address>&path=<path> v2: magnet:?xt=urn:btmh:<tagged-info-hash>&dn=<name>&tr=<tracker-url>&x.pe=<peer-address>&path=<path>

This does not break the protocol, as the magnet link can be optionally filled. The "path" field, is something predefined that the user can download or change later. If it's something everyone agrees to update (the path field could or could not be included in proposal bep_0009 as something optional).

comments?