crazy-max / docker-rtorrent-rutorrent

rTorrent and ruTorrent Docker image
MIT License
483 stars 107 forks source link

Example for overriding default temp, complete directories #105

Closed Rast1234 closed 2 years ago

Rast1234 commented 3 years ago

This is not an issue. This is sort of an explanation why and how directories are set up and how to change this behavior. Maybe it will help someone, maybe maintainer will want to add it into README...

I spent several hours understanding syntax of .rc files and understanding what should be done to change default behavior of this image.

Default behavior is:

Wanted behavior:

Default behavior is hard-coded into the image as a replacement for Autotools plugin. How it works:

Code in .rtlocal.rc creates methods like cfg.download_complete which return corresponding paths and also sets default downloads directory:

method.insert = cfg.download, private|const|string, (cat,"/downloads/")
method.insert = cfg.download_complete, private|const|string, (cat,(cfg.download),"complete/")
method.insert = cfg.download_temp, private|const|string, (cat,(cfg.download),"temp/")

# Default directory to save the downloaded torrents
directory.default.set = (cat,(cfg.download_temp))

Later these functions are used in .rtorrent.rc:

# Move finished (no need Autotools/Automove plugin on ruTorrent)
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="

We can change this line to use another method which returns /downloads directory:

method.insert = d.get_finished_dir, simple, "cat=$cfg.download=,$d.custom1="

Also we need to redefine default download directory. I did it the same way to avoid syntax errors because i'm not sure how it works:

method.insert = cfg.download_temp2, private|const|string, (cat,(cfg.download),".downloading/")
directory.default.set = (cat,(cfg.download_temp2))

Note 1: this works with labels as subfolders so Autotools plugin is not needed. Probably there is a good reason to prefer this solution over Autotools but i don't know, really.

Note 2: no one will create custom directories and set permissions. This has to be done manually.

Note 3: complete and temp dirs will be created by startup scripts anyway. you can not get rid of them. As a hack, you can write a script which removes them, and bind-mount it into s6 startup directory:

# this is a volume in docker-compose
- "/path/to /your/script/05-rm-dirs.sh/:/etc/cont-init.d/05-rm-dirs.sh"

But at this point you should probably be re-building image...

ben-64 commented 3 years ago

Do you know if it's possible to override some variables from .rtlocal.rc ? I tried to do that:

...
method.insert = cfg.watch,    private|const|string, (cat,(cfg.basedir),"watch/")
...

And the result is :

rtorrent: Error in option file: ~/.rtorrent.rc:25: Invalid key.
Rast1234 commented 3 years ago

@ben-64 I think method.insert creates new function in rtorrent, and if you try to create something that is created somewhere else or by default, you will get error because two methods with the same name can not coexist.

Try to look at files and find how variables are set, maybe something like directory.default.set = ... is the right syntax.

ownbee commented 3 years ago

I was struggling to understand this as well (new to rtorrent). And it is awfully complex (for a newbie) to change download directories since you don't expect it to be so much scripting to achieve such simple thing...

I wanted to change root download dir since there are other containers that depend on a certain structure. here is how I solved it, same as you have explained (.rtorrent.rc):

...

# Custom download locations (added by me)
directory.default.set = (cat,"/data/rtorrent_downloads/temp/")
method.insert = cfg.download_complete_custom, private|const|string, (cat,"/data/rtorrent_downloads/")

# Move finished (no need Autotools/Automove plugin on ruTorrent)
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete_custom=,$d.custom1="
                                                            ^--- change this to the method created above
method.insert = d.move_to_complete, simple, "d.directory.set=$argument.1=; execute=mkdir,-p,$argument.1=; execute=mv,-u,$argument.0=,$argument.1=; d.save_full_session="
method.set_key = event.download.finished,move_complete,"d.move_to_complete=$d.data_path=,$d.get_finished_dir="

...

Would be great if there was a way to configure directories with docker envs like:

environment:
  DOWNLOAD_ROOT: "/data/rtorrent_downloads/"
  DOWNLOAD_COMPLETE: ""
  DOWNLOAD_INCOMPLETE: "temp"
fastbytes commented 3 years ago

PR submitted..

119 - Add support for custom download paths

environment:
  DOWNLOAD_ROOT: "/downloads"
  DOWNLOAD_COMPLETE: "complete/"
  DOWNLOAD_INCOMPLETE: "temp/"
netty0 commented 2 years ago

This is not an issue. This is sort of an explanation why and how directories are set up and how to change this behavior. Maybe it will help someone, maybe maintainer will want to add it into README...

I spent several hours understanding syntax of .rc files and understanding what should be done to change default behavior of this image.

Default behavior is:

* place downloading files into `/downloads/temp`

* place finished files into `/downloads/complete`

Wanted behavior:

* place downloading files into `/downloads/.downloading`

* place finished files into `/downloads`

Default behavior is hard-coded into the image as a replacement for Autotools plugin. How it works:

Code in .rtlocal.rc creates methods like cfg.download_complete which return corresponding paths and also sets default downloads directory:

method.insert = cfg.download, private|const|string, (cat,"/downloads/")
method.insert = cfg.download_complete, private|const|string, (cat,(cfg.download),"complete/")
method.insert = cfg.download_temp, private|const|string, (cat,(cfg.download),"temp/")

# Default directory to save the downloaded torrents
directory.default.set = (cat,(cfg.download_temp))

Later these functions are used in .rtorrent.rc:

# Move finished (no need Autotools/Automove plugin on ruTorrent)
method.insert = d.get_finished_dir, simple, "cat=$cfg.download_complete=,$d.custom1="

We can change this line to use another method which returns /downloads directory:

method.insert = d.get_finished_dir, simple, "cat=$cfg.download=,$d.custom1="

Also we need to redefine default download directory. I did it the same way to avoid syntax errors because i'm not sure how it works:

method.insert = cfg.download_temp2, private|const|string, (cat,(cfg.download),".downloading/")
directory.default.set = (cat,(cfg.download_temp2))

Note 1: this works with labels as subfolders so Autotools plugin is not needed. Probably there is a good reason to prefer this solution over Autotools but i don't know, really.

Note 2: no one will create custom directories and set permissions. This has to be done manually.

Note 3: complete and temp dirs will be created by startup scripts anyway. you can not get rid of them. As a hack, you can write a script which removes them, and bind-mount it into s6 startup directory:

# this is a volume in docker-compose
- "/path/to /your/script/05-rm-dirs.sh/:/etc/cont-init.d/05-rm-dirs.sh"

But at this point you should probably be re-building image...

Thank you for this.

crazy-max commented 2 years ago

Same as

No need to change paths inside the container. The proper way to do this is to use custom bind mounts on your side.

hollanbm commented 2 years ago

Same as

* [Allow to change the "$topDirectory" variable in rutorrent #7](https://github.com/crazy-max/docker-rtorrent-rutorrent/issues/7)

* [rtorrent .rtlocal.rc forcing temp and complete folder inside download directory #44](https://github.com/crazy-max/docker-rtorrent-rutorrent/issues/44)

* [Question: Is it possible to override the default download folder #57](https://github.com/crazy-max/docker-rtorrent-rutorrent/issues/57)

No need to change paths inside the container. The proper way to do this is to use custom bind mounts on your side.

The issue is due to other applications (Sonarr/Radarr), using RPC, to retrieve info about a torrent, which returns paths relevant to the container.

The paths in the download client, need to be accessible by these application. Otherwise, you need to maintain a mapping translation between the two containers.

I have 4 instances of sonarr/radarr. It would be much simpler to change the download path inside the container, than to instead go and update 4 other applications as well as to maintain a mapping from between container paths

hollanbm commented 2 years ago

see https://github.com/hollanbm/docker-rtorrent-rutorrent if you'd like to remap the container download path with an env var

ahaghgoo commented 2 years ago

I'm coming from using a QNAP .qpkg of rtorrent where I was using AutoTools. I am wanting to eliminate the use of AutoTools and work with using the rtorrent.rc to get the same functionality going. Currently, the use of labels and using the watch folder creates this situation where the files moved from 'temp' to 'complete' creates what I've shown in the screenshot. The labelling in the 'temp' folder doesn't have this problem.

I'm wondering what I will need to do in order to create a clean workflow. If I am wanting to move the files to a folder outside the docker container, how would I "mount" the folders within the docker-compose file and how would this be used within rtorrent? Would I specify if in rtorrent.rc?

I'm sorry for the newbie questions, but I'm wondering if anyone here could assist me at elucidating further what was initially discussed. Thank you for the consideration!

Screen Shot 2022-05-27 at 19 11 08 Screen Shot 2022-05-27 at 19 12 36

hollanbm commented 2 years ago

I'm coming from using a QNAP .qpkg of rtorrent where I was using AutoTools. I am wanting to eliminate the use of AutoTools and work with using the rtorrent.rc to get the same functionality going. Currently, the use of labels and using the watch folder creates this situation where the files moved from 'temp' to 'complete' creates what I've shown in the screenshot. The labelling in the 'temp' folder doesn't have this problem.

I'm wondering what I will need to do in order to create a clean workflow. If I am wanting to move the files to a folder outside the docker container, how would I "mount" the folders within the docker-compose file and how would this be used within rtorrent? Would I specify if in rtorrent.rc?

I'm sorry for the newbie questions, but I'm wondering if anyone here could assist me at elucidating further what was initially discussed. Thank you for the consideration!

Screen Shot 2022-05-27 at 19 11 08 Screen Shot 2022-05-27 at 19 12 36

I forked and added an init script on top to let you change the default download dir

https://github.com/hollanbm/docker-rtorrent-rutorrent

as for mounting paths in the container, you’d use volume or bind mounts

ahaghgoo commented 2 years ago

Thank you Brad for the quick reply. I played around with it, but I'm not sure how this is to help me? I have half of the AutoTools enabled, meaning that I am able to use the watch folder to then label my torrents in the hierarchical structure I want to have; I tag the torrent to the type of file that it is representing; within the 'downloads' folder the file is downloaded in the 'temp' folder where the folder hierarchy created by the tagged torrent is retained; the torrented file is then moved to the 'complete' folder within the 'downloads' folder when it's finished.

The trouble is when this container moves the file to the 'complete' folder and messes things up by not retaining the same folder structure that was first created in the 'temp' folder. It's just creating one folder and flattens out the hierarchical folder structure.

So essentially this is a two-step process with the second step malfunctioning. Ideally, I'd like to create a three step process of being able to hard link the files so that I can change them around but keep the "source" file at its original location for seeding purposes. Which is now making me hesitant at moving away from AutoTools as it did three things in one; labeled torrents by how they were added through the watch folder's hierarchy; moved the completed file while reflecting the folder hierarchy by which the torrent was imported from the 'watch' folder; created a hard-linked copy of the file and moved it to a different location; continued to retain the folder structure from where the hard-linked file originated.

Is this possible to recreate without the use of AutoTools?

ahaghgoo commented 2 years ago

I just wanted to leave a concise note to my previous posts: my intended goal is to hard link the torrents, which currently only can be done via ruTorrent's AutoTools.

bholland-bh commented 2 years ago

I just wanted to leave a concise note to my previous posts: my intended goal is to hard link the torrents, which currently only can be done via ruTorrent's AutoTools.

You can't hard link across docker volume mounts. Each bind mount is treated as a separate filesystem internally in docker.

If you want to hardlink between folders, you'll need to mount a single directory, containing both subdirectories. Then you can hard link between them.

ahaghgoo commented 2 years ago

Yes, that's what I intend to do, but I'm not sure how to fully integrate the current hard-coded behavior of the docker in the manner where I can fully utilize the current tools available. I don't want to rewrite what's been done, but what I wish to do is essentially mount a volume that is accessible to the AutoTools file browser so that I can hard link the torrents "outside" of the defined "downloads" volume that I mounted. But where AutoTools opens up is at the mounted "downloads" volume and doesn't go beyond that.

By accomplishing this I can then seed the torrents within the "downloads" folder that I have defined, and then use AutoTools to organize the torrents and then hard link them to a location outside the "downloads" folder.

ahaghgoo commented 1 year ago

Note 3: complete and temp dirs will be created by startup scripts anyway. you can not get rid of them. As a hack, you can write a script which removes them, and bind-mount it into s6 startup directory:

# this is a volume in docker-compose
- "/path/to /your/script/05-rm-dirs.sh/:/etc/cont-init.d/05-rm-dirs.sh"

But at this point you should probably be re-building image...

Has anyone wrote an effective script that would accomplish this? I rather not rebuild the entire image, I just want to not have these dirs be sync'ed in my current structure that I have built around rtorrent.