4.2 /info endpoint and save_path inconsistent.

[ShadwDrgn]

qBittorrent version and Operating System

4.2 linux

If on linux, libtorrent-rasterbar and Qt version

irrelevant (as this is a functionality issue and not a crash)

What is the problem

save_path in the /info endpoint returns the download root folder and NOT the torrent's saved files path. If a subdirectory is created (as 4.2 QBT does i'm told), the path becomes wrong because the contents are actually in that subfolder, but save_path isn't updated. That folder's path can't be reliably determined when torrent name has illegal characters in it.

What is the expected behavior

save_path should be the path to the top level of the torrent's contents, and NOT to a root folder where a subfolder has been created.

Steps to reproduce

curl /api/v2/torrents/files?hash=

Extra info(if any)

This causes https://github.com/Radarr/Radarr/issues/5032 in 4.3 because 4.3 no longer seems to save to subfolders AT ALL. Radarr had to workaround the issue by appending torrent.name to the save_path, and this workaround is now broken in QBT 4.3

[bakerboy448]

For posterity, Sonarr as well https://github.com/Sonarr/Sonarr/issues/3968

[ThatNerdyPikachu]

because 4.3 no longer seems to save to subfolders AT ALL.

@FranciscoPombal Do you happen to know if this was an intentional change?

[FranciscoPombal]

@ThatNerdyPikachu IIRC, qBittorrent (> 4.2.5) no longer has the option to create subfolders for single-file torrents. It led to some bigger problems. That said, this API issue needs to be fixed to account for that.

[ThatNerdyPikachu]

no longer has the option to create subfolders for single-file torrents.

@FranciscoPombal I'm honestly not sure if the option ever worked at all - it's the default on 4.2.5 (where create subfolder is enabled by default), and I've never had a subfolder be created for a single file torrent. I've only ever used the WebUI though, maybe that's the issue? /shrug

[FranciscoPombal]

@ThatNerdyPikachu OK, I think I remember the change now: the option was renamed to "Keep top-level folder", because that's what it really did, and it was greyed-out for single-file torrents, since it doesn't make sense for those.

[ThatNerdyPikachu]

@FranciscoPombal in 4.2.5: it's not greyed out for single file torrents in web ui (I can see why, would be a tad annoying to implement) (I assume it doesn't do anything for single file torrents even if it's enabled in 4.2.5?)

So just the name was changed in master, and not the behavior at all? If just the name was changed: why is it affecting the API?

[ShadwDrgn]

@FranciscoPombal QBT 4.3 can't create subfolders for ANY TORRENT of any kind ever. Whether I check the keep-top-level option checked or not no subfolder is EVER created for single file torrents OR multifile torrents. Whether the existing torrent has folders, or doesn't... it NEVER creates a subfolder of any kind. @ThatNerdyPikachu The behavior changed. QBT 4.2 creates subfolders, and 4.3 does not under any circumstance create a folder. Note that in 4.3 the save_path IS correct because no subfolder is created but in 4.2 it's wrong because a subfolder is created.

[ThatNerdyPikachu]

@ShadwDrgn As far as I can tell, for single file torrents it is intentional that no subfolder is created. But for multi file torrents... that definitely sounds like a bug.

Open another issue for the latter?

[ShadwDrgn]

@ThatNerdyPikachu it may sound like a bug but nothing in the program in any way indicates a subfolder SHOULD be made. it doesn't seem like a bug at all to me. I have a folder set to put my torrents in and it puts them in that folder. It doesn't put them in some mystical subfolder not indicated by any setting anywhere.

If I'm going to create another issue for that, I'd need some context about what the issue even is, as the current behavior seems intuitive based on the options available. I assume the only bug is the one in this issue where 4.2 apparently DOES create subfolders but does NOT report them in save_path.

[ThatNerdyPikachu]

@ShadwDrgn For example, in 4.2.5, when I add a multi file torrent (let's call it test), and save it to /torrents, it downloads as so:

/torrents/
    test/
        a
        b
        c

As you're describing it, does 4.3 download as so:

/torrents/
    a
    b
    c

Or am I misunderstanding your explanation?

[ShadwDrgn]

@ThatNerdyPikachu That's correct. You're understanding it. It's worth noting, though, there's no indication anywhere in any options or torrent add dialog that would indicate that a subfolder named "test" should be created. That leads me to believe the behavior is NOT a bug in 4.3 but that the bug is that in 4.2 save_path says "/torrents/" and not "/torrents/test" which is where the files in fact are in 4.2.

[FranciscoPombal]

The behavior described in https://github.com/qbittorrent/qBittorrent/issues/13389#issuecomment-699565853 is the correct one. In 4.3.0, the first case is the default. If you uncheck "keep top level folder", you get the second result.

there's no indication anywhere in any options or torrent add dialog that would indicate that a subfolder named "test" should be created.

Yes there is, in 4.3.0 it's the "keep top-level folder" option, and it's ON by default.

That leads me to believe the behavior is NOT a bug in 4.3 but that the bug is that in 4.2 save_path says "/torrents/" and not "/torrents/test" which is where the files in fact are in 4.2.

The save_path should in fact point to /torrents. Imagine you've got 3 multi-file torrents, test1, test2 and test3, and save them all to /torrents. What's the name of the root folder that contains all the torrents? /torrents. Thus, torrents/ is the save path.

[ShadwDrgn]

The behavior described in #13389 (comment) is the correct one. In 4.3.0, the first case is the default. If you uncheck "keep top level folder", you get the second result.

I never get the behvior in the first case. No folder named "test" is ever made unless the contents of the torrent happens to have a folder named "test"

there's no indication anywhere in any options or torrent add dialog that would indicate that a subfolder named "test" should be created.

Yes there is, in 4.3.0 it's the "keep top-level folder" option, and it's ON by default.

It's interesting you say there is because that's NOT what that option does. In fact what the option DOES seem to do is when unchecked it DELETES the folder inside the contents of a torrent if one exists. QBT 4.3 does NOT create a subfolder whether that option is checked or not regardless of single/multi-file torrent.

The save_path should in fact point to /torrents. Imagine you've got 3 multi-file torrents, test1, test2 and test3, and save them all to /torrents. What's the name of the root folder that contains all the torrents? /torrents. Thus, torrents/ is the save path.

Assuming save_path is supposed to point to the contents of the torrent in 4.3 given the current behavior it should work as follows: unchecked "keep": save_path=/torrents checked: save_path=/torrents

This ^ is in fact the current behavior and i'm fine with it. HOWEVER in 4.2 subfolders are actually CREATED that don't exist in the torrent eg: torrent contents: Torrent name: "MY STUFF" /videos/1.avi /videos/2.avi

Actual contents of the torrent are in /torrents/MT STUFF/, so the "root" folder for the contents of "MY STUFF" is not /torrents/ it is in fact /torrents/MY STUFF as mentioned. However in 4.2 you get this: save_path=/torrents That results in there being no automatic method for finding the specific torrent unless you just guess that there's a subfolder with the torrents name. That's not always reliable/possible because some torrents have illegal chars in the name, but subfolders still get made with some name QBT determines but never reports in the API, and this is what this issue is for. EDIT: After a slew of 90 edits to make the above make sense i believe it does now. sorry if you all got notified on the billions of edits.

[ThatNerdyPikachu]

I never get the behvior in the first case. No folder named "test" is ever made unless the contents of the torrent happens to have a folder named "test"

So I'm pretty sure that another issue about this behavior specifically should be opened, so it can be fixed, if what you mean is that no matter the value of Keep top level folder (again, that naming is absurd), a subfolder is never created for multi-file torrents, which is what 4.2 does now.

[ShadwDrgn]

@ThatNerdyPikachu I don't think that option is SUPPOSED to create a subfolder whether checked or not. It's meant to delete a subfolder in the contents if one exists when unchecked, which is what it does. I'm not sure though. If it IS supposed to create a subfolder, it should be renamed "Never create subfolder" and when checked a subfolder won't be made for ANY torrent, but when unchecked a subfolder is made for multi-file torrents. ALSO the current behavior the removes the subfolder in the contents of torrents that have a subfolder in them would need to be removed. EITHER WAY save_path should reflect the path to the contents of the torrent be that a subfolder or not.

[ThatNerdyPikachu]

The behavior described in https://github.com/qbittorrent/qBittorrent/issues/13389#issuecomment-699565853 is the correct one. In 4.3.0, the first case is the default. If you uncheck "keep top level folder", you get the second result.

Sorry, I'm confused. So is that (the behavior linked in my linked comment) not what's happening in 4.3? So did they just... remove the option for it? (But apparently Keep top level folder is just a renamed Create subfolder????)

[ShadwDrgn]

Sorry, I'm confused. So is that (the behavior linked in my linked comment) not what's happening in 4.3? So did they just... remove the option for it? (But apparently Keep top level folder is just a renamed Create subfolder????)

In 4.3 subfolders are no longer created. keep top level folder doesn't create subfolders, it does the following:

torrent named "stuff" contents: /videos/1.avi /videos/2.avi

If "keep" checked results in: /torrents/videos/1.avi /torrents/videos/2.avi

If "keep" unchecked: /torrents/1.avi /torrents/2.avi

no folder named "stuff" is made under any circumstance.

[ThatNerdyPikachu]

So then, according to @FranciscoPombal (https://github.com/qbittorrent/qBittorrent/issues/13389#issuecomment-699566864), it should create subfolders, and this is a bug.

[ShadwDrgn]

@ThatNerdyPikachu If it should create a subfolder there is no indication anywhere that it should be doing that and if it should... then there are multiple bugs. One of which is that it's deleting folders and doing what you would expect an option named "keep top level folder" would do. I believe @FranciscoPombal is simply mistaken about what the option's behavior is and there is no bug. I did address that in my reply to him saying that that option does not in fact do that. If you would like to create an issue for that, then feel free. I would suggest you make sure they remove all current functionality of that option when they "fix" it too.

Frankly all I care about is save_path being accurate, which is what THIS issue is for.

[ThatNerdyPikachu]

Ah, got it. Thanks. (Yeah, afaik it (along with every other torrent client) has always made subfolders for multi-file torrents, if it isn't in 4.3, then.... yikes.)

[thalieht]

IIRC, qBittorrent (> 4.2.5) no longer has the option to create subfolders for single-file torrents.

It never did actually.

This option was just renamed to "Keep top-level folder", nothing else, because people kept thinking it creates a subfolder even if there is none (or it should: there are already issues requesting it). So as @ShadwDrgn said:

It's meant to delete a subfolder in the contents if one exists when unchecked, which is what it does.

Having said that, i have no idea if save_path was changed in the API since 4.2.5 but it wasn't from this change.

[ShadwDrgn]

After days of people arguing back and forth about this keep top level folder option, I just want to make sure everyone is clear that save_path has never been pointing to the torrent contents because of subfolders that get automatically created based on the torrent's name. I don't know if QBT was doing this or libtorrent, but it WAS happening which is why sonarr/radarr use save_path + torrent name. In 4.3 this isn't happening anymore and save_path seems inadvertently fixed, which breaks the sonarr/radarr workaround to save_path having been wrong all the time before. so now in 4.3 nothing ever gets imported.

[bakerboy448]

Clarifying point to a comment I saw: The ARR folks don’t have a client whatsoever. ARRs link to a user’s torrent client.

Dragon it sounds like you hit the nail on the head with that most recent comment.

To quote one of the Sonarr Devs on the sonarr git issue:

This is likely also why save_path points to the category folder, rather than the actual torrent sub folder. Because the 'fake' torrent folder is considered part of the torrent. I don't know if stripRootFolder is called inadvertently, or that a bug in qbit or libtorrent prevents libtorrent from properly adding the torrent name.

With wrong vs correct I wanted to point out that it doesn't matter what we believe is correct/accurate or not. The behavior qbit exhibited in the past is the relevant part and should be considered part of the api contract. Changing something isn't a problem, changing it in a way that breaks backward compatibility is. Care must be taken, regardless of whether it's a bug or simply an improvement.

[ShadwDrgn]

The problem with considering the torrent name subfolders a part of the "api contract" is that it doesn't work. It's completely arbitrary, and the subfolder created is not always the name of the torrent because sometimes the name of the torrent has illegal characters in it. I'm fine with not changing save_path as long as a NEW key is made available like content_path or something that DOES point to the correct path. Whatever is done, all I care about is that arr is able to do imports again in 4.3. Right now all automation by arr software does not work with the latest QBT.

[jobrien2001]

Does anyone know what commit started this? I want to revert back :D

[ShadwDrgn]

I assume if you want to revert you'll have to go back to 4.2.5 I definitely wouldn't hold my breath on a fix either, as there are thousands of open issues and this thread is full of lots of confusion. The sonarr/radarr folks seem to want to call it a qbt bug too so basically your whole media automation stack will not work on qbt 4.3 for the foreseeable future as far as i can tell.

[glassez]

This option was just renamed to "Keep top-level folder", nothing else, because people kept thinking it creates a subfolder even if there is none

Yes.

[ShadwDrgn]

update! good news AND bad news. I got to the root of this with the help of some folks over at radarr development discord. Some trackers give torrent files with names that do NOT match the the actual name of the torrent. When looking in the .torrent file the name is the same as the first subfolder. IF QBT did .torrent file imports the same way it did magnet imports (rename it once metadata is downloaded based on the torrent name IN the metadata) this would ALL fix itself. it seems to have NOTHING to do with 4.2 vs 4.3. it's actually because of torrent file names

[glassez]

it seems to have NOTHING to do with 4.2 vs 4.3. it's actually because of torrent file names

👍

Some trackers give torrent files with names that do NOT match the the actual name of the torrent.

Yes. The name of .torrent file and the name of torrent are different things.

the name is the same as the first subfolder.

Torrent metadata has no additional "name" field. So name of torrent follows the name of root item of its content. But once torrent is added qBittorrent allows you to change torrent name separately.

IF QBT did .torrent file imports the same way it did magnet imports (rename it once metadata is downloaded based on the torrent name IN the metadata) this would ALL fix itself.

Sorry, it is unclear for me what do you really mean.

[ShadwDrgn]

@glassez I'll try my best to clarify. When I download a torrent by it's magnet link, the torrent starts out named whatever is in the magnet link. Magnet links consist of basically an info hash and a name.

Once QBT finishes downloading metadata for that torrent it gets renamed immediately. This name (the one QBT renames it to after downloading the metadata) seems to always be the name that clients like sonarr/radarr would accept and is the same name that you would find if you did a strings on the torrent file and searched for regex: /name\d\d:/ and looked at what came after. If QBT had an option on loading a .torrent file of having this EXACT same renaming behavior it would solve all of the problems that are caused by frustrating indexers who choose to rename their torrent files for some stupid reason.

Does this clear things up? You can test this by finding ANY torrent and generating a magnet link from it's info hash using tools like: https://hardrisk.github.io/magnet/ put "testing" as the name, then import it to QBT, and watch the name magically change to what we want it to be.

What I'm suggesting is a feature for this renaming to happen on torrent files as well (preferably per torrent with an option to choose the default toggle state in global settings) we'd need to be able to have it globally set to be able to fix the sonarr/radarr issues.

[ShadwDrgn]

Also recent comments on the linked sonarr issue above should help clarify further. They have a pull request that's a WIP currently, but doesn't look like it's had much work done on it, so I'm hoping a change in QBT might be easier and/or faster.

[glassez]

Once QBT finishes downloading metadata for that torrent it gets renamed immediately.

Yes. It is. The name from Magnet is considered as temporary.

If QBT had an option on loading a .torrent file of having this EXACT same renaming behavior it would solve all of the problems that are caused by frustrating indexers who choose to rename their torrent files for some stupid reason. Does this clear things up?

Sorry, but no. I don't understand how you want to rename it. If I understand correctly, you are satisfied with the torrent name obtained from the metadata. And you are satisfied that torrents added via Magnet links end up with a name obtained from the downloaded metadata. So the torrent will have the same name regardless of whether it was added via magnet link or .torrent file. So what's the problem?

[ShadwDrgn]

I don't understand how you want to rename it.

@glassez I'm not sure how I'm still unable to communicate this. I want to rename it the exact same way magnets are renamed.

If I understand correctly, you are satisfied with the torrent name obtained from the metadata.

Correct.

you are satisfied that torrents added via Magnet links end up with a name obtained from the downloaded metadata.

Correct

So the torrent will have the same name regardless of whether it was added via magnet link or .torrent file

No it will not. If it's added in whatever manner Sonarr/Radarr add it the name is often wrong entirely. Maybe it's not adding as a torrent file, but as a .torrent link that causes it. Either way if there were the ability to have QBT automatically do whatever the magnet link behavior is on ALL torrents, this wouldn't be a problem.

So what's the problem?

The problem is when tools like Radarr/Sonarr create torrents in QBT via whatever means they use* the torrent is named based on seemingly the "filename" it is then never renamed to the metadata name.

Note: I'm talking to radarr dev team now to see if the "rename" field is being specified when adding torrents, to see if that's the root of this problem. It does not appear that Radarr/Sonarr use the rename field. https://github.com/Radarr/Radarr/blob/76cabb4927d7aa7922cb2985f098843139751cfa/src/NzbDrone.Core/Download/Clients/QBittorrent/QBittorrentProxyV2.cs#L110

and

https://github.com/Radarr/Radarr/blob/76cabb4927d7aa7922cb2985f098843139751cfa/src/NzbDrone.Core/Download/Clients/QBittorrent/QBittorrentProxyV2.cs#L134

I'm starting to wonder if the behavior that does this rename from a "temporary name" was the thing that changed between 4.2 and 4.3 that's throwing me off because on testing just now with a problem torrent i know existed in 4.3 i don't seem to have the issue (after downgrading to 4.2), but before i downgraded from 4.3 it definitely wasn't getting renamed.

[FranciscoPombal]

@glassez https://github.com/Sonarr/Sonarr/issues/3968#issuecomment-710581036

[ShadwDrgn]

After a bit of testing it seems like the import behavior in 4.2 has changed in 4.3 and torrents are no longer renamed after downloading metadata where they were before in 4.2 and this is the actual cause. Sonarr/Radarr are very close to a fix on their end (not an ideal one, but one that works).

[winklevos]

When you consider this is the example save_path from the API docs there is clearly miss alignment "save_path":"/Downloads/debian-8.1.0-amd64-CD-1.iso",

@ShadwDrgn I suspect they are looking at /api/v2/torrents/files?hash=<> and having to loop through all torrents to get details of the files / subdir

It is quite a simple fix for qbit, as torrent name, filename, and content folder are all possibly different just add a new bit of metadata to /api/torrent/info content_path: "/file.txt" or content_path: "/subfolder". This can easily be combined with save_path to get a valid reliable location.

[ShadwDrgn]

@winklevos I agree the subfolder being in a content path variable in /info would handily solve this problem without *arr having to hit the files endpoint every import!

[glassez]

I'm sorry, I've never used *arr software. Could you describe its purpose and workflow so that I can understand what you need? Why exactly do you want to know where the torrent files are located? Seems the most reliable way for client app is to obtain save path and file paths and then analyze them.

[bakerboy448]

I'm sorry, I've never used *arr software. Could you describe its purpose and workflow so that I can understand what you need? Why exactly do you want to know where the torrent files are located? Seems the most reliable way for client app is to obtain save path and file paths and then analyze them.

Sonarr/Radarr (et.al) hardlink the movie file and some support files from the torrent folder to one's media library. the issue is the paths being reported by QBT.

See this comment and related issues for more ARR discussion https://github.com/Sonarr/Sonarr/issues/3968#issuecomment-710581036

[ShadwDrgn]

I can help clarify if you'd like, though it's worth mentioning this issue isn't specific to sonarr and radarr. ANY automation that wants to efficiently hit an endpoint to get an idea of all the torrents qbt has finished and then process the contents of those torrents in some way will run in to this problem.

In 4.2 torrents got renamed on add based on the scraped metadata for that torrent and automation clients (like sonarr/radarr) could rely on save_path + torrent name being the path to the subfolder the contents are in by hitting nothing but the /info endpoint in QBT. This is because torrent name is basically always the name of the first folder present in the torrent.

Some indexers are idiots and rename their torrents which was never a problem in 4.2 because 4.2 QBT renamed them BACK to the correct name from the metadata. 4.3 doesn't do this causing save_path + torrent name to be wrong in the case of indexers (like a certain "leet" indexer") that for some reason take it upon themselves to completely rename the torrent. In these cases save_path + torrent name would be the wrong path to the contents because torrent name is simply wrong.

Radarr/Sonarr are just very complex rss/api scrapers that organize your media for you and add new shows/movies to your torrent client when they become available then when they're copmlete import them into your chosen directories and connects with media centers/etc in your environment to assign metadata and send notifications as you see fit.

Currently the *arr software devs have a pull request in to work around this issue by having to loop through all the torrents you have and hit the content endpoint just as @winklevos above mentioned, but a far more ideal solution would be for the torrents to either be named correctly, OR for QBT to offer the content path or metadata name in the /info endpoint

[winklevos]

My knowledge of C++ is lacking but something like this in /webui/api/torrentscontroller.cpp could work

Where for the torrent you can reliably get the content path from the first file. For a torrent with one file it should just return /filename.ext and for multi level content /subfolder. This way external programs can reliably determine the content location and use OS level operations for the rest.

const char KEY_PROP_CONTENT_PATH[] = "content_path";

in void TorrentsController::propertiesAction()

    if (torrent->filesCount() > 0) {
        QString fileName = torrent->filePath(0); 
        if (fileName.endsWith(QB_EXT, Qt::CaseInsensitive))
            fileName.chop(QB_EXT.size());

        string::size_type loc = fileName.find( "/", 0 );
        if( loc != string::npos ) {
            std::string cpath = fileName.substr(0, loc);
        } else {
            std::string cpath = fileName
        }

        dataDict[KEY_PROP_CONTENT_PATH] = cpath;
    }

[glassez]

Some indexers are idiots and rename their torrents

Please be more precise in your comments and try not to mix things up. What exactly do you mean? What are they renaming? Name of the .torrent file? It never mattered to qBittorrent. Name attribute of magnet links? Perhaps something has changed here, I need to check.

[winklevos]

@glassez I think qbit < 4.3 used to create a folder for torrents containing only one file, that folder was named the same as the torrent name

So /save_path/torrent_name/ gave valid location of the data

[glassez]

in void TorrentsController::propertiesAction()...

FYI, there is TorrentHandle::contentPath() getter exists.

[winklevos]

I would put it together myself but I have zero C++ experience and would need to work out how to compile.

Sounds like it would be easy to expose that in the torrent/info api?

[glassez]

@glassez I think qbit < 4.3 used to create a folder for torrents containing only one file, that folder was named the same as the torrent name

qBittorrent never creates any additional folders. It only allow to strip existing one.

[winklevos]

@glassez yes you are correct.

So for a single file torrent 4.2.1 will rename the torrent to the file name in the client when metadata is downloaded. So torrent name would match the content file path

Likewise for torrents with a dir and content under it the torrent name is the subdir or content directory

This doesn't look to be happening in 4.3.0

[glassez]

Some indexers are idiots and rename their torrents which was never a problem in 4.2 because 4.2 QBT renamed them BACK to the correct name from the metadata.

I wouldn't call the name from metadata as "correct name" since the name of torrent is separate thing in qBittorrent (as I said above you can rename torrent to something different than its content folder/file name). Seems in v4.3 qBittorrent consider the name from magnet link as explicitly set name to prevent renaming initiated behind the scenes. I wouldn't call it a bug. Just a different behavior. Someone may like the previous one, someone - the new one. But hoping for the torrent name, assuming that it matches the folder/file name, is incorrect in qBittorrent terms.

Sonarr/Radarr (et.al) hardlink the movie file and some support files from the torrent folder to one's media library. the issue is the paths being reported by QBT.

If it needs to "hardlink the movie file and some support files" why it doesn't just obtain file paths?

[ShadwDrgn]

If it needs to "hardlink the movie file and some support files" why it doesn't just obtain file paths?

Because previously torrent name always worked and I assume they don't want to have to loop through all torrents all the time hitting individual api endpoints just to get paths for every torrent. A more efficient solution would be if that data were available in the /info endpoint so that automation tools don't have to make multiple API calls. They do have a fix PR in currently that does exactly this, but it's still got some bugs.

Also note that when i say correct, I simply mean the name of the torrent as it appears in the metadata of that torrent. I'm not suggesting the entire contents of the files api endpoint be included in the /info endpoint. a simple field like metadata_name OR base_content would suffice with base_content being the first folder in the content or the filename if it's a single file. and metadata_name being obviously the name from the metadata.

[glassez]

in void TorrentsController::propertiesAction()...

FYI, there is TorrentHandle::contentPath() getter exists.

We can provide it via API.

IIRC, your soft can also remove "name" field from magnets to emulate previous behavior.