Improving application architecture (Redesign).

[glassez]

Lately I've been increasingly involved in the development of qBittorrent. In this regard, I dive into the source code more and more. And the more I do it, the more I cultivate the desire to restore order there. qBittorrent needs overhaul - that is my conclusion. I will not give here the details, so as not to waste my time. Let me just say that there are a lot of problems here, starting with excessively long functions, various inconsistencies and cross-dependencies of different classes and subsystems, and to the lack of a clear architecture. One of the important problems is the widespread dependence of our code from libtorrent code. (Those who now consider me a fool and say "qBittorrent is based on libtorrent", I will answer immediately - no comments about this, if you do not understand what I mean - you can continue to regard me as a fool.) To @sledgehammer999 mostly: The circumstances are so that I can over the next few months to do this job. I would like to discuss issues of cooperation. This work is long and extensive code changes will be made. It is clear that I'm going to target it at a later release (after 3.2). As soon the lack of support for Qt4, I would prefer not to support Qt4. And still it is necessary to strictly set the minimum supported version of Qt5. For the same reasons (major changes) some time after the beginning of my work (a very short time, I think) I will not be able more to rebase my code. And this is a problem because there will be concurrent commits. Any ideas or comments?

Smaller issues. I also have the idea to use our modification of QtSingleApplication in all qBittorrent builds. I want to take as a basis QtSingleApplication from QtCreator, which is more powerful than the one that is now represented in QtSolutions (and which we use). Using libtorrent 0.16.x also questionable, but in any case, I initially will support both branches in order to better design the abstraction layer.

Next, I'll publish materials related to this (plans, ideas, models, and so on).

[sledgehammer999]

You need to check what version of Qt are available in the main distros.

Debian testing(which will become the next stable) has qt5 support. The same goes for Ubuntu. I suspect Mint will not have a problem since it derives from Ubuntu/Debian. Fedora also has qt5. And Arch is a rolling distro so it already has it. I think those are the major distros.

[sledgehammer999]

As for v3.2.0, most of the issues I left in the corresponding milestone are related to the build system. I intend to make the usage of qt4 an explicit option. The script will look for qt5 and fail if it doesn't exist and the invoker has used the "--use-qt4" option. I'll do the same for libtorrent 0.16.x. Default will be libtorrent 1.0.x. Of course the Windows binaries will be build using qt5 and libtorrent 1.0.x.

[ngosang]

And Arch is a rolling distro so it already has it.

The QT5 applications work well. The problem is that QT5 applications do not use the default theme and look very ugly. Qt4 left (the KDE4 theme), Qt5 right.

[sledgehammer999]

The problem is that QT5 applications do not use the default theme and look very ugly.

Is that fixable? For example under GTK DEs you can use the qt4-qtconfig program. Is there a similar solution for qt5 apps?

[ngosang]

Is that fixable?

For now you can not do anything. Qt4 themes are not compatible with Qt5. The Qt guys are working to make a Qt5 theme that mimics the Oxygen theme used in Qt4.

[ngosang]

There is an ugly hack that partially works. You can run a Qt5 app with this parameter "-style=gtk+" and the app uses the Qt5 theme that it's used to run GTK apps in KDE5 desktop. Works only in the apps that doesn't enforce one specific theme. :confounded: :confounded: :confounded: https://bbs.archlinux.org/viewtopic.php?id=160206

[pmzqla]

Oxygen had been already ported to Qt5. The problem is that it depends on KF5. I don't think it will be possible to have oxygen for Qt5 without it. Funnily, qBittorrent blends in perfectly if the DE is not depending on Qt, Gnome for instance.

@ngosang -style gtk should tell qBittorrent to use the style used for GTK application, which can also be qtcurve. The integration is not perfect though. The default theme should be fusion, which is not the one of the second window in the screenshot you posted.

This is how qBittorrent looks with fusion (yes, it's not perfect):

This is how it looks in Gnome (I don't know if it's normal the white line on top of the left column, right above "TORRENTS". I never use Gnome, this screen was taken from inside a virtual machine):

[pmzqla]

I don't know if I should create a separate issue for this, but I think that QBtSession::handleTorrentFinishedAlert() has to be reworked.

Currently it's called when a download is completed and it does the followings:

• Remove .!qB from all the files
• Move the files from the temp folder
• Recheck the torrent
• Launch external program
• (send email)
• Shutdown the computer

The main problem is that the first two operations are asynchronous and the others depend on them. This causes some problems, such as #2107 #2561 and I've seen other problems reported with the extension appended.

[ngosang]

@pmzqla Thanks for the explanation.

[andoruB]

It would be really awesome if some performance improvements can be achieved. Currently I have around 250 torrents loaded in the client, and I have to wait ~3 minutes when I start qBitorrent and it constantly takes about 10% CPU when running (not sure if this is expected behaviour). In other words, same thing @sledgehammer999 mentioned in an earlier comment.

[glassez]

@andoruB Since I don't have much of torrents, I need your help (and other users with a large number of torrents) to test the new behavior, when I do it.

[andoruB]

I forgot to mention that I'm using an older version of qB, one that's made available in the Debian unstable repos (3.1.11) so this version doesn't have any of your modifications included. But I'm building qB source cloned from this git and will let you know how it works. I would be glad to help with my limited expertise :)

[sledgehammer999]

@andoruB CPU during runtime should be minimal with current git master. However, currently there isn't any optimization about the startup.

[andoruB]

Weird, the client startup is now a lot snappier, but the CPU usage during runtime is the same. I built it without the webui, could that contribute to anything?

[glassez]

Yay! My branch is finally released from non-buildable state. This does not mean that I completely finished the redesign - there are still a lot of work. But I almost finished to make the changes that affect a lot of files. Soon I will make some more edits and make preliminary tests and then publish this BIG PR. @sledgehammer999 In this regard, I ask not to merge any significant changes during this time. It's damn hard to support it!

[sledgehammer999]

Good to know. I'll try to comply. However, v3.2.0 takes precedents. I suspect only minor mods are necessary in the files under "qtlibtorrent" folder. I'll make them as PR first and give you a heads up.

[glassez]

As for v3.2.0, most of the issues I left in the corresponding milestone are related to the build system. I intend to make the usage of qt4 an explicit option. The script will look for qt5 and fail if it doesn't exist and the invoker has used the "--use-qt4" option. I'll do the same for libtorrent 0.16.x. Default will be libtorrent 1.0.x. Of course the Windows binaries will be build using qt5 and libtorrent 1.0.x.

You're going to leave support for libtorrent 0.16.x in v3.3? I already finished preparing PR. But it remains to fix support for libtorrent 0.16.x. Or should I drop it?

[sledgehammer999]

I am in a huge dilemma here. I want to drop 0.16 for v3.3.0. I was also considering backporting your restructuring back into v3.2.x. Why? Because there is a ton of PRs waiting to be merged. If I don't backport your changes then the PR authors should need to do double work. One PR for the v3.2.x branch and one for the master. I don't know if they would like to do that. So, the 0.16 dropping, it depends on if I should backport your changes or not. Any suggestions on how to proceed?

[glassez]

Okay, then I'll just finish support for libtorrent 0.16.x. Besides, it will show how decreased dependent code, but also show how to add support for other (newer) versions.

P.S. About backporting... Not sure you will be able to achieve great success in this matter. I am afraid when I see how you (and others) will review my changes - they are huge. As I expected, I won't be able to break it into small independent commits. Although I was on the way and came across some minor problems, I had to correct them on the go, not to spend time on creating and maintaining separate commits. Sorry...

[duytrung]

Using QT, I found that it is not necessary because I found that the user interface of QBittorrent is simple. There is no many things to display except a few buttons, some directory display content of torrent. Focusing on the practical functions such as download algorithms, swarm merging, etc.. probably the best thing :-)

[Chocobo1]

This is what I have in mind (in regard to the big change, not taking future milestones into consideration):

1. fix all v3.2.0 blockers and release v3.2.0
2. merge glassez's redesign asap and branch to v3.3.0
3. PR rebase onto v3.3.0

v3.2.x should only accept maintaining fixes, cherry pick important PR from v3.3.0 to backport v3.3.x will be the new developing branch, also try to give out early alpha builds for users to test and report. And hope future refactoring could be done more easily and in smaller steps.

[glassez]

PR #2892 published.

[sledgehammer999]

@glassez now that #2892 is merged I want to mention something relevant. We should continue using the BT_backup folder. Obviously you used another name because now we use a different naming scheme for the fastresumes. My proposal is to have a transitioning code that will test for the existence of files ending in .fastresume. If it finds them, it will also mark the corresponding .torrent files, otherwise they will be added as not having a fastresume by the fallback mechanism. When the UI is up, inform the user that "You updated from an older version that saved things differently. You must migrate to the new saving system. If you click yes, you will not be able to go back and use an older version than v3.3.0." If user clicks "Abort", qbt should exit. Of course in the meantime no torrents should be loaded, if it is detected that the user is coming from and old installation and before he chooses to migrate. If he chooses to migrate, we read the fastresumes and rename them according to the saved queue number. Then call the default load function.

What do you say?

[glassez]

@sledgehammer999

My proposal is to have a transitioning code

Undoubtedly it should be. But I want something else to change before that. I really don't like that we have to use two files to resume the torrent. I want to do only one .fastresume. libtorrent allows you to save metadata in it. If this works correctly, then we will be able to do it without storing separate .torrent file. This should simplify the resume logic. Once this is done (or rejected) we can work to create a transitional mechanism.

Here I have so far only one comment (offhand):

When the UI is up, inform the user that...

We also need to consider options for NoGUI builds.

[Chocobo1]

Just asking, in the foreseeable future, is there going to be another large refactor?

[glassez]

Redesign is not over yet. But the rest (I think) can be done gradually in small (relatively small) commits.

[ngosang]

@glassez This bug is still there in master https://github.com/qbittorrent/qBittorrent/pull/2892#issuecomment-98361957 Can you take a look again?

[glassez]

@ngosang Can you test #3194? This should fix it.

[sledgehammer999]

@glassez I don't think it is a good idea to merge metadata+resumedata into one file. We already have cases where torrent progress is lost due to various reasons (crashes, OS killing the program, etc). Separate .torrent files, since they written only once, ensure that the user at least has torrent added and it doesn't just vanish if crash happens.

[glassez]

Separate .torrent files, since they written only once, ensure that the user at least has torrent added and it doesn't just vanish if crash happens.

In General, the presence of only the torrent file doesn't give us anything. When we find such a file, it raises some questions without having the answers, we will not be able to do with this torrent nothing useful. Here are those questions:

1. Where to save this torrent? Because the user is unlikely to want to re-download anything big, for example.
2. What are the names of the files contained in this torrent (because they could be renamed)? What about this?

[sledgehammer999]

Clearly .torrent file by itself is the indication of an error. So the least we can do for the user is to inform him of that. With that in mind:

Where to save this torrent? Because the user is unlikely to want to re-download anything big, for example.

Add it in paused mode and point it to the current default path.

What are the names of the files contained in this torrent (because they could be renamed)?

If added in paused, the user can at least review the state and probably re-import the file.

The general idea should be: Do as little as possible in terms of destructive(or writing) operations and wait for the user to take action.

PS: Because in such a catastrophic failure we can't do much else. This is the last line of defense.

[glassez]

Well. In this case, consider #3224 and we can begin to implement a transitioning code.

[sledgehammer999]

@glassez I came here to also suggest you prioritize the transitioning code. I want to provide alpha builds in the forum, so users can help testing the new code. (of course I'll look into #3224)

[glassez]

I want to provide alpha builds in the forum, so users can help testing the new code.

In that case, I'll do it now. But first we need to finish with #3224, because they have overlapping code.

[sledgehammer999]

@glassez 2 regressions:

1. https://qbforums.shiki.hu/index.php/topic,3606.msg18026.html#msg18026
2. https://qbforums.shiki.hu/index.php/topic,3606.msg18028.html#msg18028

[glassez]

@sledgehammer999 I honestly have never used the RSS feature. Please give me some instructions how I can use this so I could run some tests.

[Matth7878]

@glassez I'm the one who did report this regression. To reproduce :

1. display rss tab with menu view->rss reader
2. go to rss tab, click on new subscription button, enter an rss url (like https://kat.cr/tv/?rss=1 ) click on added rss feeds and you should see all torrents in right pane
3. double click one => with version 3.2.0 it displays dialog to add torrent (if you did select in option to "display torrent content and some options" when adding torrent). Alpha 3.3.0 do not respect this setting and add torrent without displaying dialog

[sledgehammer999]

Please give me some instructions how I can use this so I could run some tests.

@glassez I think the problem is here. IIRC you had left some old code there and I reminded you to update it in the same way you did for torrent addition in the rest of the code. Apparently both of us missed the obvious mistake. IMO it should be:

if (Preferences::instance()->useAdditionDialog())
    AddNewTorrentDialog::show(torrentLink, this);
else
    BitTorrent::Session::instance()->addTorrent(torrentLink);

[sledgehammer999]

PS: I am not sure about the this pointer.

[glassez]

@sledgehammer999 I think you right here. I'll test it tomorrow.

[ngosang]

@glassez One more bug. (I don't know if it was already in v3.2.0) 1) Select one torrent. 2) Go to the Trackers tab. 3) Delete one tracker (or more). 4) Restart qBittorrent. 5) The tracker is there again...

[glassez]

@ngosang Please try #3290.

[ngosang]

@glassez Must be a bug in the trackers list. This happened when I added a magnet but I can't replicate. Sometimes the info displayed in the tracker list is incorrect (all trackers display 0 peers). If I change the selected torrent or close/open the lower panel the error persists. In the transfer list (in the Seeds Peers columns) the displayed value is correct. This torrent has a lot of Seeds/Peers but in the tracker list all trackers report 0 peers (even dht).

[zeule]

What about droping Qt4 support?

[glassez]

What about droping Qt4 support?

If we were talking only about Windows, we could do this for a long time. But as I understand it, there are some problems with Qt5 support in Linux.

[zeule]

But as I understand it, there are some problems with Qt5 support in Linux.

I'm not aware of them. There might be a lack of Qt5 for a NAS systems. But you hardly find a desktop distribution without Qt5.

[glassez]

@evsh you should ask somebody other. I don't know much about it. I remember we discussed it and there were problems in this.

[zeule]

@all: What were the problems with Qt5 and Linux? In this thread I found only the style problem, which is resolved long time ago.

[ngosang]

I use Linux + KDE5 + Qt5 without any problem.

[glassez]

However, @sledgehammer999 rejected it. Maybe we should ask him?