All subscriptions in RSS failed to fetch feed after update to v4.4.2

wind77 commented 2 years ago

qBittorrent & operating system versions

qBittorrent: 4.4.2 x64 Operating system: Manjaro kernel 5.17 Qt: 6.3.0 libtorrent-rasterbar: 2.0.6.0

What is the problem?

Just updated from v4.4.2 to v4.4.2-2, and after reboot OS, all existing RSS subscriptions failed to fetch feed.

There was no internet connection issue, as shown here:

Quit and restart qBT does not seem to resolve the issue.
Manual "update all" also failed.
Remove RSS subscriptions and add them back, also failed.

Steps to reproduce

No response

Additional context

No response

Log(s) & preferences file(s)

No response

glassez commented 2 years ago

Log(s) & preferences file(s)

No response

So what does the log say?

glassez commented 2 years ago

Just updated from v4.4.1 to v4.4.2, and after reboot OS, all existing RSS subscriptions failed to fetch feed.

With what is Qt version qBittorrent v4.4.1 was built in your distro? What are your feeds? What encoding do they feeds use?

wind77 commented 2 years ago

(N) 2022-05-15T18:56:42 - RSS feed at '1111' is successfully downloaded. Starting to parse it. (W) 2022-05-15T18:56:42 - Failed to parse RSS feed at '1111'. Reason: Premature end of document. (line: 1, column: 0, offset: 0). (N) 2022-05-15T18:56:42 - RSS feed at '1111' updated. Added 0 new articles. (N) 2022-05-15T18:56:43 - RSS feed at '2222' is successfully downloaded. Starting to parse it. (W) 2022-05-15T18:56:43 - Failed to parse RSS feed at '2222'. Reason: Premature end of document. (line: 1, column: 0, offset: 0). (N) 2022-05-15T18:56:43 - RSS feed at '2222' updated. Added 0 new articles. (N) 2022-05-15T18:56:43 - RSS feed at '3333' is successfully downloaded. Starting to parse it. (W) 2022-05-15T18:56:43 - Failed to parse RSS feed at '3333'. Reason: Premature end of document. (line: 1, column: 0, offset: 0). (N) 2022-05-15T18:56:43 - RSS feed at '3333' updated. Added 0 new articles. (N) 2022-05-15T18:56:44 - RSS feed at '4444' is successfully downloaded. Starting to parse it. (W) 2022-05-15T18:56:44 - Failed to parse RSS feed at '4444'. Reason: Premature end of document. (line: 1, column: 0, offset: 0). (N) 2022-05-15T18:56:44 - RSS feed at '4444' updated. Added 0 new articles. (N) 2022-05-15T18:56:45 - RSS feed at '5555' is successfully downloaded. Starting to parse it. (W) 2022-05-15T18:56:45 - Failed to parse RSS feed at '5555'. Reason: Premature end of document. (line: 1, column: 0, offset: 0). (N) 2022-05-15T18:56:45 - RSS feed at '5555' updated. Added 0 new articles.

I replaced the actual subscriptions with 1111, 2222, 3333... The whole day, the above just kept repeating itself.

wind77 commented 2 years ago

Just updated from v4.4.1 to v4.4.2, and after reboot OS, all existing RSS subscriptions failed to fetch feed.

With what is Qt version qBittorrent v4.4.1 was built in your distro? What are your feeds? What encoding do they feeds use?

My feeds are in UTF-8 encoding. Just to clarify: before update, there was no issue with feed fetching. After update, the issue happened. I did NOT made any change to the Preference nor feed, BEFORE and AFTER the update.

glassez commented 2 years ago

I asked about qBittorrent v4.4.1.

glassez commented 2 years ago

Just to clarify: before update, there was no issue with feed fetching. After update, the issue happened. I did NOT made any change to the Preference nor feed, BEFORE and AFTER the update.

AFAIK, nothing related to parsing RSS feeds was changed in qBittorrent. One thing that can affect is different Qt version since it provides basic XML parsing.

wind77 commented 2 years ago

I asked about qBittorrent v4.4.1.

No clue about v4.4.1. I searched my previous issue, and as per #16913, I was already using qBT v4.4.2 (though the Qt then was 6.2.4) and there was no RSS problem at that time. This RSS problem only surfaced last night, after I updated my Manjaro - the qBT was also updated to v4.4.2-2 in Manjaro's repo. Since the Qt is suspected, is there a way for me to get around this issue, or to verify this issue?

Chocobo1 commented 2 years ago

Looks like duplicate of #16879.

ghost commented 2 years ago

@Chocobo1 I have been having a similar issue on macOS. I built qBittorrent with the master branch 4.5.0alpha1 which combined with Qt 6 is supposed to contain the fix/patch for this - I think #16879 , but it still exhibits this issue for me

@wind77 do you use Jackett as your indexer by any chance?

My error is always the same. I just wish someone could explain what "document" it refers to.

I have removed all of the RSS feeds. I then re-add them and the first time they come in perfectly... no errors. It's on the second fetch that the icon turns red and I get the error. It's as if they are not being added to the RSS JSON file properly.

7/16/22 9:04 PM - Failed to parse RSS feed at 'http://0.0.0.0:9117/api/v2.0/indexers/rarbg/results/torznab/api?apikey=redacted&t=search&cat=2040&q='. Reason: Premature end of document. (line: 2, column: 1259, offset: 1298).

I have been told by the Jackett devs that this is an RARBG issue and they closed my bug report as fast as I could open it. They say to set Jackett to make more requests (which I have done) but that doesn't jive with what I am seeing. Why does it work the first time every time, but then fail on the second, third, etc..

Qt: 6.3.1 Libtorrent: 2.0.6.0 Boost: 1.79.0 OpenSSL: 1.1.1q zlib: 1.2.11

glassez commented 2 years ago

I have been having a similar issue on macOS

The issue isn't really the same. Otherwise, it would not have been successfully parsed even once, but you say that everything is OK the first time. In addition, the error message refers to another place in the document.

I just wish someone could explain what "document" it refers to.

It refers to XML document that is essentially the RSS data.

Why does it work the first time every time, but then fail on the second, third, etc..

It is difficult to answer this question without comparing the data returned by the server for the first and subsequent times. Could you test it using web browser, curl etc. to see the raw data?

ghost commented 2 years ago

It is difficult to answer this question without comparing the data returned by the server for the first and subsequent times. Could you test it using web browser, curl etc. to see the raw data?

I tested it with curl and the raw data comes in fine every time. I don't know how I would replicate adding the new RSS data to the existing data using curl, and checking for duplicates. Isn't that the function of qBT's RSS feature that's in question here?

It seems this is an issue with how qBT is processing the feed's data when adding it to the original data request.

To troubleshoot I did build two versions of qBT using the master branch... one with Qt 5 and one with Qt 6. All the other packages in both builds being the same version. I haven't done extensive testing, but it appears from running the version with Qt 5 that my RSS issue has disappeared. The execution log shows no RSS errors.... yet and I gave clicked the "Update all" button 5 times. Jackett's cache feature has been turned off for both versions to remove the potential for a corrupted cache.

EDIT: Using the Qt 5 build I didn't get one feed error clicking "Update all" but the error for one feed did appear again when the feed refreshed automatically after the 15 minute refresh interval! What does that suggest? It seems it's always the second listed feed (or the one at the bottom of the list of feeds)

7/17/22 10:16 AM - Failed to parse RSS feed at 'http://0.0.0.0:9117/api/v2.0/indexers/rarbg/results/torznab/api?apikey=redacted&t=search&cat=2040&q='. Reason: Premature end of document. (line: 2, column: 1259, offset: 1298).

EDIT: One observation. The red failure icon shows and the error saying that it couldn't retrieve, but It looks like the feed did update and add new items to the list.

glassez commented 2 years ago

I tested it with curl and the raw data comes in fine every time. I don't know how I would replicate adding the new RSS data to the existing data using curl, and checking for duplicates.

You don't need to do it. All you could do is get this data several times (for example, at the same interval that qBittorrent uses for updating RSS) and publish it here for analysis of what is the difference between first data and subsequent ones.

how qBT is processing the feed's data when adding it to the original data request.

Sounds confusing...

It seems it's always the second listed feed (or the one at the bottom of the list of feeds)

Hmm... An interesting pattern (if it really is a pattern). How many feeds do you have in qBittorrent RSS? Are they all from the Jackett?

ghost commented 2 years ago

I meant to say how is qBT amending the local RSS file ( I thought it was a JSON) with new articles from the feed? Could that be the issue since it only happens when it's joining the first and second pull from the RSS feed

I have two feeds both from Jackett

http://0.0.0.0:9117/api/v2.0/indexers/tag:rarbg/results/torznab/api?apikey=redacted&t=search&cat=5040&q=

http://0.0.0.0:9117/api/v2.0/indexers/rarbg/results/torznab/api?apikey=redacted&t=search&cat=2040&q=

I am not home now but I will post the data from two curl requests tonight.

glassez commented 2 years ago

I meant to say how is qBT amending the local RSS file ( I thought it was a JSON) with new articles from the feed? Could that be the issue since it only happens when it's joining the first and second pull from the RSS feed

The reported error message is related to RSS/XML parsing step so the issue happens before the joining with previously existing data.

glassez commented 2 years ago

I have two feeds both from Jackett

As I can see they have different cat fields. But also one of them has just rarbg in the path and the second one tag:rarbg. What does this difference mean?

ghost commented 2 years ago

Jackett has a feature whereby you can create "tags" which you can use for multiple indexers in your list of Jackett indexers. So any indexer that has that associated tag will be used in that consolidated feed. Using just "rarbg" in the URL gets the feed data for just that indexer. I only added the "rarbg" tag to the rarbg indexer to test if that resolved the issue.

I have tried both methods to test if there was a difference or if one method is as part of the issue. No matter which, I get errors.

glassez commented 2 years ago

@jprokos I think I know the answer. I remembered that I had already encountered this. The Jackett can return a result containing an error message that is not a valid RSS document. Thus you get the parsing error.

ghost commented 2 years ago

This is potentially good news... I tested again today. Removed the feed that was repeatedly erring. Added it again and the first run there was no error, every other run was an error parsing. I tested the same feed using curl immediately after the failures and it pulls just fine. I just don't know if the issue is from qBT or Jackett and which forum to post in.

I have some non-Jackett RSS feeds as well and they never error. But they are not using the RARBG API URL via Jackett, they are using this URL: https://rarbg.to/rssdd.php?categories=54

I am attaching output from the curl command with a successful pull and a failure. I am assuming the same failure is also occurring inside qBT. I am just wondering why after it fails once it fails every time thereafter, whereas with curl that is not the case. curl.txt

glassez commented 2 years ago

I am attaching output from the curl command with a successful pull and a failure. I am assuming the same failure is also occurring inside qBT.

So the problem is exactly the one I described above: Jackett send some data that isn't valid RSS and RSS parser fails on it.

I am just wondering why after it fails once it fails every time thereafter, whereas with curl that is not the case. curl.txt

I don't think there is any additional problem in qBittorrent. It simply sends a request to the specified address, and then processes the received data. So if it reports the same error every time, then most likely it indicates the same type of incorrect data received from the Jackett.

ghost commented 2 years ago

What are the odds that a curl request will run without error four out of five times but making the same request in qBT will show as an "error" every single time after the first time?

ghost commented 2 years ago

I believe I found my issue ( not necessarily the OP's issue), or I am at least on to something. Should I open a new issue?

Sometimes the RSS rule is automatically renamed to the indexer name after clicking "OK" when adding it, other times it has not been renamed (keeping the actual string of the rule as the name). I have always renamed it after its first run if it wasn't renamed - because it's easier to choose in the RSS downloader which rule to use.

I ran a test without renaming the subscription rule and I have not seen this issue since. I'm fairly certain it's the process of renaming the feed that is causing this issue!

Is there a cache of the subscription names and their commands in qBT that may need to be cleared out before adding a new rule - if the rules were different but renamed the same that could potentially cause a conflicts?

Originally posted by @jprokos in https://github.com/Jackett/Jackett/issues/13363#issuecomment-1187756395