Unable to pass found torrent via Jackett to Deluge

[kevinpawsey]

debug.zip To help with identifying and fixing issues, please include as much information as possible, including:

LazyLibrarian version number (at the bottom of config page)

Current Version: 143d500742aeb84ff94d2a02cfc558b8d9266d0a : Latest Version: 143d500742aeb84ff94d2a02cfc558b8d9266d0a

Operating system used (windows, mac, linux, NAS type)

Linux - Ubuntu - 17.10 fully updated

Interface in use (default, bookstrap)

Bookstrap

Which api (Goodreads, GoogleBooks, both)

both

Source of your LazyLibrarian installation (git, zip file, 3rd party package)

git

Relevant debug log with api keys and any passwords redacted

`2018-06-13 12:52:39	ERROR	SEARCHALLBOOKS	resultlist.py	downloadResult	282	Unhandled exception in downloadResult: Traceback (most recent call last): File "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/resultlist.py", line 260, in downloadResult controlValueDict["NZBurl"], newValueDict["AuxInfo"]) File "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/downloadmethods.py", line 310, in TORDownloadMethod hashid = CalcTorrentHash(torrent) File "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/downloadmethods.py", line 474, in CalcTorrentHash info = dict(decode(torrent))["info"] # python3 decode returns OrderedDict File "/home/plex/LazyLibrarian/LazyLibrarian/lib/bencode/init.py", line 169, in bdecode raise BencodeDecodeError debug.zip ("not a valid bencoded string") BencodeDecodeError: not a valid bencoded string
2018-06-13 12:52:39	INFO	SEARCHALLBOOKS	searchbook.py	search_book	298	Requesting AudioBook download: 93% http://192.168.1.24:9117/api/v2.0/indexers/audiobooktorrents/results/torznab: C S Lewis The Voyage of the Dawn Treader Narnia`

This is actually an extract of the log before debug mode, I am currently running a debug log to try and capture another 'find'

Please note - usually a single line of log is not sufficient. The lines just before the error occurs can give useful context and greatly assist with debugging.

Hi again, something else that I have just noticed that is happening is that when LL is finding a torrent via Jackett, it is having an issue with sending the torrent to Deluge.

I have attached the debug.zip file.

Let me know if there is any other information that you need 👍

Edit: Thought I would add... passing to SABnzb+ seems to work fine. Edit: another thing to add... history shows the item as marked as "Skipped", as follows: Title: J R R Tolkien The Two Towers Lord of the Rings Type: torznab AudioBook Date: 2018-06-13 15:02:06 Size: 689.28 Mb Provider: http://192.168.1.24:9117/api/v2.0/indexers/audiobooktorrents/results/torznab Downloader: None DownloadID: None URL: http://192.168.1.24:9117/dl/audiobooktorrents/?jackett_apikey=--redacted--&path=cjhZNnFtU21aT2NaV0h4aFBZZmEycmxZdmJyMGRYc1ROSnkyaHBXZE1CRW1YSVVUYXl2cWt0V3N0ODh4U1huRFFuSHBrc2xNSVVDZGhKV0g3QnRDQ3owYXFEckdXbnB4UVcrMWlOaW5IamFwQjlRM1FNZHpJeXR2a2FrNENUMTZNdWk4WkVXZTNhTExPd2hRNnR6N2FCU2xLN0JNUVg1S1AzQVZYalJEMnlNMWFJZXcrMmhvN3FxdE8xQldNMWMwTGs3aTErMUpHMHRhYjd3S3BSRzQvL2cwWldyTXEzOUZsWFBJVTBtcHFZRFpkNXJLUkcwZFVTY1lPMlBEeUNFdnlaVkRRTGt6aWdVOW9rQ2s5MVVTOTY5UFNJanloRjlJMVp2ZW01eng5ak8vNFZ1YVFWZ0RaL3ZiRitTTzNJZ3ZibEJhWVZ4dktFRFlWQll0ZUs3MUhPQUtGOXJYQzVTSlErQ0MzL2lwRE9rPQ2&file=J.+R.+R.+Tolkien+-+The+Two+Towers+-+Lord+of+the+Rings Error: None It doesn't seem to report it as an error... just in case this helps :)

[philborman]

Yes the problem is we are not calculating the "hash" of the torrent properly. nzb don't have a hash so no problems there. It's because of a workaround to suppress a warning in the code checker. All to do with python3 dictionaries being different to python2 ones. Would be better to accept the warning I think.

Are you up for a bit of simple editing to test my theory for me? If so... Keep an unmodified copy of downloadmethods.py so you can put it back later. You currently have "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/downloadmethods.py", line 474, in CalcTorrentHash info = dict(decode(torrent))["info"] # python3 decode returns OrderedDict

If you change line 474 to info = decode(torrent)["info"] ie remove the dict() I think it should fix the issue. If so I will push a proper fix later.

[kevinpawsey]

I am definitely up for that... I will do it now... one moment...

Update 1: file is edited, restarted LL and running another audiobook search... no errors as yet (but no new torrents either)

Update 2: I am a silly sausage... I forgot to clear all the 'skipped' from history... so not sure if it will re-attempt to download those ones. Although I have gone through now and removed all the Skipped I have just had another re-appear, but not sure if it failed in the same way... still no new torrent downloads

[kevinpawsey]

ok... it appears that I am getting the same issue... extract from debug log: 13-Jun-2018 18:47:11 - INFO :: SEARCHALLBOOKS : searchbook.py:search_book:298 : Requesting AudioBook download: 99% http://192.168.1.24:9117/api/v2.0/indexers/audiobooktorrents/results/torznab: J R R Tolkien Lord of the Rings Trilogy Lord of the Rings 13-Jun-2018 18:47:13 - ERROR :: SEARCHALLBOOKS : resultlist.py:downloadResult:282 : Unhandled exception in downloadResult: Traceback (most recent call last): File "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/resultlist.py", line 260, in downloadResult controlValueDict["NZBurl"], newValueDict["AuxInfo"]) File "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/downloadmethods.py", line 310, in TORDownloadMethod hashid = CalcTorrentHash(torrent) File "/home/plex/LazyLibrarian/LazyLibrarian/lazylibrarian/downloadmethods.py", line 474, in CalcTorrentHash info = decode(torrent)["info"] # python3 decode returns OrderedDict File "/home/plex/LazyLibrarian/LazyLibrarian/lib/bencode/__init__.py", line 169, in bdecode raise BencodeDecodeError("not a valid bencoded string") BencodeDecodeError: not a valid bencoded string

my download methods.py line 474 looks like this: ' info = decode(torrent)["info"] # python3 decode returns OrderedDict'

Let me know if there is anything else that you would like me to try 👍

[philborman]

I have attached a new version of downloadmethods.py, can you drop that in and try. It might fix the problem, or if not should give us more info on what's happening. Thanks downloadmethods.zip

[kevinpawsey]

Cool... dropped in the new version... restarted LL... and have cleared out the old Skipped files... will let you know what happens...

[kevinpawsey]

ok, so the search has been running and there have been several "Skipped" attempts in History now: Title: Sarah J Maas Heir of Fire Throne of Glass Type: torznab AudioBook Date: 2018-06-13 22:52:23 Size: 555.83 Mb Provider: http://192.168.1.24:9117/api/v2.0/indexers/audiobooktorrents/results/torznab Downloader: None DownloadID: None URL: http://192.168.1.24:9117/dl/audiobooktorrents/?jackett_apikey=<redacted>&path=RkQ5azd2VTFrdGxtelhJNUY4RzQyVzcyVzJCMU8wbFVzVDZiMytZSDF4K3BCN3l5aEVnMFc2c0xUNU5VTXkyTlBNWXRMSE0vb3c5cXduUklKQngzTCt3Wm43MC9qaElZUjJTd0pPRXdaMUdqVkhHNC9MYVdnTlpGZHQ1anhnQU0vRzBkK1F2VXF6TlljRTd6ZFA5VnFqb1R6dEdpK05BM1NmbmlpcmJGYjdrRTNFdVNmUUcxY3d5TksyWlQyY21QUlo0TVUxZmFsWURDdGZ6YTZKSUdXenN2bDVHUHdVdm4yRVBtellRQnk0MmtyRTdTc01Mbmh5aStZL2JLUDJ1dG1jOE5nUUVvK2FuTGt2SFFYbDdjWXhja1dYdDBWMEl3ODBGeVl4RWdXOWlyQ05UZ2FlUEptc0pUUGpneVBjYXc2K3BZNnpUS1lDR3BVWkx1eUJBUVVwMFVBdlZqSlFsR1ZsWmtMaFZNU2cwPQ2&file=Sarah+J.+Maas+-+Heir+of+Fire+-+Throne+of+Glass Error: None It looks like the new error capturing has done it's job... hopefully it will be of help, I have attached the debug log file.

Let me know if there is anything else that I can do 👍 debug.zip

[philborman]

So the new code shows we are getting an empty result back from the call to pull the torrent from jackett. Actually what jackett does is silently redirect us from the url it fed us to a magnet link, so the response is ok being empty, we just need to use the new magnet link.

Hopefully fixed in the attached file downloadmethods.zip

[kevinpawsey]

Exciting stuff... let me give it a try ;)

I am already looking at the next things for you to fix! lol

[kevinpawsey]

Looks like it hasn't fixed it unfortunately... still doing the same thing... I have attached the debug logs... debug.zip

[kevinpawsey]

I don't know if it is any help... but I thought I would look at the Jackett logs while the query was running... and I am seeing this in the Jackett logs:

2018-06-14 13:31:58.5823 Error Error downloading audiobooktorrents https://abtorrents.me/download.php?torrent=30248 BencodeNET.Exceptions.InvalidBencodeException1[BencodeNET.Objects.IBObject]: Failed to parse IBObject. Invalid beginning character of object. Found '<' at position 0. Valid characters are: 0-9, 'i', 'l' and 'd' at BencodeNET.Parsing.BencodeParser.Parse (BencodeNET.IO.BencodeStream stream) [0x0008f] in :0 at BencodeNET.Parsing.BencodeParser.Parse (System.IO.Stream stream) [0x00008] in :0 at BencodeNET.Parsing.BencodeParser.Parse (System.Byte[] bytes) [0x00007] in :0 at Jackett.Controllers.DownloadController+d__5.MoveNext () [0x001ab] in <0ed7ac9a545948678f8135aa615daab7>:0 `

Is that because of what is being asked to be done by LL?

[philborman]

Not lazylibrarian, but it does explain what's going on. Jackett is trying to download https://abtorrents.me/download.php?torrent=30248 to pass the torrent contents to us, but abtorrents sends a html page ( Found '<' at position 0 ) when jackett is expecting a torrent file. Could be a login page, could be an error page, but it's certainly not a torrent file or a magnet as they can't start with '<'
Jackett then gives lazylibrarian an empty response but with no error code, when really it should pass the error code back so we know what's happened. I can trap the empty response, but of course we won't know why we got it.
Does abtorrents work from within jackett, ie are your username/password correct and a search from the jackett webui works?

Attached file should trap the empty response downloadmethods.zip

[kevinpawsey]

Yes, the search works through Jackett, I have even tried removing and re-adding to Jackett, just to make sure as well.

I will try this latest version of the file and see what happens.

Thanks for your help :)

[kevinpawsey]

oh dear... I think I have found the issue... :/

When you said it might be an error page, I thought I would try manually going to the site and downloading the torrent to see if it works... I am getting an error saying that I don't have enough credit! :(

I am so sorry, I should have tried this before now!

Hopefully, if nothing else, it will go towards capturing this type of error and displaying it a bit different in the log files? or maybe feed it back into the blocked providers list (which is a great idea to have btw!).

I am just running the search again, and when it hits the issue again I will send you the debug logs again

[philborman]

It's a learning curve. I don't use any private trackers myself, so haven't explored this before. I would expect Jackett to at least report an error, but it seems it doesn't. The new code should add the provider to the blocklist though.

[kevinpawsey]

awesome... that would be great... thank you again for your help... I feel a little less of a plumb now I have actually been of some help! lol

... I think I can call this one closed ;)

[philborman]

Don't forget to put the original file back so the lazylibrarian update works :-)

[kevinpawsey]

... good point :)

[kevinpawsey]

Just thought I would let you know that the new fix in the latest version is not putting the offending torznab provider in the blocked list... and the attempts are being marked as skipped in history still :)

It is, however, catching the error properly, and not putting the error thrown as before in the log files: ` 2018-06-14 19:12:21	ERROR	Unable to calculate torrent hash from url/data
2018-06-14 19:12:21	ERROR	Cannot calculate torrent hash without magnet link or data

`

[philborman]

ok thanks. What are the log lines just before the ones you posted, looking for any or all of... Fetching xxx Got status xxx Redirected to xxx Got xxx response Got xxx bytes

Which lines were logged and what order would be v useful to know what path we took through the code.

[kevinpawsey]

Just happened to fail another one... looks like this with debug switched on...

` 2018-06-14 20:03:04	DEBUG	SEARCHALLBOOKS	providers.py	get_capabilities	194	Using stored capabilities for https://api.nzbplanet.net
2018-06-14 20:03:04	DEBUG	SEARCHALLBOOKS	downloadmethods.py	TORDownloadMethod	335	data:
2018-06-14 20:03:04	DEBUG	SEARCHALLBOOKS	downloadmethods.py	TORDownloadMethod	333	url: http://192.168.1.24:9117/dl/audiobooktorrents/?jackett_apikey==bHF2Qld5QVBqaXJGZWlCV3Nza21GRE1qeXdzS0Z3aWJkdTV4TmxCOUlRcUp2aVZMMFo3S1k0cCtDeW1wUEpqYXNCK2NlaWNYcm5HeHJNbW9ZcWl3OCtEUGVSOEFpeXFabVJLRWx5NFZKMFkyTWE5UVc5YTF4OVI3ZGl4bm4yNjVrZ0IrMFNWeWhBYlJ0NDgzcEovK0Z2aGIxMUNYc3p3UjhRbVBhTWt0NVFMc0hmeTdyb2wxZE9uVCsyUkRBSzFTSTV5VkZPU04xWUl5L3VpZlloSE8xei9mbWlXc0I4eEJRbUtGbTRlNk9QeFBwdDhLbm1iS3NpaW51UjNsOG0rY2pWUWtaR2x0OGdCVFFwSWl0SGxzMjdXa2dXYVhUZUlkT2VpaXd1TndlQkNPdjVuc0FacjRlYjFUY3h3clJKcjYxZmN0OTRXS3JYNEJ0OVZEY2xVZDZKanRRQndDV0huQTFzME1SaTZVZENrPQ2&file=J.+K.+Rowling+-+Complete+Harry+Potter+Series+-+Harry+Potter
2018-06-14 20:03:04	ERROR	SEARCHALLBOOKS	downloadmethods.py	TORDownloadMethod	332	Unable to calculate torrent hash from url/data
2018-06-14 20:03:04	ERROR	SEARCHALLBOOKS	downloadmethods.py	calculate_torrent_hash	495	Cannot calculate torrent hash without magnet link or data
2018-06-14 20:03:04	DEBUG	SEARCHALLBOOKS	downloadmethods.py	TORDownloadMethod	270	Got no response from requests
2018-06-14 20:03:04	DEBUG	SEARCHALLBOOKS	downloadmethods.py	TORDownloadMethod	237	Got status 404
2018-06-14 20:03:04	DEBUG	SEARCHALLBOOKS	downloadmethods.py	TORDownloadMethod	235	Fetching http://192.168.1.24:9117/dl/audiobooktorrents/?jackett_apikey==bHF2Qld5QVBqaXJGZWlCV3Nza21GRE1qeXdzS0Z3aWJkdTV4TmxCOUlRcUp2aVZMMFo3S1k0cCtDeW1wUEpqYXNCK2NlaWNYcm5HeHJNbW9ZcWl3OCtEUGVSOEFpeXFabVJLRWx5NFZKMFkyTWE5UVc5YTF4OVI3ZGl4bm4yNjVrZ0IrMFNWeWhBYlJ0NDgzcEovK0Z2aGIxMUNYc3p3UjhRbVBhTWt0NVFMc0hmeTdyb2wxZE9uVCsyUkRBSzFTSTV5VkZPU04xWUl5L3VpZlloSE8xei9mbWlXc0I4eEJRbUtGbTRlNk9QeFBwdDhLbm1iS3NpaW51UjNsOG0rY2pWUWtaR2x0OGdCVFFwSWl0SGxzMjdXa2dXYVhUZUlkT2VpaXd1TndlQkNPdjVuc0FacjRlYjFUY3h3clJKcjYxZmN0OTRXS3JYNEJ0OVZEY2xVZDZKanRRQndDV0huQTFzME1SaTZVZENrPQ2&file=J.+K.+Rowling+-+Complete+Harry+Potter+Series+-+Harry+Potter
2018-06-14 20:03:04	INFO	SEARCHALLBOOKS	searchbook.py	search_book	298	Requesting AudioBook download: 99% http://192.168.1.24:9117/api/v2.0/indexers/audiobooktorrents/results/torznab: J K Rowling Complete Harry Potter Series Harry Potter

`

Let me know if you need any other details

[philborman]

Ok that's excellent, I can trap the 404 error now I know what to look for. thanks for your help

[kevinpawsey]

no problem at all... thanks for the time that you spend on this project 🥇 🌟

[kevinpawsey]

Hi @philborman,

Just thought I would let you know that I am still seeing the '404' items ending up in the History as skipped. Just checked today, and I have about 5 pages of them :)

Is there a way of bulk-clearing the skipped items so that they will be tried again at a later date?

I don't know if it is of any use, but this is what Jackett is seeing when trying to download: `

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

• Upload
• Requests
  • Requests
  • Offers
• Torrent
  • Need Seeds
  • Bookmarks
• General
  • Site Announcements
  • Statistics
  • Rules and FAQ
  • Rules
  • Wiki
  • Torrent Freak
• Forums
• Games
  • Casino and Blackjack
• Contact Staff
  • Message Staff member
  • Message general Staff
• Bug Report
• Bitbucket

   in last 24 hours

 

Home

• Browse
• Search
• Messages
• Usercp
• Friends
• 
• Logout

kevinpawsey     

:: Personal Stats

User Class

(VIP)

Reputation

Invites

4

Bonus Points

15.3

Achievements

11

:: Torrent Stats

Share Ratio

Inf

Uploaded

10.00 GB

Downloaded

0.00 KB

Uploading Files

24

Downloading Files

0

Connectable

• Current Site Alerts:
• Karma ContributionsKarma Contribution'sFreeleech [ 0 % ]
  DoubleUpload [ 100 % ]
  Half Download [ 0 % ]

Error You don't have enough credit to download, trying seeding back some torrents =]

Audiobook Torrents page was generated in 0.00614 seconds.
Server was hit 1 time

Powered by U-232 V4
Using Valid CSS3, XHTML & PHP
Support Forum Click here

Jun 18, 2018 04:23:04 AM | Info | Found 1 releases from Audiobook Torrents for: Rick Riordan The Trials of Apollo . The Hidden Oracle `

If it helps, I think that maybe checking for the phrase "You don't have enough credit to download, trying seeding back some torrents" if you get passed this page might help... then maybe add to blacklist for 24 hours or something.

In the meantime I will disable abtorrents to stop getting quite so many errors! lol

Let me know if you want me to test anything.

[philborman]

They are left in the history table so we don't retry the same one, it's used as a blacklist. Should say "Failed" rather than "Skipped" though? Are you on current version of lazylibrarian, and are these new downloads, or left over from before you upgraded?

[kevinpawsey]

ah sorry, yes, I am on latest version (meant to state that) ... but they are definitely going in as Skipped. It would be great if they were going as failed, as there is a button for that! lol

[philborman]

ok, I need to look into where the "Skipped" comes from. For now you could use sqlite3 from the commandline? echo 'delete from wanted where status="Skipped";' | sqlite3 lazylibrarian.db

Don't know if we can check the contents of the page for "you don't have enough credit" I think all we get back is a 404 error, which is "not found"

[kevinpawsey]

perfect, the SQL query did the trick 👍

Ah yes, I remember now you saying about where you get 404 from... I think it is because Jackett reports it as an error and 'blows up' ... maybe I will see if I can mark that as a bug with them ;)

[philborman]

Found the reason for the "Skipped" entry, fix should be in next release. I think the "Skipped" was just a placeholder meaning "not sent to a downloader yet".

[kevinpawsey]

ah right... I will wait for the update and then enable the provider again to give it a test... will let you know when it runs again

UPDATE: It is now marking the 404s as failed in the History tab... now it should be just a case of hitting one button to reset it 👍

Thank you once again for your help.