Closed bdurrow closed 4 years ago
You can send me the debug infos zip to theotherp@gmx.de
What would you expect to be removed / anonymized?
Ideally all IP addresses, hostnames and information indicating what I am automating. I realize that this will redact much of the useful information. I'll send you the debug infos zip - I would rather not make it available as part of the indelible permanent record.
I'm pretty sure this test is failing: If you look carefully at the <rss... node you will see that there is an extra space where the torznab namespace definition was indicating that it was mistakenly removed. I think that all of the various tests you do for searchType would be more robust as enums instead of string equality matches. As another example, text searches like this potentially introduce vulnerabilities (attacker can introduce the string "torznab" anywhere in the url to trigger those some parts of the logic while others execute as if it is a different kind of query - exposing more edge cases).
Thank you for this incredible product. I hope you don't feel like I am bashing on it and your hard work.
That whole piece of code is incredibly hacky, as you can see. It took me ages to get it somehow working. I'm pretty sure there are better ways than replacing strings in the constructed XML :-D
No worries, I rarely get the debugging analysis along with the bug report.
Works fine for me. The XML should have a searchType
tag at the top, what does it say?
From 127.0.0.1:5076/torznab/api?t=tvsearch&cat=5000,5030,5040&extended=1&apikey=apikey&offset=0&limit=1
I get
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:torznab="http://torznab.com/schemas/2015/feed" version="2.0">
<searchType>torznab</searchType>
<channel>
<title>NZBHydra 2</title>
<link>https://www.github.com/theotherp/nzbhydra2</link>
<webMaster>theotherp@gmx.de</webMaster>
<generator>NZBHydra2</generator>
</channel>
</rss>
Mine does not:
➜ ~ curl -v 'http://<<REDACTED, not localhost>>:5076/torznab/api?t=tvsearch&cat=5030,5040&extended=1&apikey=<<REDACTED>>&offset=0&limit=1'
* Trying 192.168.55.29...
* TCP_NODELAY set
* Connected to <<REDACTED, not localhost>> (<<REDACTED, not localhost>>) port 5076 (#0)
> GET /torznab/api?t=tvsearch&cat=5030,5040&extended=1&apikey=<<REDACTED>>&offset=0&limit=1 HTTP/1.1
> Host: <<REDACTED, not localhost>>:5076
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200
< Set-Cookie: HYDRA-XSRF-TOKEN=<<REDACTED>>; Path=/
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< Content-Type: application/xml;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Wed, 13 Nov 2019 22:20:29 GMT
<
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/" version="2.0">
<channel>
<title>NZBHydra 2</title>
<link>https://www.github.com/theotherp/nzbhydra2</link>
<webMaster>theotherp@gmx.de</webMaster>
<generator>NZBHydra2</generator>
<item>
<title>Masterchef The Professionals S12E05 480p x264-mSD</title>
<link>http://<<REDACTED, not localhost>>:5076/gettorrent/api/-2569130523405219662?apikey=<<REDACTED>></link>
<enclosure url="http://<<REDACTED, not localhost>>:5076/gettorrent/api/-2569130523405219662?apikey=<<REDACTED>>" length="260350928" type="application/x-bittorrent"/>
<pubDate>Wed, 13 Nov 2019 22:18:45 +0000</pubDate>
<guid isPermaLink="false">-2569130523405219662</guid>
<description></description>
<comments>https://eztv.io/ep/1388698/masterchef-the-professionals-s12e05-480p-x264-msd/</comments>
<category>TV</category>
<size>260350928</size>
<torznab:attr name="category" value="100001"/>
<torznab:attr name="downloadvolumefactor" value="0"/>
<torznab:attr name="guid" value="-2569130523405219662"/>
<torznab:attr name="magneturl" value="magnet:?xt=urn:btih:6d2b36b556143f2f29ee3b401b5109d09ba4c30a&dn=Masterchef.The.Professionals.S12E05.480p.x264-mSD[eztv].mkv[eztv]&tr=udp:%2F%2Ftracker.coppersurfer.tk:80&tr=udp:%2F%2Fglotorrents.pw:6969%2Fannounce&tr=udp:%2F%2Ftracker.opentrackr.org:1337%2Fannounce&tr=udp:%2F%2Fexodus.desync.com:6969"/>
<torznab:attr name="minimumratio" value="1"/>
<torznab:attr name="minimumseedtime" value="172800"/>
<torznab:attr name="peers" value="0"/>
<torznab:attr name="seeders" value="0"/>
<torznab:attr name="size" value="260350928"/>
<torznab:attr name="uploadvolumefactor" value="1"/>
<torznab:attr name="hydraIndexerScore" value="70"/>
<torznab:attr name="hydraIndexerHost" value="<<REDACTED, not localhost>>"/>
<torznab:attr name="hydraIndexerName" value="EZTV"/>
</item>
</channel>
</rss>
* Connection #0 to host <<REDACTED, not localhost>> left intact
* Closing connection 0
Hm, strange. Not saying it's your fault, but considering I haven't changed any of that code in ages and nobody else is reporting this issue, there must be something different with your setup. Do you have a reverse proxy inbetween?
I'll add debug logging to the related code and publish a new version later today. Perhaps that will provide some insight.
No proxy between them. Thanks for releasing a version to help debug this. I am working on getting the code to compile on my machine but I haven’t been able to yet. Downgrading to java 8 to see if that helps.
2.8.3 has two lines of logging that will log what output type is used and why.
hmmm, Search type: null
2019-11-14 12:39:15.867 DEBUG --- [0.0-5076-exec-2] org.nzbhydra.api.ExternalApi : [Host: <<REDACTED>>] Determined to be torznab call from URL "/torznab/api"
2019-11-14 12:39:15.867 INFO --- [0.0-5076-exec-2] org.nzbhydra.api.ExternalApi : [Host: <<REDACTED>>] Received external torznab API call: NewznabParameters{t=TVSEARCH, cat=[5030, 5040], offset=0, limit=1, raw=false, o=XML, attrs=[], extended=true}
2019-11-14 12:39:15.868 DEBUG --- [0.0-5076-exec-2] org.nzbhydra.searching.CategoryProvider : [Host:<<REDACTED>>] The supplied categories 5040,5030 match multiple configured categories
2019-11-14 12:39:15.868 DEBUG --- [0.0-5076-exec-2] org.nzbhydra.searching.CategoryProvider : [Host: <<REDACTED>>] The supplied categories 5040,5030 match the configured main category TV and will be assigned to that
2019-11-14 12:39:15.869 INFO --- [0.0-5076-exec-2] org.nzbhydra.api.ExternalApi : [Search: 600981, Host: <<REDACTED>>] Executing new search
2019-11-14 12:39:15.869 DEBUG --- [0.0-5076-exec-2] org.nzbhydra.api.ExternalApi : [Search: 600981, Host: <<REDACTED>>] Determined to be torznab call from URL "/torznab/api"
...
2019-11-14 12:39:17.711 INFO --- [0.0-5076-exec-2] org.nzbhydra.searching.Searcher : [Search: 600981, Host: <<REDACTED>>] Returning results 1-1 from 210 results in cache. A total of 238 results is available from indexers of which 0 were already rejected and 28 were removed as duplicates
2019-11-14 12:39:17.712 INFO --- [0.0-5076-exec-2] org.nzbhydra.api.ExternalApi : [Search: 600981, Host: <<REDACTED>>] Search took 1844ms. Returning 1 results
2019-11-14 12:39:17.713 INFO --- [0.0-5076-exec-2] org.nzbhydra.api.ExternalApi : [Search: 600981, Host: <<REDACTED>>] Putting search result into cache
2019-11-14 12:39:17.714 DEBUG --- [0.0-5076-exec-2] org.nzbhydra.web.WebConfiguration : [Search: 600981, Host: <<REDACTED>>] Search type: null
Uhm, are you using JSON? Are you caching your results? Looks like it: Putting search result into cache
. Yeah, found the spot.
That was an interesting one...
Thank you, that fixed it! I'm not sure it is related but I started getting logged authentication errors (I don't use authentication) after this fix went live on my system. Specifically when I try to "complete the config" for an indexer by doing a cap scan I get a toast saying "You are not allowed to visit that section." and logs look like this:
2019-11-14 23:43:02.627 WARN --- [0.0-5076-exec-5] o.n.auth.LoginAndAccessAttemptService : [Search: 775897, Host: <<REDACTED, my workstation IP>>] 9 failed access attempts from IP/host <<REDACTED, my workstation IP>> in the last 24 hours. Will block access after 5 failed attempts
I will open a ticket if it turns out to be something other than my mistake.
That shouldn't be related at all, I didn't change anything in that regard.
When I hit the following URL:
I get the following namespaces:
But the attributes nodes later in the rss look like this:
I believe this is because the namespace is incorrect, when I hit jackett directly I get the following name spaces:
To be totally clear I get:
When I expect:
I found this bug when I was trying to add nzbhydra2 has a torznab indexer to sonarr. Sonarr logs the following:
I run nzbhydra2 on FreeNAS in a FreeBSD jail. I installed from the FreeBSD pkg repository which installed nzbhydra2-2.6.2 then updated to 2.8.1 using the feature to do so in nzbhydra2. I would rather not post my debuginfos zip archive because I don't feel that it does enough to anonymize the results. Here is the relevant section of the nzbhydra.log: