wymangr
commented
2 years ago From what I can tell, this is the HTML that is stored in the data variable when is searches:
data = request.request_soup(url=providerurl + category, headers=headers)
And looking at that, it appears that maybe Pirate Bay has switched to rendering it's result table to javascript and the requests library is unable to get the HTML of the table?
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/>
<title>The Pirate Bay - The galaxy's most resilient bittorrent site</title>
<link href="https://torrindex.net/static/normalize.css" rel="stylesheet" type="text/css"/>
<link href="https://torrindex.net/static/tpb.css" rel="stylesheet" type="text/css"/>
<link href="//apibay.org/" rel="dns-prefetch"/>
<link href="//torrindex.net/" rel="dns-prefetch"/>
<link href="//ilyf4amifh.com/" rel="dns-prefetch"/>
<link href="//abservinean.com/" rel="dns-prefetch"/>
<link href="//a.exdynsrv.com/" rel="dns-prefetch"/>
<link href="//a.exosrv.com/" rel="dns-prefetch"/>
<link href="//cdn.engine.spotscenered.info/" rel="dns-prefetch"/>
<link href="//syndication.exdynsrv.com/" rel="dns-prefetch"/>
<link href="//d1f05vr3sjsuy7.cloudfront.net/" rel="dns-prefetch"/>
<script>var country='CH';</script>
<script src="https://thepiratebay.org/static/main.js"></script>
<script src="https://torrindex.net/static/tinysort.min.js"></script>
<script async="" data-cfasync="false" src="//ilyf4amifh.com/pn21ywqw/z/sc/scssx/1846387/lib.js" type="text/javascript"></script>
<script data-cfasync="false" src="//d1f05vr3sjsuy7.cloudfront.net/?srvfd=908284"></script>
</head>
<body id="browse" onload="jswarnclear()">
<main>
<b><font color="RED"><label id="jscrwarn">Enable JS in your browser!</label></font></b>
<script>document.getElementById("jscrwarn").innerHTML='';</script>
<b><font color="RED"><label id="jscrwarn2">You may be blocking important javascript components, check that main.js is loaded or the webpage won't work.</label></font></b>
<h1><label id="tlt"></label></h1>
<div class="adblock" id="ad-bottom">
<div class="ad728 align-center">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/728t.png"/></a>
</div>
<div class="ad468 align-center">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/468.png" style="text-decoration:none; border-bottom:none; color:none;"/></a>
</div>
<div class="ad234 align-center">
<script type="application/javascript">
var ad_idzone = "3804619",
ad_width = "300",
ad_height = "100"
</script>
<script src="https://a.exdynsrv.com/ads.js" type="application/javascript"></script>
</div>
</div>
<div class="browse">
<section class="col-left ad120">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/120.png"/></a>
</section>
<script>print_search()</script>
<section class="col-right ad120">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/160.png"/></a>
</section>
</div>
<div class="adblock" id="ad-bottom">
<div class="ad728 align-center">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/728f.png"/></a>
</div>
<div class="ad468 align-center">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/468.png" style="text-decoration:none; border-bottom:none; color:none;"/></a>
</div>
<div class="ad234 align-center">
<script type="application/javascript">
var ad_idzone = "3804621",
ad_width = "300",
ad_height = "250"
</script>
<script src="https://a.exdynsrv.com/ads.js" type="application/javascript"></script>
</div>
</div>
</main>
<header class="row">
<script>print_header1()</script>
<input id="search" name="q" placeholder="Search here..." title="Pirate Search" type="search"/>
<script>document.getElementById("search").value = getParameterByName('q')</script>
<script>print_header2()</script>
<section class="col-right ad468" id="had468">
<a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/468.png" style="text-decoration:none; border-bottom:none; color:none;"/></a>
</section>
<section class="col-right ad234" id="had234"></section>
</header>
<script>
print_footer();
mark_selected();
</script>
<script>do_pop()</script>
<script>do_interstitial()</script>
</body>
</html>
I have been using headphones for a while now and it has worked in the past and I've always used pirate bay as my search provider. However, it hasn't found any new music in quite a long time.
Headphones Version: 83398cb10231087399e284daf9245dbfedbfd005 Pirate Bay Proxy URL: Tried blank (Default) and https://thepiratebay.org
Logs (Using RHCP as example): 26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Marking album: 0580e402-e6e8-4b5b-894a-399a10c63adb as wanted... 26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Searching for wanted albums 26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Searching for "Red Hot Chili Peppers - Unlimited Love" since it was marked as wanted 26-May-2022 09:29:43 - DEBUG :: ('CP Server Thread-11',) : Using search term: Red Hot Chili Peppers Unlimited Love 26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Searching The Pirate Bay using term: Red Hot Chili Peppers Unlimited Love 26-May-2022 09:29:43 - DEBUG :: ('CP Server Thread-11',) : Requesting URL via GET method: https://thepiratebay.org/search/Red Hot Chili Peppers Unlimited Love/0/7/101 26-May-2022 09:29:45 - INFO :: ('CP Server Thread-11',) : No results found from The Pirate Bay using term: Red Hot Chili Peppers Unlimited Love 26-May-2022 09:29:45 - INFO :: ('CP Server Thread-11',) : No more results found for: Red Hot Chili Peppers - Unlimited Love
Using the requesting URL, I see multiple results in the browser: