rembo10 / headphones

Automatic music downloader for SABnzbd
GNU General Public License v3.0
3.4k stars 600 forks source link

what.cd torrent order not respected #2640

Closed nopoz closed 8 years ago

nopoz commented 8 years ago

I'm trying to get the torrent with the most seeders downloaded using what.cd. I've managed to get the order correct as far as I can tell but the torrent that is ultimately downloaded is the wrong one. I can't tell how this particular one is being chosen.

modified searcher.py snippet

            # sort by times d/l'd
            if not len(match_torrents):
                logger.info(u"No results found from %s for %s after filtering" % (provider, term))
            elif len(match_torrents) > 1:
                logger.info(u"Found %d matching releases from %s for %s - %s after filtering" %
                            (len(match_torrents), provider, artistterm, albumterm))
                logger.info(
                    "Sorting torrents by times snatched and preferred bitrate %s..." % bitrate)
                match_torrents.sort(key=lambda x: int(x.snatched), reverse=True)
                logger.info(
                    u"New order: %s" % ", ".join(repr(torrent) for torrent in match_torrents))
                logger.info('Sorting torrents by number of seeders...')
                match_torrents.sort(key=lambda x: int(x.seeders), reverse=True)
                if gazelleformat.MP3 in search_formats:
                    logger.info('Sorting torrents by seeders...')
                    match_torrents.sort(key=lambda x: int(x.seeders))
                if search_formats and None not in search_formats:
                    match_torrents.sort(
                        key=lambda x: int(search_formats.index(x.format)))  # prefer lossless
                #                if bitrate:
                #                    match_torrents.sort(key=lambda x: re.match("mp3", x.getTorrentDetails(), flags=re.I), reverse=True)
                #                    match_torrents.sort(key=lambda x: str(bitrate) in x.getTorrentFolderName(), reverse=True)
                logger.info(
                    u"New order: %s" % ", ".join(repr(torrent) for torrent in match_torrents))

                # test output to show order...
                for torrent in match_torrents:
                    logger.info(
                    u"Test output: %s" % torrent)

            for torrent in match_torrents:
                if not torrent.file_path:
                    torrent.group.update_group_data()  # will load the file_path for the individual torrents
                resultlist.append((torrent.file_path,
                                   torrent.size,
                                   gazelle.generate_torrent_link(torrent.id),
                                   provider,
                                   'torrent', True))

headphones.log showing the order is correct - I want id 1748894 which is listed first but instead it grabs 30613366. I've tried reversing the order and it doesn't change anything.

05-May-2016 03:14:56 - INFO    :: CP Server Thread-2 : Marking 7818bb84-ed80-3488-9f50-f9bf4347e372 as Wanted
05-May-2016 03:14:56 - INFO    :: CP Server Thread-2 : Searching for wanted albums
05-May-2016 03:14:56 - INFO    :: CP Server Thread-2 : Searching for "Queens of the Stone Age - R" since it was marked as wanted
05-May-2016 03:14:56 - INFO    :: CP Server Thread-2 : Your preferred bitrate matches V0 encoding type.
05-May-2016 03:14:56 - INFO    :: CP Server Thread-2 : Attempting to log in to What.cd...
05-May-2016 03:14:56 - INFO    :: CP Server Thread-2 : Searching What.cd...
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Filtering torrents by format, maximum size, and minimum seeders...
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Remaining torrents: Torrent: Rated R - V0 (VBR) - ID: 1748894, Torrent: Rated R - V0 (VBR) - ID: 29161735, Torrent: Rated R - V0 (VBR) - ID: 30613366, Torrent: Rated R - V0 (VBR) - ID: 118769, Torrent: Rated R - V0 (VBR) - ID: 1736911
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Found 5 matching releases from What.cd for Queens of the Stone Age - R after filtering
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Sorting torrents by times snatched and preferred bitrate ['V0 (VBR)']...
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : New order: Torrent: Rated R - V0 (VBR) - ID: 1748894, Torrent: Rated R - V0 (VBR) - ID: 118769, Torrent: Rated R - V0 (VBR) - ID: 1736911, Torrent: Rated R - V0 (VBR) - ID: 29161735, Torrent: Rated R - V0 (VBR) - ID: 30613366
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Sorting torrents by number of seeders...
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : New order: Torrent: Rated R - V0 (VBR) - ID: 1748894, Torrent: Rated R - V0 (VBR) - ID: 1736911, Torrent: Rated R - V0 (VBR) - ID: 118769, Torrent: Rated R - V0 (VBR) - ID: 29161735, Torrent: Rated R - V0 (VBR) - ID: 30613366
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Test output: Torrent: Rated R - V0 (VBR) - ID: 1748894
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Test output: Torrent: Rated R - V0 (VBR) - ID: 1736911
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Test output: Torrent: Rated R - V0 (VBR) - ID: 118769
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Test output: Torrent: Rated R - V0 (VBR) - ID: 29161735
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Test output: Torrent: Rated R - V0 (VBR) - ID: 30613366
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Making sure we can download the best result
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Found best result from What.cd: <a href="https://what.cd/torrents.php?action=download&id=30613366&authkey=xxx&torrent_pass=xxx">Queens Of The Stone Age - Rated R - 2000 (UK CD MP3 V0)</a> - 87.0 MB
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Sending torrent to Transmission
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Torrent sent to Transmission successfully
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Torrent folder name: Queens Of The Stone Age - Rated R - 2000 (UK CD MP3 V0)
05-May-2016 03:14:57 - WARNING :: CP Server Thread-2 : Could not get seed ratio for What.cd
05-May-2016 03:14:57 - INFO    :: CP Server Thread-2 : Search for wanted albums complete
05-May-2016 03:19:21 - INFO    :: Thread-12 : Found "Queens Of The Stone Age - Rated R - 2000 (UK CD MP3 V0)" in torrent download folder. Verifying....

maybe there is some kind of size consideration, although I can't tell if that's the case. I don't have any size preferences listed in the config.

config.ini

[General]
config_version = 5
http_port = 8181
http_host = 0.0.0.0
http_username = ""
http_password = ""
http_root = ""
http_proxy = 0
enable_https = 0
https_cert = xxx
https_key = xxx
launch_browser = 0
api_enabled = 1
api_key = xxx
log_dir = xxx
cache_dir = xxx
git_path = xxx
git_user = rembo10
git_branch = master
do_not_override_git_branch = 0
check_github = 1
check_github_on_startup = 1
check_github_interval = 360
music_dir = xxx
destination_dir = xxx
lossless_destination_dir = xxx
preferred_quality = 2
preferred_bitrate = 226
preferred_bitrate_high_buffer = 0
preferred_bitrate_low_buffer = 0
preferred_bitrate_allow_lossless = 0
detect_bitrate = 0
lossless_bitrate_from = 0
lossless_bitrate_to = 0
auto_add_artists = 0
correct_metadata = 0
freeze_db = 0
move_files = 0
rename_files = 0
folder_format = $Artist - $Year - $Album
file_format = $Track - $Title
file_underscores = 0
cleanup_files = 0
keep_nfo = 0
add_album_art = 0
album_art_format = folder
embed_album_art = 0
embed_lyrics = 0
replace_existing_folders = 0
nzb_downloader = 0
torrent_downloader = 1
download_dir = xxx
blackhole_dir = xxx
usenet_retention = 1620
include_extras = 0
extras = ""
autowant_upcoming = 1
autowant_all = 0
autowant_manually_added = 1
keep_torrent_files = 1
prefer_torrents = 1
open_magnet_links = 0
numberofseeders = 1
torrentblackhole_dir = xxx
download_torrent_dir = xxx
search_interval = 360
libraryscan = 1
libraryscan_interval = 300
download_scan_interval = 5
update_db_interval = 24
mb_ignore_age = 365
torrent_removal_interval = 720
preferred_words = ""
ignored_words = ""
required_words = ""
lastfm_username = ""
interface = default
folder_permissions = 0755
file_permissions = 0644
music_encoder = 0
encoder = lame
xldprofile = ""
bitrate = 192
samplingfrequency = 44100
encoder_path = ""
advancedencoder = ""
encoderoutputformat = mp3
encoderquality = 2
encodervbrcbr = vbr
encoderlossless = 1
encoder_multicore = 0
encoder_multicore_count = 0
delete_lossless_files = 1
mirror = musicbrainz.org
customhost = tbueter.com
customport = 3000
customsleep = 1
hpuser = ""
hppass = ""
cue_split = 0
blackhole = 0
headphones_indexer = 0
magnet_links = 0
encoderfolder = ""
cue_split_flac_path = ""
cue_split_shntool_path = ""
custompass = ""
customuser = ""
customauth = 0
ignore_clean_releases = 0
wait_until_release_date = 0
official_releases_only = 0
keep_original_folder = 1
file_permissions_enabled = 1
soft_chroot = ""
do_not_process_unmatched = 0
rename_unprocessed = 1
rename_frozen = 1
folder_permissions_enabled = 1
[Kat]
kat = 0
kat_proxy_url = ""
kat_ratio = ""
[Mininova]
mininova = 0
mininova_ratio = ""
[Piratebay]
piratebay = 0
piratebay_proxy_url = ""
piratebay_ratio = ""
[Waffles]
waffles = 0
waffles_uid = xxx
waffles_passkey = xxx
waffles_ratio = ""
[Rutracker]
rutracker = 0
rutracker_user = ""
rutracker_password = ""
rutracker_ratio = ""
[What.cd]
whatcd = 1
whatcd_username = xxx
whatcd_password = xxx
whatcd_ratio = ""
[SABnzbd]
sab_host = xxx
sab_username = xxx
sab_password = xxx
sab_apikey = xxx
sab_category = music
[NZBget]
nzbget_username = nzbget
nzbget_password = ""
nzbget_category = ""
nzbget_host = ""
nzbget_priority = 0
[Headphones]
headphones_indexer = 0
[Transmission]
transmission_host = xxx
transmission_username = xxx
transmission_password = xxx
[uTorrent]
utorrent_host = ""
utorrent_username = ""
utorrent_password = ""
utorrent_label = ""
[Newznab]
newznab = 0
newznab_host = xxx
newznab_apikey = xxx
newznab_enabled = 1
extra_newznabs = ,
[NZBsorg]
nzbsorg = 0
nzbsorg_uid = None
nzbsorg_hash = ""
[omgwtfnzbs]
omgwtfnzbs = 0
omgwtfnzbs_uid = ""
omgwtfnzbs_apikey = ""
[Growl]
growl_enabled = 0
growl_host = ""
growl_password = ""
growl_onsnatch = 0
[Prowl]
prowl_enabled = 0
prowl_keys = ""
prowl_onsnatch = 0
prowl_priority = 0
[XBMC]
xbmc_enabled = 0
xbmc_host = ""
xbmc_username = ""
xbmc_password = ""
xbmc_update = 0
xbmc_notify = 0
[LMS]
lms_enabled = 0
lms_host = ""
[Plex]
plex_enabled = 0
plex_server_host = ""
plex_client_host = ""
plex_username = ""
plex_password = ""
plex_update = 0
plex_notify = 0
plex_token = ""
[NMA]
nma_enabled = 0
nma_apikey = xxx
nma_priority = 0
nma_onsnatch = 0
[Pushalot]
pushalot_enabled = 0
pushalot_apikey = ""
pushalot_onsnatch = 0
[Pushover]
pushover_enabled = 0
pushover_keys = xxx
pushover_onsnatch = 0
pushover_priority = 0
pushover_apitoken = xxx
[PushBullet]
pushbullet_enabled = 0
pushbullet_apikey = ""
pushbullet_deviceid = ""
pushbullet_onsnatch = 0
[Twitter]
twitter_enabled = 0
twitter_onsnatch = 0
twitter_username = ""
twitter_password = ""
twitter_prefix = Headphones
[OSX_Notify]
osx_notify_enabled = 0
osx_notify_onsnatch = 0
osx_notify_app = /Applications/Headphones
[Boxcar]
boxcar_enabled = 0
boxcar_onsnatch = 0
boxcar_token = ""
[Subsonic]
subsonic_enabled = 0
subsonic_host = ""
subsonic_username = ""
subsonic_password = ""
[Songkick]
songkick_enabled = 1
songkick_apikey = xxx
songkick_location = xxx
songkick_filter_enabled = 1
[Synoindex]
synoindex_enabled = 0
[Advanced]
album_completion_pct = 80
cache_sizemb = 32
journal_mode = wal
verify_ssl_cert = 1
ignored_files = ,
ignored_folders = ,
[MPC]
mpc_enabled = 0
[Old Piratebay]
oldpiratebay = 0
oldpiratebay_ratio = ""
oldpiratebay_url = ""
[Email]
email_onsnatch = 0
email_smtp_password = ""
email_tls = 0
email_smtp_port = 25
email_smtp_server = ""
email_enabled = 0
email_smtp_user = ""
email_ssl = 0
email_to = ""
email_from = ""
[Beets]
idtag = 0
[Torznab]
extra_torznabs = ,
torznab_host = ""
torznab_apikey = ""
torznab = 0
torznab_enabled = 1
[Strike]
strike = 0
strike_ratio = ""
[Deluge]
deluge_host = ""
deluge_paused = 0
deluge_done_directory = ""
deluge_label = ""
deluge_password = ""
[Telegram]
telegram_enabled = 0
telegram_token = ""
telegram_onsnatch = 0
telegram_userid = ""
nopoz commented 8 years ago

I figured this out and fixed it in 1d029bde543725f7818a3cd6d9f7181a8c15d335

Not totally understanding the existing sorting logic that is used with preferred quality...to fix I excluded what.cd results from further sorting after the initial seeder sort was done.