search

sabnzbd / sabnzbd

SABnzbd - The automated Usenet download tool
http://sabnzbd.org
Other
2.25k stars 338 forks source link

Crash in RAR Renamer #1456

Closed Safihre closed 4 years ago

Safihre commented 4 years ago

Testing a super-obfuscated NZB (that I can't share here), the renamer crashed. I think because the NZB is a bit messed up and contains some files twice. But NZBGet is able to process it, so I hope we can too 👍 Shared the original NZB with @sanderjo for debugging!

2020-05-21 09:39:16,411::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-0.0
2020-05-21 09:39:16,411::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-0.1.0
2020-05-21 09:39:16,412::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-0.2.0
2020-05-21 09:39:16,412::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-1.0
2020-05-21 09:39:16,413::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-1.1
2020-05-21 09:39:16,413::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-1.1.0
2020-05-21 09:39:16,414::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-1.1.1
2020-05-21 09:39:16,414::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-1.2.0
2020-05-21 09:39:16,415::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-1.2.1
2020-05-21 09:39:16,415::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.0
2020-05-21 09:39:16,416::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.1.0
2020-05-21 09:39:16,416::DEBUG::[postproc:896] Detected volume-number 6 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.3 
2020-05-21 09:39:16,417::DEBUG::[postproc:896] Detected volume-number 3 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.4 
2020-05-21 09:39:16,418::DEBUG::[postproc:896] Detected volume-number 4 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.5 
2020-05-21 09:39:16,419::DEBUG::[postproc:896] Detected volume-number 12 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.6 
2020-05-21 09:39:16,420::DEBUG::[postproc:896] Detected volume-number 5 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.7 
2020-05-21 09:39:16,420::DEBUG::[postproc:896] Detected volume-number 10 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.8 
2020-05-21 09:39:16,422::DEBUG::[postproc:896] Detected volume-number 7 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-2.9 
2020-05-21 09:39:16,423::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.0
2020-05-21 09:39:16,423::DEBUG::[postproc:896] Detected volume-number 6 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1 
2020-05-21 09:39:16,424::DEBUG::[postproc:896] Detected volume-number 8 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.0 
2020-05-21 09:39:16,425::DEBUG::[postproc:896] Detected volume-number 9 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.1 
2020-05-21 09:39:16,426::DEBUG::[postproc:896] Detected volume-number 3 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.10 
2020-05-21 09:39:16,427::DEBUG::[postproc:896] Detected volume-number 1 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.11 
2020-05-21 09:39:16,427::DEBUG::[postproc:896] Detected volume-number 7 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.12 
2020-05-21 09:39:16,428::DEBUG::[postproc:896] Detected volume-number 5 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.13 
2020-05-21 09:39:16,429::DEBUG::[postproc:896] Detected volume-number 4 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.2 
2020-05-21 09:39:16,430::DEBUG::[postproc:896] Detected volume-number 2 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.3 
2020-05-21 09:39:16,431::DEBUG::[postproc:896] Detected volume-number 10 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.4 
2020-05-21 09:39:16,432::DEBUG::[postproc:896] Detected volume-number 6 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.5 
2020-05-21 09:39:16,433::DEBUG::[postproc:896] Detected volume-number 12 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.6 
2020-05-21 09:39:16,433::DEBUG::[postproc:896] Detected volume-number 5 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.7 
2020-05-21 09:39:16,434::DEBUG::[postproc:896] Detected volume-number 8 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.8 
2020-05-21 09:39:16,435::DEBUG::[postproc:896] Detected volume-number 12 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.1.9 
2020-05-21 09:39:16,436::DEBUG::[postproc:896] Detected volume-number 1 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.10 
2020-05-21 09:39:16,437::DEBUG::[postproc:896] Detected volume-number 7 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.11 
2020-05-21 09:39:16,438::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.12
2020-05-21 09:39:16,439::DEBUG::[postproc:896] Detected volume-number 9 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.13 
2020-05-21 09:39:16,440::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.2
2020-05-21 09:39:16,441::DEBUG::[postproc:896] Detected volume-number 11 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.2.0 
2020-05-21 09:39:16,442::DEBUG::[postproc:896] Detected volume-number 2 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.3 
2020-05-21 09:39:16,443::DEBUG::[postproc:896] Detected volume-number 4 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.4 
2020-05-21 09:39:16,444::DEBUG::[postproc:896] Detected volume-number 3 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.5 
2020-05-21 09:39:16,445::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.6
2020-05-21 09:39:16,445::DEBUG::[postproc:907] No RAR-volume-number found in \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.7
2020-05-21 09:39:16,446::DEBUG::[postproc:896] Detected volume-number 11 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.8 
2020-05-21 09:39:16,447::DEBUG::[postproc:896] Detected volume-number 10 from RAR-header: \\?\C:\Users\<USERNAME>\Downloads\a b\YellowPeople.S16E20.1080p.BluRay.x264\165099-3.9 
2020-05-21 09:39:16,448::DEBUG::[postproc:909] Deobfuscate: rarvolnr is: {6: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.3': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.5': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv']}, 3: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.4': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.10': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.5': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 4: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.5': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.2': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.4': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 12: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.6': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.6': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.9': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv']}, 5: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.7': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.13': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.7': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 10: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.8': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.4': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.9': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 7: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.9': ['YellowPeople.s16e19.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.12': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.11': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 8: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.0': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.8': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv']}, 9: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.1': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.13': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 1: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.11': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.10': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 2: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.3': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.3': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}, 11: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.2.0': ['YellowPeople.s16e21.1080p.bluray.x264-rovers.mkv'], '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.8': ['YellowPeople.s16e20.1080p.bluray.x264-rovers.mkv']}}
2020-05-21 09:39:16,448::DEBUG::[postproc:910] Deobfuscate: volnrext is: {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.3': (6, 'r04'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.4': (3, 'r01'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.5': (4, 'r02'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.6': (12, 'r10'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.7': (5, 'r03'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.8': (10, 'r08'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.9': (7, 'r05'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1': (6, 'r04'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.0': (8, 'r06'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.1': (9, 'r07'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.10': (3, 'r01'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.11': (1, 'rar'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.12': (7, 'r05'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.13': (5, 'r03'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.2': (4, 'r02'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.3': (2, 'r00'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.4': (10, 'r08'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.5': (6, 'r04'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.6': (12, 'r10'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.7': (5, 'r03'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.8': (8, 'r06'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.9': (12, 'r10'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.10': (1, 'rar'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.11': (7, 'r05'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.13': (9, 'r07'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.2.0': (11, 'r09'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.3': (2, 'r00'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.4': (4, 'r02'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.5': (3, 'r01'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.8': (11, 'r09'), '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.9': (10, 'r08')}
2020-05-21 09:39:16,448::DEBUG::[postproc:930] Number of obfuscated rarsets: 2
2020-05-21 09:39:16,449::DEBUG::[postproc:939] Deobfuscate: rarsetname {'\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.1.11': 'A--YellowPeople.S16E20.1080p.BluRay.x264-ROVERS', '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-3.10': 'B--YellowPeople.S16E20.1080p.BluRay.x264-ROVERS'}
2020-05-21 09:39:16,449::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 1 and 2
2020-05-21 09:39:16,449::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 2 and 3
2020-05-21 09:39:16,449::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 3 and 4
2020-05-21 09:39:16,450::ERROR::[postproc:599] Post Processing Failed for YellowPeople.S16E20.1080p.BluRay.x264-ROVERS (see logfile)
2020-05-21 09:39:16,450::INFO::[postproc:600] Traceback: 
Traceback (most recent call last):
  File "C:\Users\<USERNAME>\Documents\GitHub\sabnzbd3\sabnzbd\postproc.py", line 379, in process_job
    par_error, re_add = parring(nzo, workdir)
  File "C:\Users\<USERNAME>\Documents\GitHub\sabnzbd3\sabnzbd\postproc.py", line 772, in parring
    if rar_renamer(nzo, workdir):
  File "C:\Users\<USERNAME>\Documents\GitHub\sabnzbd3\sabnzbd\postproc.py", line 953, in rar_renamer
    rarsetname[next_obfuscated_filename] = rarsetname[base_obfuscated_filename]
KeyError: '\\\\?\\C:\\Users\\<USERNAME>\\Downloads\\a b\\YellowPeople.S16E20.1080p.BluRay.x264\\165099-2.4'
sanderjo commented 4 years ago

I think I recreated the traceback: I created a mixed obfuscated post, with one missing rar file, and it causes the same Traceback.

2020-05-22 09:28:52,161::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 1 and 2
2020-05-22 09:28:52,161::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 2 and 3
2020-05-22 09:28:52,161::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 3 and 4
2020-05-22 09:28:52,161::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 4 and 5
2020-05-22 09:28:52,161::DEBUG::[postproc:946] Deobfuscate: Finding matches between rar sets 5 and 6
2020-05-22 09:28:52,161::ERROR::[postproc:599] Post Processing Failed for reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045 (see logfile)
2020-05-22 09:28:52,162::INFO::[postproc:600] Traceback:
Traceback (most recent call last):
  File "/home/sander/git/sabnzbd/sabnzbd/postproc.py", line 379, in process_job
    par_error, re_add = parring(nzo, workdir)
  File "/home/sander/git/sabnzbd/sabnzbd/postproc.py", line 772, in parring
    if rar_renamer(nzo, workdir):
  File "/home/sander/git/sabnzbd/sabnzbd/postproc.py", line 953, in rar_renamer
    rarsetname[next_obfuscated_filename] = rarsetname[base_obfuscated_filename]
KeyError: '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045/reftestnzb mixed obfuscated one missing rar 9db0422c3045 [02+13] - "0d7d4e7a-f4f5-4a5f-b7c2-72a787b4def1" yEnc (1+22) 15728640'

NZB is here https://raw.githubusercontent.com/sanderjo/NZBs/master/obfuscated-posts/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.nzb

sanderjo commented 4 years ago

Inspecting the YellowPeople files:

No PAR files. Some plain Matroska, so direct renaming possible? Some of the ASCII files are sfv files, so direct renaming possible

$ for f in *; do file $f ; done | sort -k 3,3 
__ADMIN__: directory
165099-1.1.1: RAR archive data, v2.0, os: Win32
165099-1.1: RAR archive data, v2.0, os: Win32
165099-1.2.1: RAR archive data, v2.0, os: Win32
165099-2.3: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-2.4: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-2.5: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-2.6: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-2.7: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-2.8: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-2.9: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.0: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.10: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume FirstVolume
165099-3.1.10: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.11: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume FirstVolume
165099-3.1.12: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.13: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.1: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.11: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.2: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.3: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.13: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.4: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.5: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.6: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.7: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.8: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1.9: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.1: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.2.0: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.3: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.4: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.5: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.8: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-3.9: RAR archive data, v2.0, os: Win32, flags: ArchiveVolume
165099-0.0: Matroska data
165099-0.1.0: Matroska data
165099-0.2.0: Matroska data
165099-3.2: Non-ISO extended-ASCII text
165099-3.7: Non-ISO extended-ASCII text
165099-2.0: JPEG image data, baseline, precision 8, 3264x2448, components 3
165099-2.1.0: JPEG image data, baseline, precision 8, 3264x2448, components 3
165099-3.0: JPEG image data, baseline, precision 8, 3264x2448, components 3
165099-1.0: ASCII text, with CRLF line terminators
165099-1.1.0: ASCII text, with CRLF line terminators
165099-1.2.0: ASCII text, with CRLF line terminators
165099-3.12: ASCII text, with CRLF line terminators
165099-3.6: ASCII text, with CRLF line terminators
sanderjo commented 4 years ago

Inspecting the CRLF files, they are all sfv, with usefu content:

$ for f in *; do echo -n "$f ... " ; file $f ; done | sort -k 4,4 | grep CRLF | awk '{ print "cat " $1 }'  | /bin/sh | sed -e 's/blabla/yellowpeople/g' 
the.yellowpeople.s16e19.1080p.bluray.x264-rovers.subs.rar 491e3075
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.subs.rar a86cf67c
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.subs.rar 6df3216c
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r00 f5ad4679
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r01 86bea7fd
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r02 e3c684a4
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r03 992f4c5f
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r04 f00cc757
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r05 efd11163
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r06 202f1dfc
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r07 8dffc551
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r08 60969c5a
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r09 d7641632
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.r10 d3c301d9
the.yellowpeople.s16e20.1080p.bluray.x264-rovers.rar 2d54cf70
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r00 f33a5300
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r01 3835cc58
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r02 eb9d1b13
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r03 aa8cd45d
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r04 4b0eb82f
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r05 8a623f7f
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r06 be5a4225
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r07 c7c62043
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r08 26ed2a3b
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r09 3b5653cf
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.r10 aad2f093
the.yellowpeople.s16e21.1080p.bluray.x264-rovers.rar 9b92b362

How about this: find the SFV files, renamve to something.sfv, and then feed it to SAB? SAB can already rename rar based on SFV info.

sanderjo commented 4 years ago

sabnzbd.log shows some rar volume numbers occur 2 times, others occur 3 times. That is quite ugly. And probably not solvable by looking at the rar numbers.

$ cat ~/.sabnzbd/logs/sabnzbd.log | grep S16E20.1080p | grep Detected | awk '{ print $5 }' | sort -n | uniq -c
      2 1
      2 2
      3 3
      3 4
      3 5
      3 6
      3 7
      2 8
      2 9
      3 10
      2 11
      3 12
sanderjo commented 4 years ago

So: proposal:

introduce function is_SFV_file(file), which returns boolean. Critria:

if True (so: SFV file), SAB renames that file to blabla.sfv and feeds it to the existing renamer.

thezoggy commented 4 years ago

what will this do if

Safihre commented 4 years ago

@thezoggy that wouldn't be any different from other posts that do have sfv but not par2: they would fail. We have had sfv support for ages and it has always worked like this. Only difference is now that we would detect it in obfuscated posts.

sanderjo commented 4 years ago

With a try-except the Traceback is gone, and SAB tells what is wrong:

no previous match for /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/reftestnzb mixed obfuscated one missing rar 9db0422c3045 [06+13] - '2d4b6efa-93ee-4991-9f1d-f0c3dad1f356' yEnc (1+15) 10487026

So far, so good.

But, I've created a very ugly, of which the first set ("A") is complete, the second set ("B") has only a first rar files, and the third set ("C") has no starting rar files so is on orphan.

So, after the rename: image

SAB nicely unrars the A set, fails on the B (of course), but then the A set is gone.

Is that OK? @Safihre

image

2020-05-25 23:54:48,528::DEBUG::[newsunpack:806] UNRAR output 
UNRAR 5.61 beta 1 freeware      Copyright (c) 1993-2018 Alexander Roshal

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part001.rar

Extracting  /home/sander/Downloads/complete/_UNPACK_reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/movie2.bin

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part002.rar

...         movie2.bin

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part003.rar

...         movie2.bin

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part004.rar

...         movie2.bin

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part005.rar

...         movie2.bin

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part006.rar

...         movie2.bin

Extracting from /home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part007.rar

...         movie2.bin                                                OK
All OK
2020-05-25 23:54:48,528::INFO::[newsunpack:810] Unpacked 1 files/folders in 0 seconds
2020-05-25 23:54:48,528::DEBUG::[newsunpack:538] rar_unpack(): Rars: ['/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part001.rar', '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part002.rar', '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part003.rar', '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part004.rar', '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part005.rar', '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part006.rar', '/home/sander/Downloads/incomplete/reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/A--reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.part007.rar']
2020-05-25 23:54:48,528::DEBUG::[newsunpack:539] rar_unpack(): Newfiles: ['/home/sander/Downloads/complete/_UNPACK_reftestnzb_mixed_obfuscated_one_missing_rar_9db0422c3045.1/movie2.bin']
Safihre commented 4 years ago

but then the A set is gone.

Why is set A gone?

sanderjo commented 4 years ago

Why is set A gone?

I created a non-obfuscated post https://raw.githubusercontent.com/sanderjo/NZBs/master/reftestnzb-one-complete-rarset-one-half-rarset-f7cea3b99a21.nzb : one complete rar-set, one half rarset (only first rar files).

That gives this result: two dirs, one in incomplete (with the incomplete rarset) and one in complete (with the unrarrable rarset)

$ ll incomplete/reftestnzb-one-complete-rarset-one-half-rarset-f7cea3b99a21/
total 14348
drwxrwxr-x 3 sander sander    4096 mei 26 08:21 ./
drwxrwxr-x 5 sander sander    4096 mei 26 08:21 ../
drwxrwxr-x 2 sander sander    4096 mei 26 08:21 __ADMIN__/
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:21 sometestfile-41MB.part1.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:21 sometestfile-41MB.part2.rar

and

$ ll complete/_UNPACK_reftestnzb-one-complete-rarset-one-half-rarset-f7cea3b99a21/
total 40968
drwxrwxr-x  2 sander sander     4096 mei 26 08:21 ./
drwx------ 15 sander sander     4096 mei 26 08:21 ../
-rw-rw-r--  1 sander sander 41943040 mei 26 08:16 my_rand_40MB.bin

I'll check again with the obfuscated incomplete set.

Safihre commented 4 years ago

Aaah, so it does unpack the set A but fails the rest of the job due to the other broken files. Well then it's fine 👍

sanderjo commented 4 years ago

Now with the same rar set but now obfuscated:

Incomplete

$ ll incomplete/reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96/
total 55312
drwxrwxr-x 3 sander sander    4096 mei 26 08:32 ./
drwxrwxr-x 6 sander sander    4096 mei 26 08:31 ../
drwxrwxr-x 2 sander sander    4096 mei 26 08:32 __ADMIN__/
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 A--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part001.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 A--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part002.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 B--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part001.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 B--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part002.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 B--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part003.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 B--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part004.rar
-rw-rw-r-- 1 sander sander 7340032 mei 26 08:32 B--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part005.rar
-rw-rw-r-- 1 sander sander 5244012 mei 26 08:32 B--reftestnzb-OBFUSCATED-one-complete-rarset-one-half-rarset-931a8828cc96.part006.rar

Ah, it first found the incomplete set, tried to unrarred that, failed and .... stopped. No attempt on the other rarset. And thus nothing in complete

Safihre commented 4 years ago

That's fine, we did our best, it's just too much of a mess in case stuff is broken.

sanderjo commented 4 years ago

Fun fact:

The YellowPeople nzb now downloads fine, no traceback, nice warnings, .... and the good files get unrarred.

image

sanderjo commented 4 years ago

@Safihre is the WARNING ok for you?