Wrong directory skipping

IvanShift commented 4 years ago

Hello! I have dir with name "South.Park.S21.1080p.WEB-DL.H.264" but scanner skip this dir. Directory with name "South.Park.S20.1080p.WEB-DL.H.264.Paramount" scanned successfully. I suppose that scanner do not like last symbols of this dir: ".264". Can you fix it?

ZeroQI commented 4 years ago

Custom scanner logs missing, located in logs folder in a subfolder named after the library name With that may be able to add terms to remove in series folder name

IvanShift commented 4 years ago

Do you mean these files? I also add a screenshot of folder with this cartoon series. As you can see, two folders are missing in the log files: “South.Park.S21.1080p.WEB-DL.H.264” and “South.Park.S22.1080p.WEB-DL.H.264”.

dir_screenshot

South Park.filelist.log South Park.scanner.log

ZeroQI commented 4 years ago

These are the right logs for the grouping folder 'South Park' on top, but none are showing season 21 files so they useless for the folders mentionned...

So if I understand right, you don't attach logs on first post and don't bother reading before attaching, while your folders are clearly not renamed in any way to facilitate the scanner's job...

Didn't you have other logs with the folder names that have issues? Are you sure the season folder is not detected as an unrecognised series?

IvanShift commented 4 years ago

Hello! "These are the right logs for the grouping folder 'South Park' on top, but none are showing season 21 files so they useless for the folders mentionned..." That is the point! This folder (with 21 оr 22 season) do not present in log files! That is why I didn't add those files in first post. Scanner just ignore folders with 21 and 22 seasons. Could you fix it?

Top folder has name "South Park" and sub folder has the same name "South Park" (you can see this sub folder on screen above), so this may confuse you.

I definitely added correct log files. You can check it. They contain all the folders shown in the screenshot, except for the 21 and 22 seasons. Also those files has not been added as an unrecognized series.

ZeroQI commented 4 years ago

Logs are empty so nothing to go on...

Move both folders, scan. Put them back, scan again. (Can't rescan if folder content didn't change) Attach both file again together with Plex Media scanner.log

IvanShift commented 4 years ago

I created folder "21-22" and put inside both folders with no result (South Park.filelist.log and South Park.scanner.log). Then a put them back and made new scan (South Park.filelist.log and Park.scanner.log). Results are the same - both logs have no entries about 21 and 22 season.

After that I put last episode (South.Park.S22E10.Bike.Parade.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv) from folder "South.Park.S22.1080p.WEB-DL.H.264" to root folder (folder.jpg) and made scan (South Park.filelist_outS22E10.log and South Park.scanner_outS22E10.log). Both seasons appear in Plex (21-22.JPG) but only this 2 seasons. Other seasons are disappear from Plex and from log-files. Very strange behavior.

south park.zip

IvanShift commented 4 years ago

Problem is in WHACK_PRE_CLEAN_RAW. If I remove "H.264" from this array everything is going to be fine. But it's good way to delete this element or not?

ZeroQI commented 4 years ago

If the folder content doesn't change Plex is not going which I why I requested to remove in t from Plex reach and putting them back. Moving them will reuse the cache and be messy

WHACK_PRE_CLEAN_RAW will remove from every folder so we need to be sure it will never be in a series title. 'H.264' should remove it if anything, I don't get why it cause issues

IvanShift commented 4 years ago

" why I requested to remove in t from Plex reach and putting them back" I also did it without any result. Only removing 'H.264 helps.

ZeroQI commented 4 years ago

Thanks for confirming. Makes no sense to me, unless your season folders being misnamed ends up scrubbed too well and the title match the other folder and trigger a bug. Being misnammed, it isn't detected as a season folder like it should if names 'Season xx'. That would explain why season folders with more then after H.264 ends up detected... Tricky to prove...

IvanShift commented 4 years ago

Maybe it is possible to log out more debug info in you script? Maybe it helps to detect reason such strange behavior?

IvanShift commented 4 years ago

Hello! Could you explain where exactly "clean_string" runs for folder names? I run clean_string with "South.Park.S21.1080p.WEB-DL.H.264" and receive "South.Park.S21"

Also season number was successfully extracted in 542 string (folder_season = int( match.group('season')) )

But what is going next?

ZeroQI commented 4 years ago

I believe it's 3 lines highter match = rx.search(clean_string(folder)) currently line 542 coincidently I am not sure what is going on, it make no sense to me... Why would it work removing this exception ?

EndOfLine369 commented 4 years ago

Hi @IvanShift, is this still an issue? If so, it looks like your having issues with a grouping folders. Ya'll have been looking at the actual scaned grouping folder logs but not the root log that actually adds up all the 'Manual Scan' grouping folders to be scanned. I would think this to be where you should clearly see your issue. The file is in the format of "_root_-<root_path>.scanner.log". This log shows you the output of any base root files along with the grouping folder evaluations.

Oh, and the point where you experienced the entries disappear when you added just one piece of that series to the root folder is expected. You added the same series in two different sections. This is a Plex limitation. In the scan, first the grouping folders are scanned at initial root scan, added to plex, the the individual series folder is scanned, same series is added, this thus removes the first add of the series from the grouping folder scan. Either the series has to be all in a grouping folder or not.

ZeroQI commented 4 years ago

@IvanShift Hi, is this still an issue?

IvanShift commented 4 years ago

Yes, I still have this issue. South Park.filelist.log South Park.scanner.log root.scanner.log root-volume2-Video-sorted-HD Cartoon Series.filelist.log

root-volume2-Video-sorted-HD Cartoon Series.scanner.log

These folders (seasons) are not added to Plex: South.Park.S21.1080p.WEB-DL.H.264 South.Park.S22.1080p.WEB-DL.H.264

IvanShift commented 4 years ago

Everything is correct if I removed "H.264" from WHACK_PRE_CLEAN_RAW. https://github.com/ZeroQI/Absolute-Series-Scanner/issues/276#issuecomment-626347869

root-volume2-Video-sorted-HD Cartoon Series.filelist.log root-volume2-Video-sorted-HD Cartoon Series.scanner.log South Park.filelist.log South Park.scanner.log root.scanner.log

ZeroQI commented 4 years ago

I can only see them added to the scan if you did remove the string

[Added] /volume2/Video/sorted/HD Cartoon Series/South Park/South.Park.S21.1080p.WEB-DL.H.264
[Added] /volume2/Video/sorted/HD Cartoon Series/South Park/South.Park.S22.1080p.WEB-DL.H.264

I can see them scanned propertly only with the string removed:

Library: 'no valid X-Plex-Token.id', root: '/volume2/Video/sorted/HD Cartoon Series', path: 'South Park/South.Park.S21.1080p.WEB-DL.H.264', files: '10', dirs: '1'
Manual scan start: 2020-07-11 13:17:25,706297
=============================================================================================================================================================
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'and': 1, 'S21E08': 1, 'HARD': 1, 'White': 1, 'Holiday': 1, 'Doubling': 1, 'Sons': 1, 'Park': 10, 'Piglets': 1, 'Houses': 1, 'S21E09': 1, 'Heroin': 1, 'Put': 1, 'Moss': 1, 'Franchise': 1, 'S21E02': 1, 'S21E01': 1, 'S21E07': 1, 'S21E06': 1, 'S21E05': 1, 'S21E04': 1, 'Tomato': 1, 'H': 10, 'S21E03': 1, 'Witches': 1, '264': 10, 'Hummels': 1, 'Prequel': 1, 'South': 10, 'a': 1, 'People': 1, 'S21E10': 1, 'Splatty': 1, 'PCness': 1, 'It': 1, 'Down': 2, 'Renovating': 1, 'SUPER': 1, 'Special': 1}
misc_words: ['South', 'Park', '264', 'H']
-------------------------------------------------------------------------------------------------------------------------------------------------------------
"South.Park.S21.1080p.WEB-DL.H.264" s21e001                         "SERIES_RX-1" "W ite.People.Renovating.Houses. . .H" "South.Park.S21E01.White.People.Renovating.Houses.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e002                         "SERIES_RX-1" "Put.It.Down" "South.Park.S21E02.Put.It.Down.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e003                         "SERIES_RX-1" "oliday.Special. . .H" "South.Park.S21E03.Holiday.Special.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e004                         "SERIES_RX-1" "Franc ise.Prequel. . .H" "South.Park.S21E04.Franchise.Prequel.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e005                         "SERIES_RX-1" "ummels.and.Heroin. . .H" "South.Park.S21E05.Hummels.and.Heroin.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e006                         "SERIES_RX-1" "Sons.a.Witc es. . .H" "South.Park.S21E06.Sons.a.Witches.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e007                         "SERIES_RX-1" "Doubling.Down" "South.Park.S21E07.Doubling.Down.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e008                         "SERIES_RX-1" "Moss.Piglets" "South.Park.S21E08.Moss.Piglets.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e009                         "SERIES_RX-1" "SUPER. ARD.PCness. . .H" "South.Park.S21E09.SUPER.HARD.PCness.1080p.WEB-DL.H.264.mkv"
"South.Park.S21.1080p.WEB-DL.H.264" s21e010                         "SERIES_RX-1" "Splatty.Tomato" "South.Park.S21E10.Splatty.Tomato.1080p.WEB-DL.H.264.mkv"
=============================================================================================================================================================
Manual scan end: 2020-07-11 13:17:25,729968
=============================================================================================================================================================
Library: 'no valid X-Plex-Token.id', root: '/volume2/Video/sorted/HD Cartoon Series', path: 'South Park/South.Park.S22.1080p.WEB-DL.H.264', files: '10', dirs: '0'
Manual scan start: 2020-07-11 13:17:25,732420
=============================================================================================================================================================
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'and': 1, 'Box': 1, 'Scoots': 1, 'Time': 1, 'Bike': 1, 'Buddha': 1, 'Got': 1, 'Problem': 1, '26': 1, 'Poo': 1, 'H264': 7, 'Nobody': 1, 'Park': 10, 'S22E08': 1, 'to': 1, 'Cereal': 2, 'S22E10': 1, 'A': 1, 'Boy': 1, 'Kids': 1, 'Unfulfilled': 1, 'UNCENSORED': 10, 'H': 3, 'Priest': 1, '264': 2, 'The': 2, 'with': 1, 'South': 10, 'a': 2, 'Parade': 1, 'Get': 1, 'Dead': 1, 'Tegridy': 1, 'Farms': 1, 'S22E09': 1, 'S22E02': 1, 'S22E03': 1, 'S22E01': 1, 'S22E06': 1, 'S22E07': 1, 'S22E04': 1, 'S22E05': 1}
misc_words: ['UNCENSORED', 'South', 'Park']
-------------------------------------------------------------------------------------------------------------------------------------------------------------
"South.Park.S22.1080p.WEB-DL.H.264" s22e001                         "SERIES_RX-1" "Dead.Kids. . . .H.264" "South.Park.S22E01.Dead.Kids.UNCENSORED.1080p.WEB-DL.H.264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e002                         "SERIES_RX-1" "A.Boy.and.a.Priest. . . .H.26" "South.Park.S22E02.A.Boy.and.a.Priest.UNCENSORED.1080p.WEB-DL.H.26.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e003                         "SERIES_RX-1" "The.Problem.with.a.Poo. . . .H.264" "South.Park.S22E03.The.Problem.with.a.Poo.UNCENSORED.1080p.WEB-DL.H.264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e004                         "SERIES_RX-1" "Tegridy.Farms. . . . .H264" "South.Park.S22E04.Tegridy.Farms.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e005                         "SERIES_RX-1" "The.Scoots. . . . .H264" "South.Park.S22E05.The.Scoots.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e006                         "SERIES_RX-1" "Time.to.Get.Cereal. . . . .H264" "South.Park.S22E06.Time.to.Get.Cereal.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e007                         "SERIES_RX-1" "Nobody.Got.Cereal. . . . .H264" "South.Park.S22E07.Nobody.Got.Cereal.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e008                         "SERIES_RX-1" "Buddha.Box. . . . .H264" "South.Park.S22E08.Buddha.Box.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e009                         "SERIES_RX-1" "Unfulfilled. . . . .H264" "South.Park.S22E09.Unfulfilled.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South.Park.S22.1080p.WEB-DL.H.264" s22e010                         "SERIES_RX-1" "Bike.Parade. . . . .H264" "South.Park.S22E10.Bike.Parade.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
=============================================================================================================================================================
Manual scan end: 2020-07-11 13:17:25,758879

It is as if it crashed with no logs, and i cannot find an explanation... If you add '1080p.WEB-DL.H.264' from WHACK_PRE_CLEAN_RAW before 'H.264', does it help?

IvanShift commented 4 years ago

Hello! There are no crashes! You can easily repeat this problem by creating this folder ('South.Park.S21.1080p.WEB-DL.H.264') on your PC and scanning it. This is scanner algorithm problem.

IvanShift commented 4 years ago

It is as if it crashed with no logs, and i cannot find an explanation... If you add '1080p.WEB-DL.H.264' from WHACK_PRE_CLEAN_RAW before 'H.264', does it help?

I will check it later

IvanShift commented 4 years ago

It is as if it crashed with no logs, and i cannot find an explanation... If you add '1080p.WEB-DL.H.264' from WHACK_PRE_CLEAN_RAW before 'H.264', does it help?

It doesn't help (

IvanShift commented 3 years ago

Hello! I finally found the lines that are causing this bug.

line 1114: if rx.search(clean_string(folder))

When folder is "South.Park.S21.1080p.WEB-DL.H.264" function clean_string(folder) returns "South.Park.S21" and after that rx.search(clean_string(folder)) match with this regexp:

line 67: cic(r'^(?P<show>.*?)[\._\- ]+S(?P<season>\d{2})$')

and finally folder will be removing at next line:

line 1115: reverse_path.remove(folder)

Could you please fix this issue?

ZeroQI commented 3 years ago

It's normal behaviour for me

folder is "South.Park.S21.1080p.WEB-DL.H.264"
clean_string(folder) remove useless tags to make it "South.Park.S21"
"South.Park.S21" match season folder so remove season folder from reverse_path to find the series folder

I recreated "South Park/South.Park.S21.1080p.WEB-DL.H.264" and the 10 files inside and scanned using latest version, no mod, and it worked with no issue...

=============================================================================================================================================================
Library: 'TV Shows', root: 'C:\Users\benja\Videos\_Plex', path: 'South Park', files: '0', dirs: '1'
Plex scan start: 2021-02-22 00:01:48,399000
=============================================================================================================================================================
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'': 1}
misc_words: []
-------------------------------------------------------------------------------------------------------------------------------------------------------------
[no files detected] #1
=============================================================================================================================================================
Plex scan end: 2021-02-22 00:01:48,403000
=============================================================================================================================================================
Library: 'TV Shows', root: 'C:\Users\benja\Videos\_Plex', path: 'South Park\South.Park.S21.1080p.WEB-DL.H.264', files: '10', dirs: '0'
Plex scan start: 2021-02-22 00:01:48,413000
=============================================================================================================================================================
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'and': 1, 'S21E08': 1, 'HARD': 1, 'White': 1, 'Holiday': 1, 'Doubling': 1, 'Sons': 1, 'Park': 10, 'Piglets': 1, 'Houses': 1, 'S21E09': 1, 'Heroin': 1, 'Put': 1, 'Moss': 1, 'Franchise': 1, 'S21E02': 1, 'S21E01': 1, 'S21E07': 1, 'S21E06': 1, 'S21E05': 1, 'S21E04': 1, 'Tomato': 1, 'S21E03': 1, 'Witches': 1, 'Hummels': 1, 'Prequel': 1, 'South': 10, 'a': 1, 'People': 1, 'S21E10': 1, 'Splatty': 1, 'PCness': 1, 'It': 1, 'Down': 2, 'Renovating': 1, 'SUPER': 1, 'Special': 1}
misc_words: ['South', 'Park']
-------------------------------------------------------------------------------------------------------------------------------------------------------------
"South Park" s21e001                         "SERIES_RX-1" "White.People.Renovating.Houses" "South.Park.S21E01.White.People.Renovating.Houses.1080p.WEB-DL.H.264.mkv"
"South Park" s21e002                         "SERIES_RX-1" "Put.It.Down" "South.Park.S21E02.Put.It.Down.1080p.WEB-DL.H.264.mkv"
"South Park" s21e003                         "SERIES_RX-1" "Holiday.Special" "South.Park.S21E03.Holiday.Special.1080p.WEB-DL.H.264.mkv"
"South Park" s21e004                         "SERIES_RX-1" "Franchise.Prequel" "South.Park.S21E04.Franchise.Prequel.1080p.WEB-DL.H.264.mkv"
"South Park" s21e005                         "SERIES_RX-1" "Hummels.and.Heroin" "South.Park.S21E05.Hummels.and.Heroin.1080p.WEB-DL.H.264.mkv"
"South Park" s21e006                         "SERIES_RX-1" "Sons.a.Witches" "South.Park.S21E06.Sons.a.Witches.1080p.WEB-DL.H.264.mkv"
"South Park" s21e007                         "SERIES_RX-1" "Doubling.Down" "South.Park.S21E07.Doubling.Down.1080p.WEB-DL.H.264.mkv"
"South Park" s21e008                         "SERIES_RX-1" "Moss.Piglets" "South.Park.S21E08.Moss.Piglets.1080p.WEB-DL.H.264.mkv"
"South Park" s21e009                         "SERIES_RX-1" "SUPER.HARD.PCness" "South.Park.S21E09.SUPER.HARD.PCness.1080p.WEB-DL.H.264.mkv"
"South Park" s21e010                         "SERIES_RX-1" "Splatty.Tomato" "South.Park.S21E10.Splatty.Tomato.1080p.WEB-DL.H.264.mkv"
=============================================================================================================================================================
Plex scan end: 2021-02-22 00:01:48,452000
=============================================================================================================================================================
Library: 'TV Shows', root: 'C:\Users\benja\Videos\_Plex', path: 'South Park\South.Park.S22.1080p.WEB-DL.H.264', files: '10', dirs: '0'
Plex scan start: 2021-02-22 00:11:28,191000
=============================================================================================================================================================
-------------------------------------------------------------------------------------------------------------------------------------------------------------
misc_count: {'and': 1, 'Box': 1, 'Scoots': 1, 'Bike': 1, 'Buddha': 1, 'Got': 1, 'Problem': 1, '26': 1, 'Poo': 1, 'H264': 7, 'Nobody': 1, 'Park': 10, 'S22E08': 1, 'to': 1, 'Cereal': 2, 'S22E10': 1, 'A': 1, 'Boy': 1, 'Kids': 1, 'Unfulfilled': 1, 'UNCENSORED': 10, 'H': 1, 'Priest': 1, 'Time': 1, 'The': 2, 'with': 1, 'South': 10, 'a': 2, 'Parade': 1, 'Get': 1, 'Dead': 1, 'Tegridy': 1, 'Farms': 1, 'S22E09': 1, 'S22E02': 1, 'S22E03': 1, 'S22E01': 1, 'S22E06': 1, 'S22E07': 1, 'S22E04': 1, 'S22E05': 1}
misc_words: ['UNCENSORED', 'South', 'Park']
-------------------------------------------------------------------------------------------------------------------------------------------------------------
"South Park" s22e001                         "SERIES_RX-1" "Dead.Kids" "South.Park.S22E01.Dead.Kids.UNCENSORED.1080p.WEB-DL.H.264.mkv"
"South Park" s22e002                         "SERIES_RX-1" "A.Boy.and.a.Priest. . . .H.26" "South.Park.S22E02.A.Boy.and.a.Priest.UNCENSORED.1080p.WEB-DL.H.26.mkv"
"South Park" s22e003                         "SERIES_RX-1" "The.Problem.with.a.Poo" "South.Park.S22E03.The.Problem.with.a.Poo.UNCENSORED.1080p.WEB-DL.H.264.mkv"
"South Park" s22e004                         "SERIES_RX-1" "Tegridy.Farms. . . . .H264" "South.Park.S22E04.Tegridy.Farms.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South Park" s22e005                         "SERIES_RX-1" "The.Scoots. . . . .H264" "South.Park.S22E05.The.Scoots.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South Park" s22e006                         "SERIES_RX-1" "Time.to.Get.Cereal. . . . .H264" "South.Park.S22E06.Time.to.Get.Cereal.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South Park" s22e007                         "SERIES_RX-1" "Nobody.Got.Cereal. . . . .H264" "South.Park.S22E07.Nobody.Got.Cereal.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South Park" s22e008                         "SERIES_RX-1" "Buddha.Box. . . . .H264" "South.Park.S22E08.Buddha.Box.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South Park" s22e009                         "SERIES_RX-1" "Unfulfilled. . . . .H264" "South.Park.S22E09.Unfulfilled.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
"South Park" s22e010                         "SERIES_RX-1" "Bike.Parade. . . . .H264" "South.Park.S22E10.Bike.Parade.UNCENSORED.1080p.WEB-DL.AAC2.0.H264.mkv"
=============================================================================================================================================================
Plex scan end: 2021-02-22 00:11:28,219000

IvanShift commented 3 years ago

Hello! Please, add to folder "South Park" new one: "South.Park.S20.1080p.WEB-DL.H.264.Paramount" and scan it again. And you will see that only one folder will be scanned: "South.Park.S20.1080p.WEB-DL.H.264.Paramount"

South Park.scanner (one folder s21).log South Park.scanner (two folder S21 and S20).log

This is my folder structure for test (2 folders with a file inside):

 Folder Z:\sorted\Test\South Park
22.02.2021  10:36    <DIR>          South.Park.S20.1080p.WEB-DL.H.264.Paramount
22.02.2021  10:29    <DIR>          South.Park.S21.1080p.WEB-DL.H.264

 Folder Z:\sorted\Test\South Park\South.Park.S20.1080p.WEB-DL.H.264.Paramount
20.02.2021  10:35                 0 South.Park.S20E01.Member.Berries.1080p.WEB-DL.H.264.Paramount.mkv

 Folder Z:\sorted\Test\South Park\South.Park.S21.1080p.WEB-DL.H.264
20.02.2021  10:36                 0 South.Park.S21E02.Put.It.Down.1080p.WEB-DL.H.264.mkv

ZeroQI commented 3 years ago

I could reproduce it that way, thanks for the precision. According the root scanner.log file, the scanner considers it a grouping folder when they are actually season folders

https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner.py Lines 064-069, last entry is transparent folder, removed automatically, is not considered another depth

SEASON_RX       = [                                                                                                                                                              ### Seasons Folders
                    cic(r'^Specials'),                                                                                                                                           # Specials (season 0)
                    cic(r'^(Season|Series|Book|Saison|Livre|Temporada|S)[ _\-\.]*(?P<season>\d{1,4})'),                                                                                    # Season / Series / Book / Saison / Livre / S
                    cic(r'^(?P<show>.*?)[\._\- ]+S(?P<season>\d{2})$'),                                                                                                          # (title) S01
                    cic(r'^(?P<season>\d{1,2})'),                                                                                                                                # ##
                    cic(r'^(Saga|(Story )?Ar[kc])')]

The issue seem to be that "South.Park.S20.1080p.WEB-DL.H.264.Paramount" is not considered season folder by cic(r'^(?P.*?)[._- ]+S(?P\d{2})$'), i believe the $ sign and "H264.Paramount" not scrubbed is at fault

'$' for season regex... Anything after "title Sxx" failed the season detection... Tested and it detected all 3 seasons instead of just the 20th one...

IvanShift commented 3 years ago

Unfortunately it doesn't help. In this case (2 dirs "South.Park.S20.1080p.WEB-DL.H.264.Paramount" and "South.Park.S21.1080p.WEB-DL.H.264") it helps but on all folder structures scanner fails: I have next folder structure inside "South Park": Scanner successfully scans at first 3 folders:

"South Park" - inside I have 01-13 series
"South Park - Season 14 BluRay 1080p [2x2, VO-production]"
"South.Park.2012.Season.16.WEB-DL.1080p.2xDVO.MVO.VO.ENG.Subs"

All other folders are skipping:

South.Park.S15.1080p.BDRip.Rus.Eng
South.Park.S17.1080p.BDRip.Rus.Eng
South.Park.S18.720p.WEB-DL.DD5.1.H264-CasStudio
South.Park.s19.720p.WEB-DL.Rus.Eng.Sub
South.Park.S20.1080p.WEB-DL.H.264.Paramount
South.Park.S21.1080p.WEB-DL.H.264
South.Park.S22.1080p.WEB-DL.H.264
South.Park.S23.UNCENSORED.1080p.WEB-DL.H.264-THD
South.Park.S24

South Park.scanner.log South Park.filelist.log

The same situation with only one season cartoon: Top folder "Solar Opposites", inside "Solar.Opposites.S01.WEBDL.1080p.NewStudio"which was successfully skipped when scanning: Solar Opposites.filelist.log Solar Opposites.scanner.log

ZeroQI commented 3 years ago

The season folder naming are atrocious, but if we can have the scanner see them all as seasons, it will work One folder wrong and all other folders are skipped, in sort the lowercase s should be the cause

Line 67 replace: cic(r'^(?P<show>.*?)[\._\- ]+S(?P<season>\d{2})'), # (title) S01 With cic(r'^(?P<show>.*?)[\._\- ]+[Ss](?P<season>\d{2})'), # (title) S01

IvanShift commented 3 years ago

There are no changes for "South Park" or "Solar" Opposites"(

ZeroQI commented 3 years ago

Reproduced the issue, answer was in the root folder scanner.log fileroot...-_Plex.scanner.log Solved, the season folder was not picked up (as totally not standard naming) as such but as grouping folder messing up everything. Simplified reged for season and made very permissive Please test and report, but should be good this time

IvanShift commented 3 years ago

Hello! South Park was scanned completely! Thank you! South Park.scanner.log South Park.filelist.log

But some other series still not scanned, for example: Solar Opposites.filelist.log Solar Opposites.scanner.log

Main dir: "Solar Opposites" Inside: "Solar.Opposites.S01.WEBDL.1080p.NewStudio" File list:

Solar.Opposites.S01E01.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E02.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E03.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E04.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E05.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E06.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E07.WEBDL.1080p.NS.RGzsRutracker.mkv
Solar.Opposites.S01E08.WEBDL.1080p.NS.RGzsRutracker.mkv

ZeroQI commented 3 years ago

Solar Opposites/Solar.Opposites.S01.WEBDL.1080p.NewStudio/Solar.Opposites.S01E01.WEBDL.1080p.NS.RGzsRutracker.mkv

Episode folders are NOT supported officially, as the scanner can't tell grouping folders season folders and episode folders appart, and here the foldername doesn't even match the filename.

I need root scanner logs, probably named _root_volume4-Video-sorted-HD Cartoon Series-_Plex.scanner.log, as it may work once fixed, even detected as grouping folder but cannot make garantee

IvanShift commented 3 years ago

This is log-files: root-volume4-Video-sorted-HD Cartoon Series.filelist.log root-volume4-Video-sorted-HD Cartoon Series.scanner.log

ZeroQI commented 3 years ago

Apologies, they are not episode folders (1 folder per episode), but it sees no ep /Solar Opposites/Solar.Opposites.S01.WEBDL.1080p.NewStudio/Solar.Opposites.S01Exx.WEBDL.1080p.NS.RGzsRutracker.mkv Please try that fix, i can't reproduce the issue this time, it is working for me

Note: When pushing an update with just the bug link in comments, it closes the ticket...

ZeroQI commented 3 years ago

Is everything ok now?

IvanShift commented 3 years ago

Could you please be more specific or rephrase? What exactly should I do? I tried last version but there is no changes in "Solar Opposites": Solar Opposites.filelist.log Solar Opposites.scanner.log

But with "South Park" I have regression: South Park.filelist.log South Park.scanner.log

root-volume4-Video-sorted-HD Cartoon Series.filelist.log root-volume4-Video-sorted-HD Cartoon Series.scanner.log

ZeroQI commented 3 years ago

Have created same folders same file, but don't have the problem, and South park is still detected....

=============================================================================================================================================================
Library: 'TV Shows', root: 'C:\Users\benja\Videos\_Plex', path: 'Solar Opposites', files: '0', dirs: '1'
Plex scan start: 2021-02-25 17:36:03,348000
plexignore_files: '[]', plexignore_dirs: '[]'
=============================================================================================================================================================
[folder] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio
[no files detected] continuing, single folder
=============================================================================================================================================================
Plex scan end: 2021-02-25 17:36:03,348000
=============================================================================================================================================================
Library: 'TV Shows', root: 'C:\Users\benja\Videos\_Plex', path: 'Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio', files: '8', dirs: '0'
Plex scan start: 2021-02-25 17:36:03,356000
plexignore_files: '[]', plexignore_dirs: '[]'
=============================================================================================================================================================
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E01.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E02.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E03.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E04.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E05.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E06.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E07.WEBDL.1080p.NS.RGzsRutracker.mkv
[file] Solar Opposites\Solar.Opposites.S01.WEBDL.1080p.NewStudio\Solar.Opposites.S01E08.WEBDL.1080p.NS.RGzsRutracker.mkv
=============================================================================================================================================================
Plex scan end: 2021-02-25 17:36:03,358000

Please create Plex Token file, update ASS and rescan:

Plex Media Server/X-Plex-Token.id containing the Plex token id by loggin on plex.tv/desktop, then https://plex.tv/devices.xml, it will be MediaContainer/Device ... token="xxxxxxxxxxxxxx"

IvanShift commented 3 years ago

Unfortunately, there are no changes ( root-volume4-Video-sorted-HD Cartoon Series.filelist.log root-volume4-Video-sorted-HD Cartoon Series.scanner.log Solar Opposites.filelist.log Solar Opposites.scanner.log

Where exactly should I put file with token? I put it here:

IvanShift commented 3 years ago

This is Plex Media Scanner.log with some errors: Plex Media Scanner.log

ZeroQI commented 3 years ago

The2nd character in "Кит Stupid Show" cause a crash, i do not know how to solve beyond commenting the offending line 559

Library: 'no valid X-Plex-Token.id', did you put the plex token in the file?

Please create a library file for me to fully reproduce your library, so i do not need to ask you to test every change https://github.com/ZeroQI/Absolute-Series-Scanner/wiki/Batch-file-to-create-filelist.txt-and-dummy-library-from-it

IvanShift commented 3 years ago

Hello! You should correct this: chcp 1252\>nul to this: chcp 1252>nul

As for me code-page 1252 is not correct. I'm using 1251. I also made this file with UTF-8 code-page: filelist_1251.txt filelist_65001 (UTF8).txt filelist_1252.txt

Few comments:

"Futurama" uses xACVxx as as episode number (https://en.wikipedia.org/wiki/List_of_Futurama_episodes#Episodes). It's Production Code. But don't mind it. This is only this series with this numbering.
Local shows in the folder "2x2" use Russian characters, which is probably why the scanner crashes.

Library: 'no valid X-Plex-Token.id', did you put the plex token in the file?

Yes, I did it: Maybe I should place this file to another folder?

IvanShift commented 3 years ago

The2nd character in "Кит Stupid Show" cause a crash, i do not know how to solve beyond commenting the offending line 559

Here's how to fix this error: https://stackoverflow.com/questions/21129020/how-to-fix-unicodedecodeerror-ascii-codec-cant-decode-byte https://stackoverflow.com/questions/10288016/usage-of-unicode-and-encode-functions-in-python

You can also use error-processing modes of encoding functions: https://pyneng.readthedocs.io/en/latest/book/16_unicode/errors.html

ZeroQI commented 3 years ago

Aetf published a fix that works well with your library

I have seen Futurama had "Volume I" numbering and coded to consider that a season folder and have it working BUT the episode numbering [1ACV01, ...] is specific to futurama so ends up all as s01e501+

I never saw production code before, and too random in length and specific per company to use as proper numbering so cannot fix that i am afraid: https://en.wikipedia.org/wiki/Production_code_number#:~:text=A%20production%20code%20number%2C%20also,episodes%20within%20a%20television%20series.

Anything left?

IvanShift commented 3 years ago

Thank! You can close this issue successfully! Everything is fine now! May be you can add detection of Futurama (only this pattern xACVxx)? But it not necessary because this is only one show with such numbering.

IvanShift commented 3 years ago

Hello! I'm testing it on Anime folder and see some skipping again( Juuni Kokuki.filelist.log Juuni Kokuki.scanner.log Scanner does not detect any video files.

This file list was created inside folder "Juuni Kokuki": filelist_1251.txt

Could you check it?

ZeroQI commented 3 years ago

Best practices

Series folder in library root
Weird characters in the library name (now it works)
Do not create a grouping folder (series folder not in library root) unless needed, as they are scanned every time unlike series folder at root level (unless only one file in grouping folder but why is the point of doing that?) as they are a HACK
Avoid putting a single folder in a grouping folder, algoryth have issues if subfolders without videos are present
Do not leave random folders (audio, subs) if possible, if it cause issues, add them to IGNORE_DIRS_RX_RAW

Library: "Аниме-сериалы" > grouping folder "Juuni Kokuki" > Series folder "Juuni Kokuki (The Twelve Kingdoms, Двенадцать королевств) [TV 45, 2002][BDRip][MC]" > "Subs" + "Audio" subdirs

The scanner reached the folder and believed it to be a grouping folder so leave for the root level scan to add files, but library root =scan saw it wasn't and was left un-scanned...

I just pushed a new version, you have to do the following as folder scans not added at root level are cached even when they crash:

update
move from root folder the series
rescan
clear bundles
put back folder
rescan
confirm it is working

So you need to do that even after the code is fixed IF the issue is not in a grouping folder with multiple series folder indide directly (only one series folder so we have to), making scanner development a pain in the neck, especially since i didn't know in which iteration it failed, and it took 6 hours to fix, so please consider donating

IvanShift commented 3 years ago

Hello! From the beginning it was only series folder (series folder in library root), without grouping folder. But scanner works very bad on one-season folder (some crashes and ignore folders exist using yesterday version). So I created group folders today morning for every anime even it has only one season and situation became better. With last version all group folders are scanned. Thanks!

Should I try to put some one-season anime folders to root directory and scan it again for testing purpose?

ZeroQI commented 3 years ago

To be honest single folder at root is more stable normally but unusual code pages gave various issues in the library name, series folder name and also episode filenames...

Also, scanners do cache results so if the series folder is at root level, it will not rescan it unless it changed but if it crash you need to do the plex dance and that's really annoying when you are looking for an issue and need to test multiple things

ZeroQI / Absolute-Series-Scanner

Wrong directory skipping #276