javinizer / Javinizer

(NSFW) Organize your local Japanese Adult Video (JAV) library
MIT License
606 stars 63 forks source link

Recheck parsers #38

Closed fluxtheory closed 4 years ago

fluxtheory commented 4 years ago

Hey Boss,

I ran Javinizer over my video collection and unfortunately it only managed to sort half of the JAV. After I enabled verbose output and all of the parsers enabled in the settings it turns out they were being skipped due to non-matches. However if I were to go visit R18 or JAVlibrary manually I can see this is not the case. Please have a look into this.

log.txt

jvlflame commented 4 years ago

First of all, make sure that the Python cloudscraper module is updated.

pip install cloudscraper -U

There was a cloudflare update on that recently broke functionality from scraping from javlibrary. I have also found that occasionally the user token generated will be invalid and you'll have to restart your shell and run Javinizer again for javlibrary scraping to be functional again.

For cases like AOZ-249 not being found on R18, if you look at the actual movie ID it's AOZ-249z. The automatic file cleaning that Javinizer performs doesn't account for that, so it'll try to clean it to AOZ-249 which won't match the movie ID. For cases such as these, you'll need to rename the file and use the -Strict parameter to sort them.

fluxtheory commented 4 years ago

my cloudscraper currently is up to date (v1.2.34)

fluxtheory commented 4 years ago

Just updated to cloudscraper v1.2.36, can confirm issue is still recurring.

[2020-05-05T10:24:59][Javinizer] (3 of 1413) Sorting [AMBS-056A.mp4] VERBOSE: [2020-05-05T10:25:03][Get-R18DataObject] Search [AMBS-056] not matched on R18 VERBOSE: [2020-05-05T10:25:06][Get-R18DataObject] Search [AMBS-056] not matched on R18 VERBOSE: [2020-05-05T10:25:10][Get-DmmDataObject] Search [AMBS-056] not matched on Dmm VERBOSE: [2020-05-05T10:25:10][Get-JavlibraryUrl] Search [AMBS-056] not matched on JAVLibrary VERBOSE: [2020-05-05T10:25:10][Get-JavlibraryUrl] Search [AMBS-056] not matched on JAVLibrary VERBOSE: [2020-05-05T10:25:10][Get-JavlibraryUrl] Search [AMBS-056] not matched on JAVLibrary WARNING: [2020-05-05T10:25:10][Test-RequiredMetadata] Skipped: [AMBS-056A.mp4] 7 null fields: [coverurl, genre, id, maker, title, releasedate, releaseyear]

jvlflame commented 4 years ago

It still seems as though cloudscraper is experiencing issues, though it does work for me intermittently.

If you run Javinizer -Find AMBS-056 multiple times, does it fail every time?

fluxtheory commented 4 years ago

Seems like it.

PS C:\Users\Bailey> Javinizer -Find AMBS-056
[2020-05-06T12:27:03][Javinizer] Function started
VERBOSE: [2020-05-06T12:27:08][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:27:13][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:27:17][Get-DmmDataObject] Search [AMBS-056] not matched
on Dmm
VERBOSE: [2020-05-06T12:27:17][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
VERBOSE: [2020-05-06T12:27:17][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
VERBOSE: [2020-05-06T12:27:17][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
[2020-05-06T12:27:17][Javinizer] Function ended

PS C:\Users\Bailey> Javinizer -Find AMBS-056
[2020-05-06T12:27:20][Javinizer] Function started
VERBOSE: [2020-05-06T12:27:24][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:27:29][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:27:33][Get-DmmDataObject] Search [AMBS-056] not matched
on Dmm
VERBOSE: [2020-05-06T12:27:33][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
VERBOSE: [2020-05-06T12:27:33][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
VERBOSE: [2020-05-06T12:27:33][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary

[2020-05-06T12:27:33][Javinizer] Function ended
PS C:\Users\Bailey> Javinizer -Find AMBS-056
[2020-05-06T12:27:36][Javinizer] Function started
VERBOSE: [2020-05-06T12:27:41][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:27:45][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:27:50][Get-DmmDataObject] Search [AMBS-056] not matched
on Dmm
VERBOSE: [2020-05-06T12:27:50][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
VERBOSE: [2020-05-06T12:27:50][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
VERBOSE: [2020-05-06T12:27:50][Get-JavlibraryUrl] Search [AMBS-056] not matched
on JAVLibrary
[2020-05-06T12:27:50][Javinizer] Function ended
fluxtheory commented 4 years ago

Okay, I think we're getting somewhere. Check out the output:

PS C:\Users\Bailey> Javinizer -Find AMBS-056 -R18 -DMM -Aggregated
[2020-05-06T12:36:46][Javinizer] Function started
VERBOSE: [2020-05-06T12:36:50][Get-R18DataObject] Search [AMBS-056] not matched
on R18
VERBOSE: [2020-05-06T12:36:54][Get-DmmDataObject] Search [AMBS-056] not matched
on Dmm

Search          : AMBS-056
Id              :
Title           :
AlternateTitle  :
Description     :
ReleaseDate     :
ReleaseYear     :
Runtime         :
Director        :
Maker           :
Label           :
Series          :
Rating          :
RatingCount     :
Actress         : {}
Genre           :
ActressThumbUrl :
CoverUrl        :
ScreenshotUrl   :
TrailerUrl      :
DisplayName     : []
FolderName      :  [] -  ()
FileName        :
PS C:\Users\Bailey> Javinizer -Find 'https://www.r18.com/videos/vod/movies/detai
l/-/id=h_237ambs00056/'
[2020-05-06T12:37:16][Javinizer] Function started

Url             : https://www.r18.com/videos/vod/movies/detail/-/id=h_237ambs00
                  056/
Id              : AMBS-056
ContentId       : h_237ambs00056
Title           : She's A Sex Slut Because Papa's A Pervert! Compulsory
                  Creampie Sex With 8 Tiny Girls
Description     :
Date            : 2019-11-01
Year            : 2019
Runtime         : 241
Director        :
Maker           : Planet Plus
Label           : Anbibaresotsu
Series          :
Actress         : {Shuna Kagami, Shuri Atomi, Ku Aine, Yukari Miyazawa.}
Genre           : {Beautiful Girl, Small Tits, Stepfamily, Creampie.}
ActressThumbUrl : {https://pics.r18.com/mono/actjpgs/kagami_syuna.jpg,
                  https://pics.r18.com/mono/actjpgs/atomi_syuri.jpg,
                  https://pics.r18.com/mono/actjpgs/aine_kuu.jpg,
                  https://pics.r18.com/mono/actjpgs/miyazawa_yukari.jpg.}
CoverUrl        : https://pics.r18.com/digital/video/h_237ambs00056/h_237ambs00
                  056pl.jpg
ScreenshotUrl   : {https://pics.r18.com/digital/video/h_237ambs00056/h_237ambs0
                  0056jp-1.jpg, https://pics.r18.com/digital/video/h_237ambs000
                  56/h_237ambs00056jp-2.jpg, https://pics.r18.com/digital/video
                  /h_237ambs00056/h_237ambs00056jp-3.jpg, https://pics.r18.com/
                  digital/video/h_237ambs00056/h_237ambs00056jp-4.jpg.}
TrailerUrl      : {https://awscc3001.r18.com/litevideo/freepv/h/h_2/h_237ambs00
                  056/h_237ambs00056_sm_w.mp4, https://awscc3001.r18.com/litevi
                  deo/freepv/h/h_2/h_237ambs00056/h_237ambs00056_dm_w.mp4, http
                  s://awscc3001.r18.com/litevideo/freepv/h/h_2/h_237ambs00056/h
                  _237ambs00056_dmb_w.mp4}

[2020-05-06T12:37:19][Javinizer] Function ended

So it doesnt appear to be a parsing issue, but a URL issue where it cant seem to locate the right address given the jav identifier.

fluxtheory commented 4 years ago

Problem appears to be resolved in version 1.6.0.

Thanks again boss.