search

theMK2k / Media-Hoarder

Media Hoarder - THE media frontend for data hoarders and movie lovers
https://media.hoarder.software
Other
254 stars 4 forks source link

Issue: Not scraping anything at all unless imdb in file name or if linked. #66

Closed kolbdog323 closed 4 months ago

kolbdog323 commented 4 months ago

I tired removing the items rescann then reqadd and rescan and no scraping and imdb scrape check doesnt even show any errors.

kolbdog323 commented 4 months ago

Only scrapes if have imdbid in name just year wont work anymore.

kolbdog323 commented 4 months ago

It wont scrape any movies or shows it does this in the delper preview error only shows the first one or two words of the title if i try to rescan it example tried scraping The Mummy (1999) and in the developer tool view it showed it only searched the first word " The" then no matches found.

kolbdog323 commented 4 months ago

But it works if i have the imdib in file name or if i directly link to it otherwise it wont find it and i dont want to put the imdbid in every show or movie i want to scrape.

kolbdog323 commented 4 months ago

If its already scaned the tvshow the episodes will still scrape it just wont scrape anyhtjhing new movies or shows unless it has the imdbid or if use the link function.

kolbdog323 commented 4 months ago

Any comment about this issue?

theMK2k commented 4 months ago

I'm off today, will investigate as soon as I return. How does the scraper check look like for you right now?

kolbdog323 commented 4 months ago

Working no errors except for series. But titles will only scrape if have the imdbid in the file name or folder name or if i link to it otherwise wont scrape if only have year.

kolbdog323 commented 4 months ago

Episodes will scrape if already added the show just wont scrape new movies or tvshows with only the year or years just with imdbid.

kolbdog323 commented 4 months ago

Its seems by deleting the unistall registry in regedit and reinstalling media hoarder fixed it for the windows version. Not sure about portable version yet.

kolbdog323 commented 4 months ago

Never mind the issue is back.

kolbdog323 commented 4 months ago

It just scans and adds it to library but no actaul imdb scraping.

kolbdog323 commented 4 months ago

Example just tried scraping Forged in Fire. I tried Forged in Fire, and Forged in Fire(2015-) Didnt scrape but once i added the Forged in Fire (2015-) [tt4680444] It will scrape. same with any movie or tv show have to put in the ttid othrwise won't scrape.

theMK2k commented 4 months ago

Thanks.

The IMDB Scraping process has two phases:

1/ Detection

2/ Scraping

During the detection phase, Media Hoarder tries to find the correct IMDB entry to your media. Of course, if the filename / directory name contains the tt this task is trivial and Media Hoarder doesn't have to do anything but assume the tt is correct and move on with phase 2.

If the filename / directory name has no tt*, Media Hoarder tries to find the correct IMDB entry by running a search on the IMDB site with some of the words of the filename / directory name (the stuff you also saw in the logs of the developer tools).

I highly suspect that currently something doesn't work with IMDB search during phase 1. It also looks like Media Hoarder currently does not provide enough info when this fails alltogether.

I will investigate this and let you know.

kolbdog323 commented 4 months ago

Thanks not sure why will only scrape if have tt* in folder name or file name now it was working a couple days ago normally.

kolbdog323 commented 4 months ago

It will scrape as long as the tt is in the name even for special episodes. But if a show is already added it will scrape all episodes normally with no tt needed for any new episodes. But the movie and tv shows all require tt now in the name otherwise won't scrape otherwise i have to link the show or movie once it is added if don't put tt in name. Won't scrape if just the year or no year is in the folder name for tv shows or just year for movie all require tt*.

theMK2k commented 4 months ago

Oh yes, that's right.

For a TV show episode it's a bit different:

Scraping a TV show episode's data relies on the fact that the IMDB entry is already correctly detected for the show (not the episode), then Media Hoarder looks up the episodes list of the show and matches that against the season and episode number provided in the episode's filename.

theMK2k commented 4 months ago

getting closer...

I just created a Avengers - Endgame [2019].mkv and can reproduce what you reported: the detection of the IMDB entry fails (with an error):

image

kolbdog323 commented 4 months ago

That is exactly what i got in the developer tools just not in the offical imdb scaping checker shows no erros.

getting closer...

I just created a Avengers - Endgame [2019].mkv and can reproduce what you reported: the detection of the IMDB entry fails (with an error):

image

kolbdog323 commented 4 months ago

But mine went futher then that it showed this for the error as well: imdb-scraper.js:2036 [scrapeIMDBFindPageSearchV3] ERROR: Error: GraphQL Error: PersistedQueryNotFound at h (imdb-scraper.js:116) at x (imdb-scraper.js:1988) at processTicksAndRejections (internal/process/task_queues.js:93) at async c (find-imdb-tconst.js:145) at async Ee (store.js:2899) at async De (store.js:2625) at async ye (store.js:2561) at async X (store.js:423)

theMK2k commented 4 months ago

Good, the culprit has been found, IMDB indeed have updated the "find" function (and Media Hoarder uses this to detect the IMDB entries for a file/directory without tt* in its name).

So I had to provide an update for that, which I just did. Please restart Media Hoarder and the update will be fetched automatically.

Please re-test and let me know how it went.

I will create a new beta release later that includes the "find" function in the Scraper checks, too.

kolbdog323 commented 4 months ago

Does that work for the portable version resetting it as well? Since i reset it and it is working again not sure if that will work for the portable version as well.

theMK2k commented 4 months ago

yes, every version (stable, beta, portable or installed) fetches the latest IMDB function definitions

kolbdog323 commented 4 months ago

It will take some time to rescrape all of the movies and shows I have since I had to delete Media Hoarder because I thought the version I had was the problem. Thank you for fixing it again.

theMK2k commented 4 months ago

Thank you for helping in making Media Hoarder better :)

kolbdog323 commented 4 months ago

No problem. Even though I'm not a programmer I'm glad I can help in anyway I can even though some of the suggestions I make are not doable.

theMK2k commented 4 months ago

As promised, an updated beta v1.4.0-b4 is released: https://github.com/theMK2k/Media-Hoarder/releases/tag/v1.4.0-b

I'll close this issue now, please re-open if there are still problems.