IMDb List - 100 Movie Limit

[tlardner]

Is this limited to the first 100 movies on an IMDb list? For example, I am searching this list, and it is matching 21 movies, and prompting to add 79 to Radarr. Is there a way to get around this limit?

[ArnoutVerbeken]

I tend to agree. It seems it is only capable of grabbing the first 100 movies (so page1) of a list that has more items. I hope you can fix this. The other scripts that you refer to in your readme can grab more, but always require input, while yours can be automated which I like a lot.

Thanks for this very helpful script!

[bcthomas1977]

Same when adding to a collection based on genre. Processes the first 100 only

[spekta-23]

This fixes the issue for everything, sorry I am new so just posting here for anyone.

In imdb_tools.py at line 22 it should be changed to the below (initiate title_ids, define a range, I have just set to 10 pages as it will except out after that, minor change to the request.get URL to include page number and then extend the empty title_ids instead). I have tested this up to about 480 items on a list. I was also able to verify it added 200+ to radarr without issue.

title_ids = []

for i in range(1,10):
  try:
    r = requests.get(imdb_url + '?page={}'.format(i), headers={'Accept-Language': library_language})
  except requests.exceptions.MissingSchema:
    return
  tree = html.fromstring(r.content)
  title_ids.extend(tree.xpath("//div[contains(@class, 'lister-item-image')]"
      "//a/img//@data-tconst"))
if title_ids:

[ExDominatus]

Hi @spekta-23, I tried your fix but I keep getting Syntax errors.

Traceback (most recent call last): File "plex_auto_collections.py", line 8, in import image_server File "/home/user/Plex-Auto-Collections-master/image_server.py", line 4, in from config_tools import ImageServer File "/home/user/Plex-Auto-Collections-master/config_tools.py", line 12, in from plex_tools import get_actor_rkey File "/home/user/Plex-Auto-Collections-master/plex_tools.py", line 6, in import imdb_tools File "/home/user/Plex-Auto-Collections-master/imdb_tools.py", line 22 title_ids = [] ^ SyntaxError: invalid syntax

[pairofcrocs]

Hi @spekta-23, I tried your fix but I keep getting Syntax errors.

Traceback (most recent call last): File "plex_auto_collections.py", line 8, in import image_server File "/home/user/Plex-Auto-Collections-master/image_server.py", line 4, in from config_tools import ImageServer File "/home/user/Plex-Auto-Collections-master/config_tools.py", line 12, in from plex_tools import get_actor_rkey File "/home/user/Plex-Auto-Collections-master/plex_tools.py", line 6, in import imdb_tools File "/home/user/Plex-Auto-Collections-master/imdb_tools.py", line 22 title_ids = [] ^ SyntaxError: invalid syntax

Getting the same thing. Did you find a fix for this?

[pairofcrocs]

This fixes the issue for everything, sorry I am new so just posting here for anyone.

In imdb_tools.py at line 22 it should be changed to the below (initiate title_ids, define a range, I have just set to 10 pages as it will except out after that, minor change to the request.get URL to include page number and then extend the empty title_ids instead). I have tested this up to about 480 items on a list. I was also able to verify it added 200+ to radarr without issue.

title_ids = []

for i in range(1,10):
  try:
    r = requests.get(imdb_url + '?page={}'.format(i), headers={'Accept-Language': library_language})
  except requests.exceptions.MissingSchema:
    return
  tree = html.fromstring(r.content)
  title_ids.extend(tree.xpath("//div[contains(@class, 'lister-item-image')]"
      "//a/img//@data-tconst"))
if title_ids:

Can you post you entire imdb_tools.py?

[spekta-23]

import re import requests from lxml import html from tmdbv3api import TMDb from tmdbv3api import Movie from tmdbv3api import Collection from tmdbv3api import Person import config_tools

def imdb_get_movies(config_path, plex, data): tmdb = TMDb() movie = Movie() tmdb.api_key = config_tools.TMDB(config_path).apikey imdb_url = data if imdb_url[-1:] == " ": imdb_url = imdb_url[:-1] imdb_map = {} library_language = plex.Library.language title_ids = []

for i in range(1,10):
  try:
    r = requests.get(imdb_url + '?page={}'.format(i), headers={'Accept-Language': library_language})
  except requests.exceptions.MissingSchema:
    return
  tree = html.fromstring(r.content)
  title_ids.extend(tree.xpath("//div[contains(@class, 'lister-item-image')]"
      "//a/img//@data-tconst"))
if title_ids:
    for m in plex.Library.all():
        if 'themoviedb://' in m.guid:
            if not tmdb.api_key == "None":
                tmdb_id = m.guid.split('themoviedb://')[1].split('?')[0]
                tmdbapi = movie.details(tmdb_id)
                imdb_id = tmdbapi.imdb_id
            else:
                imdb_id = None
        elif 'imdb://' in m.guid:
            imdb_id = m.guid.split('imdb://')[1].split('?')[0]
        else:
            imdb_id = None

        if imdb_id and imdb_id in title_ids:
            imdb_map[imdb_id] = m
        else:
            imdb_map[m.ratingKey] = m

    matched_imbd_movies = []
    missing_imdb_movies = []
    for imdb_id in title_ids:
        movie = imdb_map.pop(imdb_id, None)
        if movie:
            matched_imbd_movies.append(plex.Server.fetchItem(movie.ratingKey))
        else:
            missing_imdb_movies.append(imdb_id)

    return matched_imbd_movies, missing_imdb_movies

def tmdb_get_movies(config_path, plex, data): try: tmdb_id = re.search('.*?(\d+)', data) tmdb_id = tmdb_id.group(1) except AttributeError: # Bad URL Provided return

t_movie = Movie()
tmdb = Collection()
tmdb.api_key = config_tools.TMDB(config_path).apikey  # Set TMDb api key for Collection
if tmdb.api_key == "None":
    raise KeyError("Invalid TMDb API Key")
t_movie.api_key = tmdb.api_key  # Copy same api key to Movie
t_col = tmdb.details(tmdb_id)
t_movs = []
for tmovie in t_col.parts:
    t_movs.append(tmovie['id'])

# Create dictionary of movies and their guid
# GUIDs reference from which source Plex has pulled the metadata
p_m_map = {}
p_movies = plex.Library.all()
for m in p_movies:
    guid = m.guid
    if "themoviedb://" in guid:
        guid = guid.split('themoviedb://')[1].split('?')[0]
    elif "imdb://" in guid:
        guid = guid.split('imdb://')[1].split('?')[0]
    else:
        guid = "None"
    p_m_map[m] = guid

matched = []
missing = []
# We want to search for a match first to limit TMDb API calls
# Too many rapid calls can cause a momentary block
# If needed in future maybe add a delay after x calls to let the limit reset
for mid in t_movs:  # For each TMBd ID in TMBd Collection
    match = False
    for m in p_m_map:  # For each movie in Plex
        if "tt" not in p_m_map[m] is not "None":  # If the Plex movie's guid does not start with tt
            if int(p_m_map[m]) == int(mid):
                match = True
                break
    if not match:
        imdb_id = t_movie.details(mid).entries['imdb_id']
        for m in p_m_map:
            if "tt" in p_m_map[m]:
                if p_m_map[m] == imdb_id:
                    match = True
                    break
    if match:
        matched.append(m)
    else:
        missing.append(t_movie.details(mid).entries['imdb_id'])

return matched, missing

def tmdb_get_summary(config_path, data, type): collection = Collection() person = Person() collection.api_key = config_tools.TMDB(config_path).apikey person.api_key = collection.api_key collection.language = config_tools.TMDB(config_path).language person.language = collection.language

if type == "overview":
    return collection.details(data).overview
elif type == "biography":
    return person.details(data).biography

[spekta-23]

Let me know if that takes care of it for you, dont forget to use Python3.6 I had issues with that myself

[pairofcrocs]

Let me know if that takes care of it for you, dont forget to use Python3.6 I had issues with that myself

Would you be willing to upload your imdb_tools.py file? The formatting didn't translate over github.

Thank you!!!

[jolbol1]

Had a similar problem and managed to get @spekta-23 code to work. Just a note though I do use MZA's fork, but I doubt this script is very different. I uploaded the imd_tools.py here

[meisnate12]

I know this is late but this issue has been fixed in the fork mza921/Plex-Auto-Collections as well as many other improvements and bug fixes