search

pavan412kalyan / imdb-movie-scraper

33 stars 14 forks source link

Getting the next_page element fails as IMDB changed its markup #16

Open bogdanl opened 3 months ago

bogdanl commented 3 months ago

Hi, I wanted to try the download_complete_data_scrape_with_pagination_by_listURL script but I get a 'NoneType' object is not subscriptable Invalid-URL error as it seems that now IMDB has a "load 250 more" link, rather than a next page element to paginate through the results of a page like https://www.imdb.com/search/title/?title_type=feature&languages=te&count=250&view=advanced

Just checking if there's a workaround or is this a new change that your code hasn't accounted for yet?

Basically I'm looking for a more recent dataset with all the movies, including the ones from this year.

pavan412kalyan commented 3 months ago

Hey Hi, There is a new change in the IMDB UI how pagination works. I should make some changes to fix.

Thanks for trying out.

bogdanl commented 1 month ago

@pavan412kalyan Hi! Were you able to reproduce this issue?