tboothman / imdbphp

PHP library for retrieving film and tv information from IMDb
247 stars 84 forks source link

Use GraphQL to fetch paginated keywords #310

Open jetrosuni opened 1 year ago

jetrosuni commented 1 year ago

Fixes #308

Replaces legacy scraping logic with working GraphQL query.

jetrosuni commented 1 year ago

Note: The failing tests are not related to this PR. testKeywords_all works.

duck7000 commented 1 year ago

I came up with this solution, i don't really know which is better? i think that graphQlGetAll() not intended is to use it like you did but i can be wrong i guess?

    public function keyword()
    {
        if (empty($this->all_keywords)) {
            $query = <<<EOF
query Keywords(\$id: ID!) {
  title(id: \$id) {
    keywords(first: 9999) {
      edges {
        node {
          keyword {
            text {
              text
            }
          }
        }
      }
    }
  }
}
EOF;

            $data = $this->graphql->query($query, "Keywords", ["id" => "tt$this->imdbID"]);
            foreach ($data->title->keywords->edges as $edge) {
                $this->all_keywords[] = $edge->node->keyword->text->text;
            }
        }
        return $this->all_keywords;
    }
jetrosuni commented 1 year ago

Yeah, I guess most movies on IMDb have less than 9999 keywords, so your proposed solution might well be enough :) With graphQlGetAll()the logic will continue to fetch more keywords if the limit of 9999 results would ever get hit. I'm okay with either solution – @tboothman can make the call :)

duck7000 commented 9 months ago

I have hit a 250 items limit from imdb GraphQL API on cast. I changed my method to use the paginated function to get all cast members. So your solution might be better as it gets all pages, although i did not hit this limit with keywords so imdb does not have that 250 item limit on all methods i guess.

So far only cast and episodes seems to have that limit

jetrosuni commented 5 months ago

Any issues with this PR, or can it be merged? Happy to help and fix if issues are pointed out!

duck7000 commented 5 months ago

Well i don't think that any of the maintainers take your PR serious as @jreklund just released a new version with a fixed keywords method but not with GraphQL, unbelievable! And i even did offer him to use my work to fix imdbphp..

jetrosuni commented 5 months ago

I'm okay with the maintainers doing the best they see fit and just happy that the library is not completely abandoned. I've now based this PR on top of the latest release. Hopefully one day it can be merged :)

jcvignoli commented 5 months ago

Well i don't think that any of the maintainers take your PR serious as @jreklund just released a new version with a fixed keywords method but not with GraphQL, unbelievable! And i even did offer him to use my work to fix imdbphp..

Please calm down and stop complaining. You started your fork, which is fine, so please show some respect to the orginal makers of IMDbPHP. Thanks.

duck7000 commented 5 months ago

I'm not complaining at all, i just think that those fixes don't fix anything, and what does get fixed (in this PR) the right way is ignored. I almost feel sorry for the maker of this PR as his work is fine but is ignored. I mean it is al here why don't use it? This has nothing to do with respect, i do respect jreklund! I just don't understand it.

But okay fine, good luck with it.