dawoudt / JustWatchAPI

Python 3 JustWatch.com API - https://justwatch.com
MIT License
324 stars 44 forks source link

This is no longer working due to Just watch API changes #66

Open voodooenglishman opened 1 year ago

voodooenglishman commented 1 year ago

I believe this no longer works could be wrong as I can't find an older version of the justwatch API documentation but I believe its changed since the last update to this

Tried running one of my usual scripts and was given errors down to this API in my code

Was getting 404 errors

https://apis.justwatch.com/docs/api/

Looking at the current API and the code from this it seems the URLs are no longer pointing correctly

Example being the new base URL which is "https://apis.justwatch.com/contentpartner/v2/content" which is different from the one given within this API

Thank you

NRaabjerg commented 1 year ago

I am experiencing the same issue. I believe JustWatch have closed public API access. I will try to look further into it, but hopefully someone will be able to get around it at some point.

dseomn commented 1 year ago

I'm getting 404s trying to access content and search pages, but https://apis.justwatch.com/content/locales/state and https://apis.justwatch.com/content/providers/locale/en_US are still working for me. Over the past few days I also noticed missing seasons fields in some TV show pages, which all got fixed eventually. Maybe JustWatch is just doing some sort of migration and it's not going smoothly for this API?

But if the current API is going away, https://github.com/lufinkey/node-justwatch-api/issues/17 mentions a graphql API that might work instead?

byte4geek commented 1 year ago

Same for me

anemochore commented 1 year ago

Same here. I searched and found reddit post and then I was able to get it work again. (changed whole payload)

DiegoFleitas commented 1 year ago

Idem, GraphQL might be the only viable option now

dseomn commented 1 year ago

In case it helps anyone, here's some code I got working using the GraphQL API. For what it's worth, the new API feels significantly more powerful than the old API in some ways, though it also wasn't a trivial migration to the new API.

https://github.com/dseomn/rock-paper-sand/blob/70bf3542f65b6a3ca92f0784e64bcc29476a393d/rock_paper_sand/justwatch.py#L64

anemochore commented 1 year ago

Well, I suspect if my TamperMonkey script will help, but this is my code. (Somehow reverse engineered with help of reddit post previously mentioned) My scenario is searching by title and then getting original title, year, type, imdb rating, etc. https://github.com/anemochore/imdbOnWatcha/blob/master/utils.js#L80

dseomn commented 1 year ago

Edit: sorry for the noise, this was an issue with my code, not with the API. I was caching all "successful" HTTP requests, including ones that return JSON with an errors key. So it looked like the API kept returning the error below, but it actually just returned it once and my code cached that.

I just started getting errors like the below from the graphql API. Note that the HTTP response code was 200 (or some other non-error code), and the mention of 429 is in the response body. Is anybody else seeing something like this with the new API?

{'errors': [{'message': 'rpc error: code = Unknown desc = getting offers: GetTitleOffers: reading from source: error response received from Elasticsearch: 429 Too Many Requests', 'path': ['urlV2', 'node', 'seasons', 0, 'episodes', 5, 'offers'], 'extensions': {'code': 'INTERNAL_ERROR'}}, {'message': 'rpc error: code = Unknown desc = getting offers: GetTitleOffers: reading from source: error response received from Elasticsearch: 429 Too Many Requests', 'path': ['urlV2', 'node', 'seasons', 1, 'episodes', 9, 'offers'], 'extensions': {'code': 'INTERNAL_ERROR'}}], 'data': None}
anemochore commented 1 year ago

@dseomn Error message says 'Too Many Requests.' Maybe you are requesting too many. In my case, no such errors occured.

dseomn commented 1 year ago

See the "Edit: ..." text I added to that comment. It was just a transient issue, probably from too many requests to their backend, not too many requests from me. The fix was easy: https://github.com/dseomn/rock-paper-sand/commit/692e63c1a2772d6354b4a0f69bafd457b2635110

DiegoFleitas commented 1 year ago

Thanks for sharing that demo, @dseomn! By the way, for anyone else tinkering with this, I think I encountered a quirk regarding the newline characters used in GraphQL queries: there's a different behavior when using \r\n versus \n. So, if you come across any strange behavior, checking the newline characters might be a good place to start. I believe the API may have cached an error response for a seemingly correct request. The errors ceased when I switched to using \r\n as the newline character in my queries.

DanielSouzaBertoldi commented 1 year ago

After reading the code of both projects listed here and the reddit post, I'm still not sure on how to fetch info of a media by passing along only its ID + objectType. Checking out the /graphql requests in the JustWatch site I can only find operations for GetTitlePopularityRank and GetSimilarTitles 🤔 has anyone figured out what's the query for this?

dseomn commented 1 year ago

https://github.com/dseomn/rock-paper-sand/blob/9a26853a0c7a5c68a536b30e69e2cc16c5e4f572/rock_paper_sand/justwatch.py#L233 looks up details by node ID, which is basically just object type + object ID. E.g., object type "SHOW" and object ID "42" is node ID "ts42". I think movies are "tm...", seasons are "tss..." and episodes are "tse...", but I didn't double check that.

DanielSouzaBertoldi commented 1 year ago

@dseomn nice, thanks for the quick reply! Instead of just asking for the offers data I also added the following to the Movie fragment just to test it out:

fragment Movie on Movie {
    __typename
    id
    content(country: $country, language: $language) {
        fullPath
        title
        originalReleaseYear
        originalTitle
        scoring {
        imdbScore
        imdbVotes
        __typename
        }
        externalIds {
        imdbId
        __typename
        }
        __typename
    }
    offers(country: $country, platform: WEB) {
        monetizationType
        availableToTime
        availableFromTime
        package {
            clearName
            technicalName
        }
    }
}

And it worked, nice. The worst part is figuring out what field name to use for a given data.

naxels commented 12 months ago

For people catching up, the GraphQL API endpoint is: https://apis.justwatch.com/graphql

bagajohny commented 11 months ago

Can I use this to export my watchlist? if yes how do I do it?

DiegoFleitas commented 11 months ago

Can I use this to export my watchlist? if yes how do I do it?

It's likely you can do it using the graphql API directly but I think you'll have to invest your own time to get somewhere, I can give you a head start for what's worth

query

query GetTitleListV2($country: Country!, $titleListFilter: TitleFilter, $titleListSortBy: TitleListSortingV2! = LAST_ADDED, $titleListType: TitleListTypeV2!, $titleListAfterCursor: String, $watchNowFilter: WatchNowOfferFilter!, $first: Int! = 10, $language: Language!, $sortRandomSeed: Int! = 0, $profile: PosterProfile, $backdropProfile: BackdropProfile, $format: ImageFormat, $platform: Platform! = WEB, $includeOffers: Boolean = false) {
  titleListV2(
    after: $titleListAfterCursor
    country: $country
    filter: $titleListFilter
    sortBy: $titleListSortBy
    first: $first
    titleListType: $titleListType
    sortRandomSeed: $sortRandomSeed
  ) {
    totalCount
    pageInfo {
      startCursor
      endCursor
      hasPreviousPage
      hasNextPage
      __typename
    }
    edges {
      ...WatchlistTitleGraphql
      __typename
    }
    __typename
  }
}

fragment WatchlistTitleGraphql on TitleListEdgeV2 {
  cursor
  node {
    id
    objectId
    objectType
    offerCount(country: $country, platform: $platform)
    offers(country: $country, platform: $platform) @include(if: $includeOffers) {
      id
      presentationType
      monetizationType
      retailPrice(language: $language)
      type
      package {
        id
        packageId
        clearName
        __typename
      }
      standardWebURL
      elementCount
      deeplinkRoku: deeplinkURL(platform: ROKU_OS)
      __typename
    }
    content(country: $country, language: $language) {
      title
      fullPath
      originalReleaseYear
      shortDescription
      scoring {
        imdbScore
        imdbVotes
        tmdbScore
        tmdbPopularity
        __typename
      }
      posterUrl(profile: $profile, format: $format)
      backdrops(profile: $backdropProfile, format: $format) {
        backdropUrl
        __typename
      }
      upcomingReleases(releaseTypes: [DIGITAL]) {
        releaseDate
        __typename
      }
      isReleased
      __typename
    }
    likelistEntry {
      createdAt
      __typename
    }
    dislikelistEntry {
      createdAt
      __typename
    }
    watchlistEntryV2 {
      createdAt
      __typename
    }
    customlistEntries {
      createdAt
      __typename
    }
    watchNowOffer(country: $country, platform: $platform, filter: $watchNowFilter) {
      id
      standardWebURL
      package {
        id
        packageId
        clearName
        __typename
      }
      retailPrice(language: $language)
      retailPriceValue
      currency
      lastChangeRetailPriceValue
      presentationType
      monetizationType
      availableTo
      __typename
    }
    ... on Movie {
      seenlistEntry {
        createdAt
        __typename
      }
      __typename
    }
    ... on Show {
      tvShowTrackingEntry {
        createdAt
        __typename
      }
      seenState(country: $country) {
        seenEpisodeCount
        releasedEpisodeCount
        progress
        caughtUp
        lastSeenEpisodeNumber
        lastSeenSeasonNumber
        __typename
      }
      __typename
    }
    __typename
  }
  __typename
}

vars

{
  "titleListSortBy": "LAST_ADDED",
  "first": 20,
  "sortRandomSeed": 0,
  "platform": "WEB",
  "includeOffers": false,
  "titleListFilter": {
    "ageCertifications": [],
    "excludeGenres": [],
    "excludeProductionCountries": [],
    "objectTypes": [
      "SHOW"
    ],
    "productionCountries": [],
    "genres": [],
    "packages": [],
    "excludeIrrelevantTitles": false,
    "presentationTypes": [],
    "monetizationTypes": []
  },
  "watchNowFilter": {
    "packages": [],
    "monetizationTypes": []
  },
  "language": "es",
  "country": "ES",
  "titleListType": "WATCHLIST",
  "titleListAfterCursor": ""
}

Btw for this kind of query it's likely you'll require to put your own Bearer token along with the headers which I don't think most of queries we've been using need. Also note objectType can be used to get SHOW / MOVIE on the response & the vars on mine are set to spanish

puneetverma24 commented 11 months ago

Please share the complete schema information.

bagajohny commented 11 months ago

@DiegoFleitas
I am trying the method given in the following github repo but I am getting some error. Can you help me understand it?

https://github.com/ijanos/justwatchdumper/issues/1

This is the link to the issue I created. The method given in this repo seems easier to implement for me. So I will be grateful if you can help me.

DiegoFleitas commented 11 months ago

Please share the complete schema information.

I don't have access to it & I don't think anyone else in this thread does since JustWatch has introspection disabled for their graphql schema, AFAIK everyone just inspected the requests being sent from JustWatch website 🤷‍♂️

@DiegoFleitas I am trying the method given in the following github repo but I am getting some error. Can you help me understand it?

ijanos/justwatchdumper#1

This is the link to the issue I created. The method given in this repo seems easier to implement for me. So I will be grateful if you can help me.

Sorry, you are on your own with that. Plus that discussion is no longer relevant to this repo. You are welcome to ask here and if you explain what you are trying to do with enough details I might be able to help or provide a simpler solution

BenutzerEinsZweiDrei commented 9 months ago

Just use https://github.com/Electronic-Mango/simple-justwatch-python-api instead