Can no longer fetch all exercises because response is too large

[jb-1980]

when trying to access https://www.khanacademy.org/api/v1/exercises the response is HTTP response was too large: 34717350. The limit is: 33554432.

While that is the official endpoint for exercises, what I really want is a list of all the names and titles of every exercise available. It would be nice if there were a graphql endpoint for this.

[csilvers]

We're thinking about the future of our public API, and we'll keep the graphql suggestion in mind!

I don't know if you're passing an accept-encoding header to compress the response, but maybe that would help? I don't know offhand where that limit is being enforced.

[jb-1980]

Thanks for the quick response.

I just tried the link in Google Chrome, which passes the header accept-encoding: gzip, deflate, br.

I was able to recurse through the topictree endpoint with the following javascript (if someone else is interested):

const parseExerciseData = data => {
  if (Array.isArray(data)) {
    return data.reduce((acc, child) => {
      if (child.children) {
        return { ...acc, ...parseExerciseData(child.children) }
      } else if (child.kind === "Exercise") {
        return {
          ...acc,
          [child.id]: {
            name: child.name,
            title: child.title
          }
        }
      }
    }, {})
  }

  if (data.children) {
    return parseExerciseData(data.children)
  }

  if (data.kind === "Exercise") {
    return {
      [data.id]: {
        name: data.name,
        title: data.title
      }
    }
  }
}

const fetchExercises = cb => {
  let exercises = localStorage.getItem("khan-academy-exercises")
  if (exercises === null) {
    const url =
      "https://jgilgen.pythonanywhere.com/api/v1/topictree?kind=Exercise"
    fetch(url)
      .then(res => res.json())
      .then(json => {
        exercises = Object.values(parseExerciseData(json))
        localStorage.setItem(
          "khan-academy-exercises",
          JSON.stringify(exercises)
        )
        cb(exercises)
      })
      .catch(err => {
        console.error(err)
      })
  } else {
    cb(JSON.parse(exercises))
  }
}

fetchExercises(exercises => console.log(exercises))

I had to set up a proxy server for the topictree endpoint. Any update on #139?

[csilvers]

I'm glad you found a working solution. I'm afraid there's not been any update on #139 yet. I am no CORS expert but the folks here who are know about the issue.

[jb-1980]

I am going to close this. While the issue still remains, I believe the topic tree workaround is sufficient. If all the meta data that was produced by the /api/v1/exercises endpoint is needed, it is still accessible by /api/v1/exercises/{exercise_name} endpoint.