Leetcode API not working for some users

[Abhijeet-AR]

Throwing "Internal Server Error" for users uwi and superluminal.

[saurabh-prakash]

@Abhijeet-AR the leetcode details are essentially web-scraping that doesn't work well in the long-run. Web scraping is good for long one-time scripts/jobs. I'd recommend rewriting it and making it API based so that the code doesn't break when LeetCode UI changes.

Let me know what you think. The API based approach require 3 headers to be sent

1. csrf-token

2. cookie containing csrf-token
3. referer

These can be get via selenium visit.

[saurabh-prakash]

Sample curl (pastable in Postman)

curl --location --request POST 'https://leetcode.com/graphql' \
--header 'referer: https://leetcode.com/uwi/' \
--header 'cookie: csrftoken=072x9rHWNg1kkNXwiJ3BH2RNd8sKxZ5jbByD8upwAX2sq9sLHxyLko43D4yq4UE7; __cfduid=d200def86257d1bc29ba00c6c8b3ad20c1610345109; csrftoken=0HAcMpAuL6Fte9Af18y98z2T3gvV84PHTy673AOkl3q8mGebGIOHHgLwaUzBo11x' \
--header 'Content-Type: application/json' \
--data-raw '{
    "operationName": "getUserProfile",
    "variables": {
        "username": "saurabhprakash"
    },
    "query": "query getUserProfile($username: String!) {  allQuestionsCount {    difficulty    count  }  matchedUser(username: $username) {    contributions {    points      questionCount      testcaseCount    }    profile {    reputation      ranking    }    submitStats {      acSubmissionNum {        difficulty        count        submissions      }      totalSubmissionNum {        difficulty        count        submissions      }    }  }}"
}'

EDIT: Removed x-csrf-token header EDIT 2: Minimalistic query to get all the details we currently fetch in a single graphql query

[Abhijeet-AR]

I didn't know there is an API for Leetcode. Using API could make it easier to maintain. But I still can't find any documentation for it 🤔. How did you manage to get the query format? (I'm amateur😄).

[saurabh-prakash]

There might not be a public-facing documentation for the LeetCode APIs, but we can still use the 3-4 APIs that the UI sends (that already has the valid graphql queries) [if we cannot form an exact single query for getting the details that we need]

[Abhijeet-AR]

I think it's better we create a separate branch for LeetCode and work on it.

[saurabh-prakash]

@Abhijeet-AR Here's a PR for the same. Let me know if something is not clear and/or you want to learn anything else.