search

nikitastupin / clairvoyance

Obtain GraphQL API schema even if the introspection is disabled
Apache License 2.0
1.02k stars 92 forks source link

KeyError: 'Query' #22

Open nikitastupin opened 3 years ago

nikitastupin commented 3 years ago

I'm also having the same error when running the tool, however changing the bucket size to 256 didn't solve the issue for me either. OS: Ubuntu 20.04 LTS Python: 3.8.10

Here is the output when using the bucket size to 256

python3 -m clairvoyance -o /home/nade/Desktop/schema.json -w google-10000-english-usa.txt https://www.example.com/graphql -vv
[DEBUG][2021-08-21 11:44:25 oracle.py:419]  Root typenames are: {'queryType': None, 'mutationType': None, 'subscriptionType': None}
[DEBUG][2021-08-21 11:44:25 oracle.py:441]  __typename = Query
[DEBUG][2021-08-21 11:44:27 oracle.py:81]   Sent 256 fields, recieved 256 errors in 1.947997 seconds
[DEBUG][2021-08-21 11:44:28 oracle.py:81]   Sent 256 fields, recieved 255 errors in 0.825902 seconds
[DEBUG][2021-08-21 11:44:29 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.431477 seconds
[DEBUG][2021-08-21 11:44:29 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.526348 seconds
[DEBUG][2021-08-21 11:44:30 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.5381 seconds
[DEBUG][2021-08-21 11:44:30 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.683985 seconds
[DEBUG][2021-08-21 11:44:45 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.369622 seconds
[DEBUG][2021-08-21 11:44:45 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.513586 seconds
[DEBUG][2021-08-21 11:44:46 oracle.py:81]   Sent 256 fields, recieved 256 errors in 0.468517 seconds
[DEBUG][2021-08-21 11:44:46 oracle.py:81]   Sent 16 fields, recieved 16 errors in 0.18961 seconds
[DEBUG][2021-08-21 11:44:46 oracle.py:444]  Query.fields = {'_', 'getVariant', 'getCategoryIds', 'getVariants', 'getMemberships', 'getProduct', 'getProductIds', 'getCategory', 'calculateTax', 'version'}
[WARNING][2021-08-21 11:44:46 oracle.py:302]    Unknown error message: '_ disabled'
[DEBUG][2021-08-21 11:44:46 oracle.py:462]  Skip probe_args() for '_' of type 'Boolean'
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nade/GraphQL pentest/clairvoyance/clairvoyance/__main__.py", line 96, in <module>
    schema = oracle.clairvoyance(
  File "/home/nade/GraphQL pentest/clairvoyance/clairvoyance/oracle.py", line 466, in clairvoyance
    schema.types[typename].fields.append(field)
KeyError: 'Query'

And here is the output when running on default ammount of bucket size (4096)

[DEBUG][2021-08-21 11:46:39 oracle.py:419]  Root typenames are: {'queryType': None, 'mutationType': None, 'subscriptionType': None}
[DEBUG][2021-08-21 11:46:52 oracle.py:441]  __typename = Query
[DEBUG][2021-08-21 11:46:59 oracle.py:81]   Sent 4096 fields, recieved 4095 errors in 6.233555 seconds
[DEBUG][2021-08-21 11:47:06 oracle.py:81]   Sent 4096 fields, recieved 4096 errors in 6.304185 seconds
[DEBUG][2021-08-21 11:47:10 oracle.py:81]   Sent 1808 fields, recieved 1808 errors in 3.456586 seconds
[DEBUG][2021-08-21 11:47:10 oracle.py:444]  Query.fields = {'getVariants', 'getVariant', 'getMemberships', 'calculateTax', 'version', 'getProductIds', 'getCategory', '_', 'getCategoryIds', 'getProduct'}
[WARNING][2021-08-21 11:47:17 oracle.py:194]    Unknown error message: Unknown argument "facilities" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:17 oracle.py:194]    Unknown error message: There can be only one argument named "color".
[WARNING][2021-08-21 11:47:17 oracle.py:194]    Unknown error message: There can be only one argument named "favorite".
[WARNING][2021-08-21 11:47:17 oracle.py:194]    Unknown error message: Unknown argument "variables" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:17 oracle.py:194]    Unknown error message: There can be only one argument named "labor".
[WARNING][2021-08-21 11:47:17 oracle.py:194]    Unknown error message: There can be only one argument named "favorites".
[WARNING][2021-08-21 11:47:26 oracle.py:194]    Unknown error message: Unknown argument "nationwide" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:26 oracle.py:194]    Unknown error message: Unknown argument "variation" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:26 oracle.py:194]    Unknown error message: Unknown argument "variations" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:26 oracle.py:194]    Unknown error message: Unknown argument "validation" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:26 oracle.py:194]    Unknown error message: Unknown argument "warranties" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:26 oracle.py:194]    Unknown error message: There can be only one argument named "harbor".
[WARNING][2021-08-21 11:47:29 oracle.py:194]    Unknown error message: Unknown argument "guarantees" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:29 oracle.py:194]    Unknown error message: Unknown argument "vacancies" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:29 oracle.py:194]    Unknown error message: Unknown argument "variance" on field "Query.getVariants". Did you mean "variantIds"?
[WARNING][2021-08-21 11:47:29 oracle.py:194]    Unknown error message: Unknown argument "varieties" on field "Query.getVariants". Did you mean "variantIds"?
[DEBUG][2021-08-21 11:47:29 oracle.py:452]  Query.getVariants.args = set()
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/nade/GraphQL pentest/clairvoyance/clairvoyance/__main__.py", line 96, in <module>
    schema = oracle.clairvoyance(
  File "/home/nade/GraphQL pentest/clairvoyance/clairvoyance/oracle.py", line 466, in clairvoyance
    schema.types[typename].fields.append(field)
KeyError: 'Query'

Edit: Tried it also in a fresh installed Kali Linux (python 3.9.2) and it also gets the same error

Originally posted by @kleiton0x00 in https://github.com/nikitastupin/clairvoyance/issues/16#issuecomment-903091682

kleiton0x00 commented 3 years ago

Well I don't know what to say anymore, tried the fix-issues-16-and-20 however no difference was made.

nikitastupin commented 3 years ago

Unfortunately, fix-issues-16-and-20 fixes another issue but it's good to know that you've tested it for your case.

If you'll provide the exact endpoint you're running clairvoyance against it would greatly facilitate debugging 😃

kleiton0x00 commented 3 years ago

Sure, im have been testing the tool on this website, and also here (which has introspection enabled)

slowmistio commented 3 years ago

error:

python3 -m clairvoyance -vv -o ../opensea-schema.json -w /Users/x/google-10000-english/google-10000-english-usa-no-swears-short.txt https://api.xx.io/graphql/ Traceback (most recent call last): File "/usr/local/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/local/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/Users/x/GraphQL-clairvoyance/clairvoyance/main.py", line 96, in schema = oracle.clairvoyance( File "/Users/x/GraphQL-clairvoyance/clairvoyance/oracle.py", line 435, in clairvoyance root_typenames = fetch_root_typenames(config) File "/Users/x/GraphQL-clairvoyance/clairvoyance/oracle.py", line 418, in fetch_root_typenames data = response.json().get("data", {}) File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 910, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/Cellar/python@3.9/3.9.1_6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Sim4n6 commented 2 years ago

I'm debugging the problem with KhanAcademy Graphql.

It seems that fetch_root_typenames() is causing the problem. The HTTP POST request to the GraphQL endpoint using this query query { __typename } returns a 400 Operation not permitted. status code. Thus, the line 418 of oracle.py returns data equals an empty dictionary. which means Root typenames are: {'queryType': None, 'mutationType': None, 'subscriptionType': None} which in turns means :

I hope I helped a little bit ...

barrett092 commented 2 years ago

Getting same error.

barrett092 commented 2 years ago

Commented out 473 and its working now..

nikitastupin commented 2 years ago

Hi 👋

@Sim4n6, thanks for debugging! https://www.khanacademy.org/api/internal/graphql just blocks fetching typename for root types (same goes for https://www.beautypie.com/graphql mentioned earlier but https://anilist.co/graphql works fine for me). However, we can obtain typename for root types using probe_typename function. I started fixing the issue in #30, hopefully this will help.

@barrett092, I'm not sure that the program will give you any meaningful output if you'll comment out the line.

Anyways, it seems that it's better to rewrite the codebase as discussed in #26 because now it requires too much time to understand the very fragile code to fix even a small issue.