search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.52k stars 283 forks source link

Main driver error while updating bug bucket #804

Closed Oleggg2000 closed 10 months ago

Oleggg2000 commented 10 months ago

Description

I'm trying to ran Restler test mode which i'm pretty sure should run request one by one despite the fact that they fail or not like smoke testing right? In fact i get KeyError: '4fd09c71930ec37115c070d09c733ff17d015673_main_driver_500\r\nServer:'

Steps to reproduce

I'm currentry modify grammar.py for my testing purpose that i can fuzz "restler_multipart_formdata" using fix from #341 and a little modify multipart_formdata.py by comment out lines 61-64 to let myself include fuzzinable dict in multipart.

So the first of request from **grammar.py**:
# Endpoint: /api2.3/wss/sign, method: Post
request = requests.Request([
    primitives.restler_static_string("POST "),
    primitives.restler_basepath(""),
    primitives.restler_static_string("/"),
    primitives.restler_static_string("api2.3"),
    primitives.restler_static_string("/"),
    primitives.restler_static_string("wss"),
    primitives.restler_static_string("/"),
    primitives.restler_static_string("sign"),
    primitives.restler_static_string(" HTTP/1.1\r\n"),
    primitives.restler_static_string("Accept: application/json\r\n"),
    primitives.restler_static_string("Host: 172.168.100.20:9000\r\n"),
    primitives.restler_refreshable_authentication_token("authentication_token_tag"),
    primitives.restler_multipart_formdata("txt_file"),
    primitives.restler_static_string("--_CUSTOM_BOUNDARY_\r\n"),
    primitives.restler_static_string('Content-Disposition: form-data; name="data"\r\n\r\n'),
    primitives.restler_static_string("{"),
    primitives.restler_static_string(""""id_pki_user":"""),
    primitives.restler_fuzzable_int("1"),
    primitives.restler_static_string(""","description":"""),
    primitives.restler_fuzzable_string("fuzzstring"),
    primitives.restler_static_string(""","signatures":"""),
    primitives.restler_static_string("{"),
    primitives.restler_static_string(""""id_cert":"""),
    primitives.restler_fuzzable_int("1"),
    primitives.restler_static_string(""","reference_uri":"""),
    primitives.restler_fuzzable_string("fuzzstring"),
    primitives.restler_static_string(""","actor_or_role":"""),
    primitives.restler_fuzzable_string("fuzzstring"),
    primitives.restler_static_string(""","security_token_id":"""),
    primitives.restler_fuzzable_string("fuzzstring"),
    primitives.restler_static_string("}"),
    primitives.restler_static_string(""","soap_version":"""),
    primitives.restler_fuzzable_int("1"),
    primitives.restler_static_string("}"),
    primitives.restler_static_string("\r\n--_CUSTOM_BOUNDARY_--\r\n"),
    primitives.restler_static_string("\r\n"),

],
requestId="/api2.3/wss/sign"
)

Expected results

Smok test all requests regardless their return status

Actual results

So i'm ok with the response from server 514 error, but i can't go further to add that error. By the way is there any chance to add some "available" error codes for server reply?

Output of file *network.testing.txt**:

Generation-1: Rendering Sequence-1

  Request: 1 (Remaining candidate combinations: 2)
  Request hash: 1694de0a16d4d619ae2858f21da6688df1e6b505

      - restler_static_string: 'POST '
      - restler_static_string: ''
      - restler_static_string: '/'
      - restler_static_string: 'api2.3'
      - restler_static_string: '/'
      - restler_static_string: 'wss'
      - restler_static_string: '/'
      - restler_static_string: 'sign'
      - restler_static_string: ' HTTP/1.1\r\n'
      - restler_static_string: 'Accept: application/json\r\n'
      - restler_static_string: 'Host: 172.168.100.20:9000\r\n'
      + restler_refreshable_authentication_token: [token_auth_method, token_refresh_interval, ...]
      - restler_multipart_formdata: '_OMITTED_BINARY_DATA_'
      - restler_static_string: '--_CUSTOM_BOUNDARY_\r\n'
      - restler_static_string: 'Content-Disposition: form-data; name="data"\r\n\r\n'
      - restler_static_string: '{'
      - restler_static_string: '"id_pki_user":'
      - restler_fuzzable_int: '1'
      - restler_static_string: ',"description":'
      - restler_fuzzable_string: '"fuzzstring"'
      - restler_static_string: ',"signatures":'
      - restler_static_string: '{'
      - restler_static_string: '"id_cert":'
      - restler_fuzzable_int: '1'
      - restler_static_string: ',"reference_uri":'
      - restler_fuzzable_string: '"fuzzstring"'
      - restler_static_string: ',"actor_or_role":'
      - restler_fuzzable_string: '"fuzzstring"'
      - restler_static_string: ',"security_token_id":'
      - restler_fuzzable_string: '"fuzzstring"'
      - restler_static_string: '}'
      - restler_static_string: ',"soap_version":'
      - restler_fuzzable_int: '1'
      - restler_static_string: '}'
      - restler_static_string: '\r\n--_CUSTOM_BOUNDARY_--\r\n'
      - restler_static_string: '\r\n'

2023-08-17 09:38:56.411: Sending: 'POST /api2.3/wss/sign HTTP/1.1\r\nAccept: application/json\r\nHost: 172.168.100.20:9000\r\n_OMITTED_AUTHTOKEN\r\nContent-Type: multipart/form-data; boundary=_OMITTED_BINARYDATA--\r\n\r\n'

2023-08-17 09:38:56.428: Received: 'HTTP/1.1 500\r\nServer: nginx/1.22.1\r\nDate: Thu, 17 Aug 2023 02:40:25 GMT\r\nContent-Type: application/json; charset=UTF-8\r\nContent-Length: 190\r\nConnection: keep-alive\r\n\r\n{"code":514,"message":"Для данной роли недоступно выполнение операций с подтверждением за другого пользователя"}\n'

2023-08-17 09:38:56.451: Attempting to reproduce bug... 2023-08-17 09:38:56.469: Sending: 'POST /api2.3/wss/sign HTTP/1.1\r\nAccept: application/json\r\nHost: 172.168.100.20:9000\r\n_OMITTED_AUTHTOKEN\r\nContent-Type: multipart/form-data; boundary=_OMITTED_BINARYDATA--\r\n\r\n'

2023-08-17 09:38:56.488: Received: 'HTTP/1.1 500\r\nServer: nginx/1.22.1\r\nDate: Thu, 17 Aug 2023 02:40:25 GMT\r\nContent-Type: application/json; charset=UTF-8\r\nContent-Length: 190\r\nConnection: keep-alive\r\n\r\n{"code":514,"message":"Для данной роли недоступно выполнение операций с подтверждением за другого пользователя"}\n'

2023-08-17 09:38:56.500: Done replaying sequence.

main.txt: Random seed: 12345 Setting fuzzing schemes: directed-smoke-test 2023-08-17 09:38:56.161: Going to fuzz a set with 8 requests 2023-08-17 09:38:56.161: Request-0: Value Combinations: 2 2023-08-17 09:38:56.161: Request-1: Value Combinations: 2 2023-08-17 09:38:56.161: Request-2: Value Combinations: 2 2023-08-17 09:38:56.161: Request-3: Value Combinations: 2 2023-08-17 09:38:56.161: Request-4: Value Combinations: 2 2023-08-17 09:38:56.161: Request-5: Value Combinations: 2 2023-08-17 09:38:56.161: Request-6: Value Combinations: 2 2023-08-17 09:38:56.161: Request-7: Value Combinations: 2 2023-08-17 09:38:56.161: Avg. Value Combinations per Request: 2 2023-08-17 09:38:56.161: Median Value Combinations per Request: 2.0 2023-08-17 09:38:56.161: Min Value Combinations per Request: 2 2023-08-17 09:38:56.161: Max Value Combinations per Request: 2 2023-08-17 09:38:56.161: Total dependencies: 0

2023-08-17 09:38:56.166: Generation: 1 / Sequences Collection Size: 8 (After directed-smoke-test Extend)

Rendering request 0 from scratch

Failed to write bug bucket log: [Errno 22] Invalid argument: 'D:\PKIS_Tests\restler-fuzzer\restler_working_dir\Test\RestlerResults\experiment29708\bug_buckets\main_driver_500\r\nServer:_1.replay.txt' 2023-08-17 09:38:56.530: Final Swagger spec coverage: 0 / 8 2023-08-17 09:38:56.530: Rendered requests: 0 / 8 2023-08-17 09:38:56.530: Rendered requests with "valid" status codes: 0 / 0 2023-08-17 09:38:56.530: Num fully valid requests (no resource creation failures): 0 2023-08-17 09:38:56.530: Num requests not rendered due to invalid sequence re-renders: 0 2023-08-17 09:38:56.530: Num invalid requests caused by failed resource creations: 0 2023-08-17 09:38:56.530: Total Creations of Dyn Objects: 0 2023-08-17 09:38:56.530: Total Requests Sent: {'gc': 0, 'main_driver': 0} 2023-08-17 09:38:56.530: Bug Buckets: {'main_driver_500\r\nServer:': 1}

EngineStdOut.txt 2023-08-17 10:19:37.563: Initializing: Garbage collection every 30 seconds. 2023-08-17 10:19:37.568: Generation: 1 2023-08-17 10:19:37.910: Terminating garbage collection. Waiting for max 300 seconds. Traceback (most recent call last): File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\engine\core\fuzzer.py", line 43, in run self._num_total_sequences = driver.generate_sequences( File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\engine\core\driver.py", line 698, in generate_sequences seq_collection = render_with_cache(seq_collection, fuzzing_pool, checkers, File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\engine\core\driver.py", line 446, in render_with_cache valid_renderings = render_one(prefix_seq_to_render, sequences_count, checkers, generation, global_lock, garbage_collector) File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\engine\core\driver.py", line 210, in render_one renderings = current_seq.render(candidate_values_pool, global_lock) File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\engine\core\sequences.py", line 627, in render BugBuckets.Instance().update_bug_buckets( File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\engine\bug_bucketing.py", line 235, in update_bug_buckets logger.update_bug_buckets(self._bug_buckets, sent_request_data_list, bug_hash, additional_log_str=additional_log_str) File "D:\PKIS_Tests\restler-fuzzer\restler_bin\engine\utils\logger.py", line 805, in update_bug_buckets print(f"Hash: {Bugs_Logged[bucket_hash].bug_hash}", file=log_file) KeyError: '4fd09c71930ec37115c070d09c733ff17d015673_main_driver_500\r\nServer:'

If u need more info pls let me know. Appreciate for future reply!

Environment details

OC: Windows, Python 3.10.7, dotnet 6.0.407, RESTler version: 9.2.2

Oleggg2000 commented 10 months ago

In search of reason i've got a bug with status_code attribute of response incstance (HttpResponse class). Beside code it has a piece of response. That's why BugBuckets.Instance().update_bug_buckets() fails. It expects an plaine error code, but gets '500 Server:' image

Oleggg2000 commented 10 months ago

The problem is that server sends back kind of incorrect response. HttpResponse class assumes to get first raw like HTTP/1.1 200 OK\r\n with some sub status like OK, Bad Request or Not Modified HTTP/1.1 500 Server: nginx/1.22.1 Date: Thu, 17 Aug 2023 04:34:30 GMT Content-Type: application/json; charset=UTF-8 Content-Length: 190 Connection: keep-alive

{"code":514,"message":"Для данной роли недоступно выполнение операций с подтверждением за другого пользователя"}