search

microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.52k stars 283 forks source link

RESTler not handling OpenAPI Spec with Chinese Character Example Data #808

Open tpardee opened 9 months ago

tpardee commented 9 months ago

Description

We have an OpenAPI spec with some example data in Chinese characters. RESTler will accept this data in the compile step but will give an error when trying to test or fuzz against it.

I've fixed this issue locally and can do a pull request if you agree this is a bug.

Steps to reproduce

  1. Have an openAPI spec that has some example data in Chinese. example paths: /xxx/xxx: post: requestBody: content: application/json: schema: type: object example: Name: 公司

  2. Do a RESTler compile with that OpenAPI spec

  3. Do a RESTler test with the generated output

Expected results

Expect to get normal test output report

Actual results

Standard out will have ERROR: Restler engine failed. See logs in xxxx directory for more information.

EngineStdOut.txt log will have the below message

Failed to process grammar file: c:\src\restler-fuzzer\tscbc_fuzzing\Compile\grammar.json; 'charmap' codec can't decode byte 0x81 in position 3577: character maps to

Environment details

Windows 11, Python 3.11.1, .Net 7.0.102, RESTler 9.2.2

marina-p commented 9 months ago

Hello @tpardee,

Yes, this is a bug. Please send the PR with your local fix.

Thank you!

Marina

tpardee commented 8 months ago

The legal process within my company is pretty cumbersome for me to contribute and this is fixed with a one line code change. Is it possible to report the change rather than pull request it?

Change is at restler.py line 525 currently code is with open(grammar_path, 'r') as grammar:

fixed code is with open(grammar_path, 'r', encoding='utf-8') as grammar:

if it still needs to be done through pull request please let me know.

marina-p commented 8 months ago

Looks good, will merge this change.

Thanks,

Marina