RESTler
What is RESTler?
RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services. For a given cloud service with an OpenAPI (formerly Swagger) specification, RESTler analyzes its entire specification, and then generates and executes tests that exercise the service through its REST API.
RESTler intelligently infers producer-consumer dependencies among request types from the OpenAPI definition. During testing, it checks for specific classes of bugs and dynamically learns how the service behaves from prior service responses. This intelligence allows RESTler to explore deeper service states reachable only through specific request sequences and to find more bugs.
RESTler is described in these peer-reviewed research papers:
- RESTler: Stateful REST API Fuzzing (ICSE'2019)
- Checking Security Properties of Cloud Service REST APIs (ICST'2020)
- Differential Regression Testing for REST APIs (ISSTA'2020)
- Intelligent REST API Data Fuzzing (FSE'2020)
If you use RESTler in your research, please cite the (default) ICSE'2019 paper (BibTeX).
RESTler includes multiple test generation strategies. In order to get a comprehensive comparative view w.r.t. to (i) efficiency
(i.e., how quickly can RESTler find crashes) and (ii) effectiveness
(i.e., how many crashes can RESTler find in a give time frame),
we recommend comparing against all documented fuzzing_mode(s)
because each one provides a different trade-off between breadth and depth of state space exploration.
We also recommend running test mode before any fuzzing, as described below,
to discover and fix setup issues (e.g. adding required pre-requisite parameter values to the dictionary) prior to fuzzing.
RESTler was created at Microsoft Research and is still under active development.
For an overview and demo on how to get started, see Webinar - Fuzzing to Improve the Security and Reliability of Cloud Services.
Build Status
Setting up RESTler
RESTler was designed to run on 64-bit machines with Windows or Linux. Experimental support for macOS is also enabled.
Build instructions
Docker
In the root of this repo, run
docker build -t restler .The resulting docker container will have RESTler available in directory /RESTler/restler with main binary Restler.
You can then use this docker image as basis to add the application under test to execute fuzzing inside isolated docker containers.
Local
Prerequisites: Install Python 3.8.2 and .NET 6.0, for your appropriate OS.
Create a directory where you'd like to place the RESTler binaries:
mkdir restler_bin
Switch to the repo root directory and run the following Python script:
python ./build-restler.py --dest_dir <full path to restler_bin above>
Note: if you get nuget error NU1403 when building, a quick workaround is to clear your cache with this command
dotnet nuget locals all --clear
Using RESTler
RESTler runs in 4 main modes (in order):
- Compile: from an OpenAPI JSON or YAML definition (and optionally examples), generate a RESTler grammar. See Compiling.
- Test: execute quickly all of the endpoints+methods in a compiled RESTler grammar for debugging the test setup and compute what parts of the OpenAPI definition are covered. This mode is also called a smoketest. See Testing. To use custom test engine settings, see Test Engine Settings.
- Fuzz-lean: execute once every endpoint+method in a compiled RESTler grammar with a default set of checkers to see if bugs can be found quickly. See Fuzzing.
- Fuzz: bug hunting - explore a RESTler fuzzing grammar in smart breadth-first-search mode (deeper search mode) for finding more bugs. Warning: This type of fuzzing is more aggressive and may create outages in the service under test if the service is poorly implemented (e.g., fuzzing might create resource leaks, perf degradation, backend corruptions, etc.). See Fuzzing.
Quick Start
For a quick intro with simple examples, see this Tutorial.
To quickly try RESTler on your API, see Quick Start.
Bugs found by RESTler
There are currently two categories of bugs found by RESTler.
- Error code: currently, any time a response with status code
500("Internal Server Error") is received, a bug is reported. - Checkers: each checker tries to trigger specific bugs by executing targeted additional requests or sequences of requests at certain points during fuzzing, determined by context. Some checkers try to find additional 500s, while other checkers try to find specific logic bugs such as resource leaks or hierarchy violations. For a full description of checkers, see Checkers.
When a bug is found, RESTler reports bugs triaged in bug buckets, and provides a replay log that can be used to reproduce the bug (see Replay).
Advanced Topics
For tips on using RESTler effectively, please see Best Practices and Improving API Coverage.
See also these Frequently Asked Questions.
If you're interested in using RESTler at scale as part of your CI/CD pipeline, check out the REST API Fuzz Testing self-hosted service.
Questions
If you have a request/suggestion/question, please file an issue. See Contributing.md for instructions.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
For more information, see Contributing.md.
Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
Data collection
The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
For more information, see Telemetry.md.
Reporting Security Issues
Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) at secure@microsoft.com. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the Security TechCenter.
For additional details, see Security.md.