NLP2REST

NLP2REST is a tool designed to extract REST API rules from natural language descriptions, thereby enhancing the efficiency and capabilities of REST API testing tools. This work has been published at ISSTA 2023, "Enhancing REST API Testing with NLP Techniques".

Bibtex Citation Here

Quick Start

If you have a Swagger or an OpenAPI Specification, you can quickly use our tool.

First, download the following pretrained models and download requirements:

wget --no-check-certificate 'https://drive.google.com/uc?export=download&id=1-jawBqo3c3eMRkXF8Y73oLEFNSOphbpF' -O rest_model
wget --no-check-certificate 'https://docs.google.com/uc?export=download&confirm=any_non_empty_string_here&id=1j1XA1dufDgqSkIGlQn97-WeKElaL8708' -O rest_model.wv.vectors_ngrams.npy

cd rule_extractor
pip3 install -r requirements.txt
python3 -m nltk.downloader stopwords
python3 -m nltk.downloader wordnet

To extract rules using a given specification, execute the following command in the rule_extractor directory:

python nlp2rest.py --extract_rules --spec_path ../specifications/swagger/fdic.yaml --settings ./settings2.yaml --model_name rest_model

The extracted rules are saved to a file named found_rules.json in the project root directory.

Detailed Instructions

This section contains the components necessary to run NLP2REST, an approach designed to automatically enhance OpenAPI specifications with rules extracted from natural language description fields. These rules include constraints and example values.

In addition, this section provides all the components required to replicate the experiment described in our paper, including testing tools and benchmark APIs.

Environment

This project has been tested and is known to work well on the following setup:

Ubuntu 20.04
Google Cloud EC2 machine with 24-core CPU and 128GB memory

Setup

We provide a setup script to install the necessary packages and set up the environment for the project:

./setup.sh

It will take around 10 minutes. If you are not using Ubuntu 20.04, please check the required packages in setup.sh and install them manually.

Steps to Use NLP2REST

1. REST API(s)

NLP2REST is designed to be applied to one or more target REST APIs. The APIs should be accessible at the URLs detailed in their OpenAPI specifications. We have provided several REST APIs in the services directory along with a manuscript for running these services. Alternatively, you can use any public APIs accessible online that have OpenAPI specifications. For a quick trial of our approach, we suggest using the FDIC REST API. This is an online API and its specification is available in our specifications directory.

2. Deployment of the Rule Extractor

This is a Python-based service accessible via a REST API. It's designed to extract formal OpenAPI rules from natural language descriptions. To run the Rule Extractor, please adhere to the instructions provided in the rule_extractor directory. In brief, you will need to:

Obtain a model either by training one yourself or by downloading the pre-trained model from the Google Drive link supplied in the README.
Build and run the Python application. We provide a Dockerfile for rapid deployment of the service. Alternatively, you can follow the instructions to install the service directly on your system.
The service is designed to run on localhost on port 4000, i.e., [http://localhost:4000/]().

3. Run the Rule Validator

This is a bespoke strategy built upon the RestTestGen framework, which is available in the rule_validator directory. The strategy parses the OpenAPI specification of the chosen API, uses the Rule Extractor to draw out rules from natural language descriptions, and interacts with the target API to validate the extracted rules. To employ our strategy within RestTestGen, you need to configure RestTestGen to run the strategy named NlpStrategy. For guidance, refer to the official RestTestGen README file available in the rule_validator directory or in the official RestTestGen repository. The results of RestTestGen will be stored in an output directory. This directory will hold the enhanced OpenAPI specification and reports of the HTTP interactions generated by the Rule Validator. Please note that RestTestGen expects the Rule Extractor to be running at http://localhost:4000/. If the Rule Extractor is running on a different port or host, update the baseUrl field in the RuleExtractorProxy class of RestTestGen. Currently, we found some bugs in the generated specifications. While fixing the bugs, please use validator.py. You can use it using this command:

python3 validator.py {Json OpenAPI Specification location}

It will generate validation-result.json which is a report of after validation.

You have successfully generated an enhanced OpenAPI specification and/or extracted rule report. Using these rules significantly boost the performance of REST API testing tools. To utilize these tools, please check the instructions in the tools directory.

4. Validate the Result

For each service, we have documented the identified keywords and corresponding values we discovered in this Google sheets. To validate our findings, you can cross-check if the enhanced specification includes the corresponding keyword and value entries present in the Google sheet.

codingsoo / nlp2rest

readme