andygello555 / sttp

A HTTP client that is also a scripting language/test framework. Can be used as an external framework for testing Web APIs as well as a tool for scraping static web pages.
GNU General Public License v3.0
2 stars 0 forks source link

sttp

Abstract

The idea for creating a simple scripting language for the development and testing of Web APIs came from my use, reliance and creation of such APIs in my work life as well as in my spare time. I’ve often found the available tools for such development (such as Postman or Insomnia) quite limited when it comes to control-flow before or after requesting a resource from an API. Thus, the idea of a scripting language for this very purpose came about. The language will include variable declaration/definition, control-flow (if and for statements), function definitions, short builtin functions for every HTTP method, and JSON manipulation via json-path. The language will also be dynamically typed with values being stored as JSON parsable strings. My hopes are that this will make working with JSON Web APIs (the standard when it comes to popular Web APIs) easier and more intuitive. This means that the only supported “types” will be strings, integers, floats, JSON objects and lists.

Source code

Libraries/Code used

How to use

You can either run the sttp interpreter from within the repository, or by creating an executable that can be run later. You can provide the interpreter with the following inputs:

Prerequisites

You must have go 1.17 (or later) installed.

Running from within the repository

go run . [ FILE | DIRECTORY | INPUT ]

Building an executable and running it

go build -o sttp
./sttp [ FILE | DIRECTORY | INPUT ]

Examples

Examples for sttp can be found within the _examples/ directory. Each example is stored within its own directory holding the following files:

Some of these examples utilise the echo-chamber web API. The web server for which is located within: _examples/echo_chamber/ directory. The usage of which can be found here.

Running tests

The Go tests for sttp can be run using the following command when inside the src/ directory:

go test ./...

You might need to increase the limit for the number of open sockets on your system (via ulimit or Windows alternative). This is because Go is known to run some test cases in parallel in order to speed up test execution. In conjunction to this, there are also a lot of tests that manage interactions between the echo chamber web API, and batched HTTP requests made via sttp.

Debugging info

Debugging info can be enabled by appending the -v flag to the command mentioned above. This will lead to a lot of output when running tests, so it might be beneficial to only run specific *_test.go files/test cases by using the -run PATTERN option.

Benchmarks

Benchmarks used for testing performance of the batch statement can be run using the following command:

go test -run=XXX -bench="Benchmark(No)?Batch" -benchtime=5x -cpu=8 -count=3

As with tests, this can chew through a lot of sockets on your system, so it is advised you tinker around with the -benchtime and -count flags.

Echo-chamber Web API

Located in: _examples/echo_chamber/

A simple node.js based web API server which echoes back information about any HTTP request made to it. This was created in order to have a web-API for testing sttp with. If the query param format=html is provided in the request then the response will be a mirror of the JSON response but will be returned as HTML. The server is forked 6 times creating 6 worker processes to create a rudimentary form of load balancing. This is in the hope that multiple requests can be handled at once.

The following are examples of some requests and responses:

GET 127.0.0.1:3000?hello=world

{
    "code": null,
    "headers": {
        "accept": "*/*",
        "accept-encoding": "gzip, deflate",
        "connection": "keep-alive",
        "host": "127.0.0.1:3000",
        "user-agent": "HTTPie/2.6.0"
    },
    "method": "GET",
    "query_params": {
        "hello": "world"
    },
    "url": "http://127.0.0.1:3000/?hello=world",
    "version": "1.1"
}
POST 127.0.0.1:3000/helloworld {"hello": "world"}

{
    "body": {
        "hello": "world"
    },
    "code": null,
    "headers": {
        "accept": "application/json, */*;q=0.5",
        "accept-encoding": "gzip, deflate",
        "connection": "keep-alive",
        "content-length": "18",
        "content-type": "application/json",
        "host": "127.0.0.1:3000",
        "user-agent": "HTTPie/2.6.0"
    },
    "method": "POST",
    "query_params": {},
    "url": "http://127.0.0.1:3000/helloworld",
    "version": "1.1"
}
GET 127.0.0.1:3000/api/hello?format=html

<html lang="en">
    <head>
        <title>GET: http://127.0.0.1:3000/api/hello?format=html</title>
    </head>
    <body>
        <h1>GET: http://127.0.0.1:3000/api/hello?format=html</h1>
        <div>
            <ul>
                <li>method: GET</li>
                <li>url: http://127.0.0.1:3000/api/hello?format=html</li>
                <li>
                    query_params:
                    <ul>
                        <li>format: html</li>
                    </ul>
                </li>
                <li>
                    headers:
                    <ul>
                        <li>host: 127.0.0.1:3000</li>
                        <li>user-agent: HTTPie/2.6.0</li>
                        <li>accept-encoding: gzip, deflate</li>
                        <li>accept: */*</li>
                        <li>connection: keep-alive</li>
                    </ul>
                </li>
                <li>code: null</li>
                <li>version: 1.1</li>
            </ul>
        </div>
    </body>
</html>

Prerequisites

Have node.js installed. No packages need to be installed.

How to use

node main.js

This will start the web server on 127.0.0.1:3000.