search

engineer-man / piston

A high performance general purpose code execution engine.
https://emkc.org/run
MIT License
1.83k stars 230 forks source link
code-execution hacktoberfest hacktoberfest2022 lxc nodejs piston shell

readme

engineer-man piston Piston

A high performance general purpose code execution engine.


GitHub last commit GitHub issues GitHub pull requests

About • Public API • Getting Started • Usage • Supported Languages • Principles • Security • License • Documentation


About

Piston is a high performance general purpose code execution engine. It excels at running untrusted and possibly malicious code without fear from any harmful effects.


It's used in numerous places including:


Official Extensions

The following are approved and endorsed extensions/utilities to the core Piston offering.


Public API


When using the public Piston API, use the following two URLs:

GET  https://emkc.org/api/v2/piston/runtimes
POST https://emkc.org/api/v2/piston/execute

Important Note: The Piston API is rate limited to 5 requests per second. Effective May 7, 2024, no additional unlimited keys will be granted and existing keys will be revoked on Jan 1, 2025. The public instance is at capacity and the public limit is already very generous. For usage beyond 5 requests/second, you should consider self hosting.


Getting Started

All In One

Host System Package Dependencies

After system dependencies are installed, clone this repository:

# clone and enter repo
git clone https://github.com/engineer-man/piston

Installation

# Start the API container
docker-compose up -d api

# Install all the dependencies for the cli
cd cli && npm i && cd -

The API will now be online with no language runtimes installed. To install runtimes, use the CLI.

Just Piston (no CLI)

Host System Package Dependencies

Installation

docker run \
    -v $PWD:'/piston' \
    --tmpfs /piston/jobs \
    -dit \
    -p 2000:2000 \
    --name piston_api \
    ghcr.io/engineer-man/piston

Piston for testing packages locally

Host System Package Dependencies

Installation

# Build the Docker containers
./piston start

# For more help
./piston help


Usage

CLI

The CLI is the main tool used for installing packages within piston, but also supports running code.

You can execute the cli with cli/index.js.

# List all available packages
cli/index.js ppman list

# Install latest python
cli/index.js ppman install python

# Install specific version of python
cli/index.js ppman install python=3.9.4

# Run a python script using the latest version
echo 'print("Hello world!")' > test.py
cli/index.js run python test.py

# Run a python script using a specific version
echo 'print("Hello world!")' > test.py
cli/index.js run python test.py -l 3.9.4
cli/index.js run python test.py -l 3.x
cli/index.js run python test.py -l 3

If you are operating on a remote machine, add the -u flag like so:

cli/index.js -u http://piston.server:2000 ppman list

API

The container exposes an API on port 2000 by default. This is used by the CLI to carry out running jobs and package management.

Runtimes Endpoint

GET /api/v2/runtimes This endpoint will return the supported languages along with the current version and aliases. To execute code for a particular language using the /api/v2/execute endpoint, either the name or one of the aliases must be provided, along with the version. Multiple versions of the same language may be present at the same time, and may be selected when running a job.

HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "language": "bash",
        "version": "5.1.0",
        "aliases": [
            "sh"
        ]
    },
    {
        "language": "brainfuck",
        "version": "2.7.3",
        "aliases": [
            "bf"
        ]
    },
    ...
]

Execute Endpoint

POST /api/v2/execute This endpoint requests execution of some arbitrary code.

{
    "language": "js",
    "version": "15.10.0",
    "files": [
        {
            "name": "my_cool_code.js",
            "content": "console.log(process.argv)"
        }
    ],
    "stdin": "",
    "args": ["1", "2", "3"],
    "compile_timeout": 10000,
    "run_timeout": 3000,
    "compile_memory_limit": -1,
    "run_memory_limit": -1
}

A typical response upon successful execution will contain 1 or 2 keys run and compile. compile will only be present if the language requested requires a compile stage.

Each of these keys has an identical structure, containing both a stdout and stderr key, which is a string containing the text outputted during the stage into each buffer. It also contains the code and signal which was returned from each process.

HTTP/1.1 200 OK
Content-Type: application/json

{
    "language": "js",
    "version": "15.10.0",
    "run": {
        "stdout": "[\n  '/piston/packages/node/15.10.0/bin/node',\n  '/piston/jobs/9501b09d-0105-496b-b61a-e5148cf66384/my_cool_code.js',\n  '1',\n  '2',\n  '3'\n]\n",
        "stderr": "",
        "output": "[\n  '/piston/packages/node/15.10.0/bin/node',\n  '/piston/jobs/9501b09d-0105-496b-b61a-e5148cf66384/my_cool_code.js',\n  '1',\n  '2',\n  '3'\n]\n",
        "code": 0,
        "signal": null
    }
}

If a problem exists with the request, a 400 status code is returned and the reason in the message key.

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
    "message": "html-5.0.0 runtime is unknown"
}


Supported Languages

awk, bash, befunge93, brachylog, brainfuck, bqn, c, c++, cjam, clojure, cobol, coffeescript, cow, crystal, csharp, csharp.net, d, dart, dash, dragon, elixir, emacs, emojicode, erlang, file, forte, forth, fortran, freebasic, fsharp.net, fsi, go, golfscript, groovy, haskell, husk, iverilog, japt, java, javascript, jelly, julia, kotlin, lisp, llvm_ir, lolcode, lua, matl, nasm, nasm64, nim, ocaml, octave, osabie, paradoc, pascal, perl, php, ponylang, powershell, prolog, pure, pyth, python, python2, racket, raku, retina, rockstar, rscript, ruby, rust, samarium, scala, smalltalk, sqlite3, swift, typescript, basic, basic.net, vlang, vyxal, yeethon, zig,


Principle of Operation

Piston uses Docker as the primary mechanism for sandboxing. There is an API within the container written in Node which takes in execution requests and executees them within the container safely. High level, the API writes any source code to a temporary directory in /piston/jobs. The source file is either ran or compiled and ran (in the case of languages like c, c++, c#, go, etc.).


Security

Docker provides a great deal of security out of the box in that it's separate from the system. Piston takes additional steps to make it resistant to various privilege escalation, denial-of-service, and resource saturation threats. These steps include:


License

Piston is licensed under the MIT license.