orhun / rustypaste

A minimal file upload/pastebin service.
https://blog.orhun.dev/blazingly-fast-file-sharing
MIT License
799 stars 48 forks source link
file-sharing file-upload file-upload-server file-upload-service file-uploading hacktoberfest multipart-uploads paste-service pastebin pastebin-service rust upload upload-file upload-images

GitHub Release Crate Release Coverage Continuous Integration Continuous Deployment Docker Builds Documentation

Rustypaste is a minimal file upload/pastebin service.

$ echo "some text" > awesome.txt

$ curl -F "file=@awesome.txt" https://paste.site.com
https://paste.site.com/safe-toad.txt

$ curl https://paste.site.com/safe-toad.txt
some text

Table of Contents - [Features](#features) - [Installation](#installation) - [From crates.io](#from-cratesio) - [Arch Linux](#arch-linux) - [Alpine Linux](#alpine-linux) - [FreeBSD](#freebsd) - [Binary releases](#binary-releases) - [Build from source](#build-from-source) - [Feature flags](#feature-flags) - [Testing](#testing) - [Unit tests](#unit-tests) - [Test Fixtures](#test-fixtures) - [Usage](#usage) - [CLI](#cli) - [Expiration](#expiration) - [One shot files](#one-shot-files) - [One shot URLs](#one-shot-urls) - [URL shortening](#url-shortening) - [Paste file from remote URL](#paste-file-from-remote-url) - [Cleaning up expired files](#cleaning-up-expired-files) - [Delete file from server](#delete-file-from-server) - [Override the filename when using `random_url`](#override-the-filename-when-using-random_url) - [Server](#server) - [List endpoint](#list-endpoint) - [HTML Form](#html-form) - [Docker](#docker) - [Nginx](#nginx) - [Contributing](#contributing) - [License](#license)

Features

Installation

Packaging status [![Packaging status](https://repology.org/badge/vertical-allrepos/rustypaste.svg)](https://repology.org/project/rustypaste/versions)

From crates.io

cargo install rustypaste

Arch Linux

pacman -S rustypaste

Alpine Linux

rustypaste is available for Alpine Edge. It can be installed via apk after enabling the community repository.

apk add rustypaste

FreeBSD

pkg install rustypaste

Binary releases

See the available binaries on the releases page.

Build from source

git clone https://github.com/orhun/rustypaste.git
cd rustypaste/
cargo build --release

Feature flags

To enable a feature for build, pass --features flag to cargo build command.

For example, to reuse the OpenSSL present on a distro already:

cargo build --release --no-default-features --features openssl

Testing

Unit tests
cargo test -- --test-threads 1
Test Fixtures
./fixtures/test-fixtures.sh

Usage

The standalone command line tool (rpaste) is available here.

CLI

function rpaste() {
  curl -F "file=@$1" -H "Authorization: <auth_token>" "<server_address>"
}

* consider reading authorization headers from a file. (e.g. -H @rpaste_auth)

# upload a file
$ rpaste x.txt

# paste from stdin
$ rpaste -

Expiration

$ curl -F "file=@x.txt" -H "expire:10min" "<server_address>"

supported units:

One shot files

$ curl -F "oneshot=@x.txt" "<server_address>"

One shot URLs

$ curl -F "oneshot_url=https://example.com" "<server_address>"

URL shortening

$ curl -F "url=https://example.com/some/long/url" "<server_address>"

Paste file from remote URL

$ curl -F "remote=https://example.com/file.png" "<server_address>"

Cleaning up expired files

Configure [paste].delete_expired_files to set an interval for deleting the expired files automatically.

On the other hand, following script can be used as cron for cleaning up the expired files manually:

#!/bin/env sh
now=$(date +%s)
find upload/ -maxdepth 2 -type f -iname "*.[0-9]*" |
while read -r filename; do
    [ "$(( ${filename##*.} / 1000 - "${now}" ))" -lt 0 ] && rm -v "${filename}"
done

Delete file from server

Set delete_tokens array in config.toml to activate the DELETE endpoint and secure it with one (or more) auth token(s).

$ curl -H "Authorization: <auth_token>" -X DELETE "<server_address>/file.txt"

The DELETE endpoint will not be exposed and will return 404 error if delete_tokens are not set.

Override the filename when using random_url

The generation of a random filename can be overridden by sending a header called filename:

curl -F "file=@x.txt" -H "filename: <file_name>" "<server_address>"

Server

To start the server:

$ rustypaste

If the configuration file is not found in the current directory, specify it via CONFIG environment variable:

$ CONFIG="$HOME/.rustypaste.toml" rustypaste

Authentication

To enable basic HTTP auth, set the AUTH_TOKEN environment variable (via .env):

$ echo "AUTH_TOKEN=$(openssl rand -base64 16)" > .env
$ rustypaste

There are 2 options for setting multiple auth tokens:

If neither AUTH_TOKEN, AUTH_TOKENS_FILE nor [server].auth_tokens are set, the server will not require any authentication.

Exception is the DELETE endpoint, which requires at least one token to be set. See deleting files from server for more information.

See config.toml for configuration options.

List endpoint

Set expose_list to true in config.toml to be able to retrieve a JSON formatted list of files in your uploads directory. This will not include oneshot files, oneshot URLs, or URLs.

$ curl "http://<server_address>/list"

[{"file_name":"accepted-cicada.txt","file_size":241,"expires_at_utc":null}]

This route will require an AUTH_TOKEN if one is set.

HTML Form

It is possible to use an HTML form for uploading files. To do so, you need to update two fields in your config.toml:

For an example, see examples/html_form.toml

Docker

Following command can be used to run a container which is built from the Dockerfile in this repository:

$ docker run --rm -d \
  -v "$(pwd)/upload/":/app/upload \
  -v "$(pwd)/config.toml":/app/config.toml \
  --env-file "$(pwd)/.env" \
  -e "RUST_LOG=debug" \
  -p 8000:8000 \
  --name rustypaste \
  orhunp/rustypaste

You can build this image using docker build -t rustypaste . command.

If you want to run the image using docker compose, simply run docker-compose up -d. (see docker-compose.yml)

Nginx

Example server configuration with reverse proxy:

server {
    listen 80;
    location / {
        proxy_pass                         http://localhost:8000/;
        proxy_set_header Host              $host;
        proxy_set_header X-Forwarded-For   $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        add_header X-XSS-Protection        "1; mode=block";
        add_header X-Frame-Options         "sameorigin";
        add_header X-Content-Type-Options  "nosniff";
    }
}

If you get a 413 Request Entity Too Large error during upload, set the max body size in nginx.conf:

http {
    # ...
    client_max_body_size 100M;
}

Contributing

Pull requests are welcome!

Consider submitting your ideas via issues first and check out the existing issues.

License

All code is licensed under The MIT License.