search

RTradeLtd / Temporal

☄️ Temporal is an easy-to-use, enterprise-grade interface into distributed and decentralized storage
https://temporal.cloud
MIT License
227 stars 40 forks source link
data ethereum ethereum-swarm golang i2p infrastructure ipfs ipfs-cluster ipns pinning storage swarm temporal

readme

The hosted service for this project is being shutdown. For more information consult this github issue

Temporal ☄️

About Temporal · Web Interfaces · Usage and Features · Contributing · License · Thanks

GoDocs available Travis Build Status Release Clean code


Multi-Language

jaywcjlove/sb

About Temporal

Temporal is an enterprise-grade storage solution that allows you easily integrate with distributed storage technologies like IPFS, without sacrificing functionality with an easy to use API leveraging all the benefits the distributed web has to offer.

Temporal's API comes in two flavors, hosted or on-site. Should you not have the resources, or interest in maintaining your own infrastructure you can take advantage of our hosted API running in our very own datacenter. Those who have the interest, and/or resources may deploy Temporal within your own environments. For those that deploy Temporal themselves, we offer paid for support, installation, tutorials, and product usage information sessions allowing organizations to leverage all the capabilities that Temporal offers.

Temporal is modular such that the underlying protocols it connects to, can easily be upgraded, and replaced with without having to change the overall architecture. See our protocol-expansion.md documentation for details on extending the available functionality.

We have comprehensive API documentation available here as well as an in-depth wiki which contains additional information such as architectural diagrams, design decisions, and more.

Benefits Over Direct IPFS HTTP API Access

Goals

Versioning Policy

Information about our versioning policies are available in VERSIONING.md

Web Interfaces

For those less interested in API usage, we have a web interface which can be used with two methods of access:

Please note that support for the I2P Interface is very experimental at the moment and does not offer HTTPS, as well as very infrequent updates.There are no stability, or even functionality guarantees for the I2P interface.

How We're Different

Temporal gives everyone the chances to run their own enterprise grade IPFS infrastructure without needing to rely on third-party API providers that dominate the blockchain development market. However if you are unable to do this, you can use our hosted API running in our very own datacenter, using the exact same open-source code you see here.

By using our hosted API you can experience the same level of enterprise quality service that you can setup in your own environments, whether that be cloud VMs, your own datacenter, or a server under your bed. Additionally, by building your applications decentralized or centralized with Temporal, you won't be vendor locked in if you decide to transition to a self-hosted infrastructure, because all it takes is changing a single URL that your application points to.

Temporal provides a first of its kind API that outweighs every other third-party IPFS and decentralized storage API on the market so you can fully leverage all the benefits the distributed web has to offer.

Funding

Currently the project is paid for by RTrade Technologies Ltd, and we will not be doing an ICO. Funding is derived from private investment, mining farm profits, as well as purchasing of RTC.

Should you wish to consider donations, or private investment, email admin@rtradetechnologies.com.

Should you wish to contribute not just to Temporal, but to the overall success of RTrade and our platform, you may purchase RTC for ETH from our RTCETH Smart Contract

Media

Channels:

Selected Content:

Data Privacy

Our datacenter and cloud environments are all located within Canada, which has exceptional data-privacy laws. We comply with all laws and regulations surrounding data storage regulation within Canada. Should you feel like there is any discrepancy here, please contact us at admin@rtradetechnologies.com and we will be happy to resolve your concerns, and if there's anything we need to change, we will do so.

One of the big concerns with IPFS, and even cloud data storage in general is encryption. As IPFS doesn't yet support native data encryption, we allow users to encrypt their data using AES256-CFB. While this is better than storing data without encryption on IPFS, there are still some concerns with encrypted data storage on IPFS. Namely, if anyone is ever able to discover the content hash and pin the content, it will always be available to them. This is of big concern when using encryption algorithms as it is theoretically possible for someone to persist that data within their own storage system until the desired ciphers are broken, and they can crack the encryption algorithm. If this is something that is of concern to you, an even better solution is to encrypt your data, and store in on private networks. We have plans to eventually migrate to AES256-GCM which is more secure than AES256-CFB, and allow encryption of data with IPFS keys.

Usage and Features

Before attempting to use Temporal you will need to install it. Even if you are going to be using our dockerized tooling, an install of Temporal is needed primarily for configuration file initialization.

Please note that a full-blown Temporal instance including the payment processing backend can take awhile, and requires an API key for ChainRider as well as a fully synced geth node, and bchd node. We will not be covering the setup of either chainrider, geth, and bchd, please consult appropriate documentation for setting those up. Should you want to read about our payment processing backedn see RTradeLtd/Pay

For a minimal setup you do not need to configure anything related to the aforementioned topics (geth, pay, bch, etc..). It is worth mentioning though that running a minimal setup doesn't disable the payment API calls, so if someone were to place these API calls against your minimal setup it might result in unexpected errors, such as panics. If this happens please open a bug report on github.

The rest of this usage documentation will be covering a bare-minimum Temporal setup which does not include any payment processing capabilities. Thus you will not be able to "purchase credits" the remedy to this is to manually alter user account balances, or promote a user to partner tier, registering an organization, and then creating all new users under that organization. This effectively side-steps the billing process, and requires no manual management of user credits.

For details on organization management, and the entire API please consult our api docs.

Installing Temporal

The first thing you need to do is install Temporal, for which there are two main options:

1) Compiling from source 2) Downloading pre-built binaries

Compiling From Source:

This is quite a bit more complicated, and requires things like a proper golang version installed. Unless you have specific reason to compile from source, it is recommended you skip this and stick with download the pre-built binaries.

If you do want to download and build from source, be aware the download process can take A LONG TIME depending on your bandwidth and internet speed. Usually it takes up to 30 minutes.

Should you still want to do this, download the install_from_source.sh script. This will ensure you have the proper go version, download the github repository, compile the cli, and install it to /bin/temporal.

#! /bin/bash

# setup install variabels
GOVERSION=$(go version | awk '{print $3}' | tr -d "go" | awk -F "." '{print $2}')
WORKDIR="/tmp/temporal-workdir"
# handle golagn version detection
if [[ "$GOVERSION" -lt 11 ]]; then
    echo "[ERROR] golang is less than 1.11 and will produce errors"
    exit 1
fi
if [[ "$GOVERSION" -lt 12 ]]; then
    echo "[WARN] detected golang version is less than 1.12 and may produce errors"
fi
# create working directory
mkdir "$WORKDIR"
cd "$WORKDIR"
# download temporal
git clone https://github.com/RTradeLtd/Temporal.git
cd Temporal
# initialize submodules, and download all dependencies
make setup
# make cli binary
make install

Downloading Pre-Built Binaries:

To download pre-built binaries, currently available for Linux and Mac OSX platforms, head on over to our github releases page.

Download a binary for your appropriate platform, additionally you can download sha256 checksums of the released binaries for post-download integrity verification.

You'll then want to copy the pre-built binary over to anywhere in your PATH environment variable.

Example for downloading and installing the Linux version:

# download the binary
wget https://github.com/RTradeLtd/Temporal/releases/download/v2.2.7/temporal-v2.2.7-linux-amd64
# download the binary checksum
wget https://github.com/RTradeLtd/Temporal/releases/download/v2.2.7/temporal-v2.2.7-linux-amd64.sha256
# store downloaded checksum output
CK_HASH=$(cat *.sha256  | awk '{print $1}')
# calculate sha256 checksum of downloaded binary
DL_HASH=$(sha256sum temporal-v2.2.7-linux-amd64 | awk '{print $1}')
# compare checksums, and copy file to `PATH` if ok
# if this doesn't show ok then your binary is corrupted or has been tampered with
if [[ "$CK_HASH" == "$DL_HASH" ]]; then  echo "ok" && sudo cp temporal-v2.2.7-linux-amd64 /usr/local/bin; fi

Configuration Initialization

After downloading Temporal, regardless of your setup process you will need a configuration file. To generate a config file at /home/doggo/config.json run the following command:

temporal -config /home/doggo/config.json init

Alternatively you can set the environment variable CONFIG_DAG and temporal init, along with all other commands will read from this location. It is recommended that you do this as it makes using the cli a lot easier

It is extremely important you keep this in a directory that is only accessible to the users required to run the servivce as it contains usernames and passwords to key pieces of Temporal's infrastructure.

For an example bare-minium configuration file, check out testenv/config.json

Note that if you did the install from source method, you will already have a config file in your home directory called temporal-config.json.

Manual Setup

This exact process will vary a bit depending on the environment you are installing Temporal in. At the very least you are required to use Postgres, and RabbitMQ. The operating systems you install those, and the supplementary services on is entirely up to you, but we recommend using Ubuntu 18.04LTS. For instructions on setting up Postgres see their documentation. For instructions on setting up RabbitMQ consult their documentation. We do go into a bit of a setup process for RabbitMQ in the confluence page linked below, although it is always good to read official sources.

For the manual setup process using Ubuntu 18.04LTS consult our confluence page. For the manual setup process using other operating systems, please read the confluence page and adjust the commands as needed. The confluence page covers filling out the needed parts of the configuration file.

Dockerized Setup

The docker-compose file defaults to placing everything in /data/temporal, so for this part of the tutorial we will be using that particular default. If you want to override it you can use the BASE=/path/to/base variable.

First off you'll need to copy the Temporal config file to /data/temporal/config.json then you can proceed with the rest of the steps. Ensure that the config file is pointing to a postgresql and rabbitmq docker container, or server that is reachable by the docker containers that will be started up.

Additionally you'll need to make sure that any tls certificates, and files needed by the api service are appropriately located within /data/temporal.

To download the docker-compose file:

$> curl https://raw.githubusercontent.com/RTradeLtd/Temporal/master/temporal.yml --output temporal.yml

Then afterwards to execute the docker-compose file using the latest version of the temporal docker image run

$> env TEMPORAL=latest docker-compose -f temporal.yml up

The standalone Temporal Docker image is available on Docker Hub.

API Documentation

Our API documentation has been redesigned to use slate, hosted through IPFS. The main way to view it is through our gateway. However, in theory it is viewable across any gateway by navigating to /ipns/docs.api.temporal.cloud

Features

Supported Technologies

Currently we fully support all non-experimental IPFS and IPNS feature-sets. Features like UnixFS, MFS are on-hold until their specs, and implementations become more stable and usable within production environments.

System Monitoring

Temporal is designed to be monitored with a combination of Zabbix, and Grafana + Prometheus. Zabbix is used for operating system, and hardware level metric collection, while Grafana + Prometheus are used to scrape API metrics, along with IPFS and IPFS Cluster node metrics. We include Zabbix templates, and Grafana graphs within the setup/configs folder.

License

In order to better align with the same open source values that originally inspired this project, Temporal has been changed to an MIT license for its production release. Originally, I (postables), intended to release under Apache 2.0, however I think to truly help the open-source, and IPFS movement launching under the MIT license is needed.

FOSSA Status

Thanks

Without open source, Temporal wouldn't be possible, as such we would like to extend our thanks to all of the open source projects on which Temporal depends on. If you notice any are missing from the list below, please open an issue and we will add it to the list.