search

oxen-io / oxen-core

Oxen core repository, containing oxend and oxen cli wallets
https://oxen.io
Other
317 stars 120 forks source link
blockchain cryptocurrency masternode privacy ring-signatures

readme

Oxen

Copyright (c) 2018-2022 The Oxen Project.
Portions Copyright (c) 2014-2019 The Monero Project.
Portions Copyright (c) 2012-2013 The Cryptonote developers.

Development resources

Vulnerability disclosure

Information

Oxen is a private cryptocurrency based on Monero. Oxen currently offers an incentivised full node layer, over the coming months we will be looking to support a secondary p2p network (Lokinet) and a messenger that offers private communications based on the Signal protocol (Session).

More information on the project can be found on the website and in the whitepaper.

Oxen is an open source project, and we encourage contributions from anyone with something to offer. For more information on contributing, please contact team@oxen.io

Compiling Oxen from source

Dependencies

The following table summarizes the tools and libraries required to build. A few of the libraries are also included in this repository (marked as "Vendored"). By default, the build uses the library installed on the system, and ignores the vendored sources. However, if no library is found installed on the system, then the vendored source will be built and used. The vendored sources are also used for statically-linked builds because distribution packages often include only shared library binaries (.so) but not static library archives (.a).

Dep Min. version Vendored Debian/Ubuntu pkg Arch pkg Fedora Optional Purpose
GCC 8.1.0 NO g++[1] base-devel gcc NO
CMake 3.13 NO cmake cmake cmake NO
pkg-config any NO pkg-config base-devel pkgconf NO
Boost 1.65 NO libboost-all-dev[2] boost boost-devel NO C++ libraries
libzmq 4.3.0 YES libzmq3-dev zeromq zeromq-devel NO ZeroMQ library
sqlite3 3.24.0 YES libsqlite3-dev sqlite sqlite-devel NO ONS, batching
libsodium 1.0.9 YES libsodium-dev libsodium libsodium-devel NO cryptography
libcurl 4.0 NO libcurl4-dev curl curl-devel NO HTTP RPC
libuv (Win) any NO (Windows only) -- -- NO RPC event loop
libgmp any NO libgmp-dev -- -- NO BLS precision math
libunwind any NO libunwind8-dev libunwind libunwind-devel YES Stack traces
liblzma any NO liblzma-dev xz xz-devel YES For libunwind
libreadline 6.3.0 NO libreadline-dev readline readline-devel YES Input editing
Doxygen any NO doxygen doxygen doxygen YES Documentation
Graphviz any NO graphviz graphviz graphviz YES Documentation
Qt tools 5.x NO qttools5-dev qt5-tools qt5-linguist YES Translations
libhidapi ? NO libhidapi-dev hidapi hidapi-devel YES Hardware wallet
libusb ? NO libusb-dev libusb libusb-devel YES Hardware wallet
libprotobuf ? NO libprotobuf-dev protobuf protobuf-devel YES Hardware wallet
protoc ? NO protobuf-compiler protobuf protobuf-compiler YES Hardware wallet

[1] On Ubuntu Bionic you will need the g++-8 package instead of g++ (which is version 7) and will need to run export CC=gcc-8 CXX=g++-8 before running make or cmake.

[2] libboost-all-dev includes a lot of unnecessary packages; see the apt command below for a breakdown of the minimum set of required boost packages.

Install all dependencies at once on Debian/Ubuntu:

sudo apt update && sudo apt install build-essential cmake pkg-config libboost-all-dev libzmq3-dev libsodium-dev libunwind8-dev liblzma-dev libreadline6-dev doxygen graphviz libpgm-dev libsqlite3-dev libcurl4-dev

Install all dependencies at once on macOS with the provided Brewfile: brew update && brew bundle --file=contrib/brew/Brewfile

FreeBSD one liner for required to build dependencies pkg install git gmake cmake pkgconf boost-libs libzmq4 libsodium sqlite3

Build instructions

Oxen uses the CMake build system which is used by creating a build directory and invoke cmake before building.

On Linux and macOS

You do not have to build from source if you are on debian or ubuntu as we have apt repositories with pre-built oxen packages on deb.oxen.io.

You can install these using:

$ sudo curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
$ echo "deb https://deb.oxen.io $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/oxen.list
$ sudo apt update
$ sudo apt install oxend

if you want to build a dev build you can do the following after installing the dependancies above:

$ git clone --recursive https://github.com/oxen-io/oxen-core.git
$ cd oxen-core
$ git submodule update --init --recursive
$ mkdir build
$ cd build
$ cmake ..
$ make -j$(nproc)

Dependencies need to be built with -fPIC. Static libraries usually aren't, so you may have to build them yourself with -fPIC. Refer to their documentation for how to build them.

On the Raspberry Pi (and similar ARM-based devices)

The build process is exactly the same, but note that some parts of the build require around 3GB of RAM which is more memory than most Raspberry Pi class devices have available. You can work around this by enabling 2GB (or more) of swap, but this is not particularly recommended, particularly if the swap file is on the SD card: intensive writes to a swap file on an SD card can accelerate how quickly the SD card wears out. Devices with 4GB of RAM (such as the 4GB model of the Pi 4B, and some other SBC ARM devices) can build without needing swap.

As an alternative, pre-built oxen debs are available for ARM32 and ARM64 for recent Debian/Raspbian/Ubuntu distributions and are often a much better alternative for SBC-class devices. If you still want to compile from source, ensure you have enough memory (or swap -- consult your OS documentation to learn how to enable or increase swap size) and follow the regular linux build instructions above.

On Windows:

Binaries for Windows are built on Windows using the MinGW toolchain within MSYS2 environment. The MSYS2 environment emulates a POSIX system. The toolchain runs within the environment and cross-compiles binaries that can run outside of the environment as a regular Windows application.

Preparing the build environment

Cloning

Building

On FreeBSD:

The project can be built from scratch by following instructions for Linux above(but use gmake instead of make). If you are running Oxen in a jail, you need to add sysvsem="new" to your jail configuration, otherwise lmdb will throw the error message: Failed to open lmdb environment: Function not implemented.

On OpenBSD:

You will need to add a few packages to your system. pkg_add cmake gmake zeromq cppzmq libiconv boost.

The doxygen and graphviz packages are optional and require the xbase set. Running the test suite also requires py-requests package.

Build oxen: env DEVELOPER_LOCAL_TOOLS=1 BOOST_ROOT=/usr/local gmake release-static

Note: you may encounter the following error, when compiling the latest version of oxen as a normal user:

LLVM ERROR: out of memory
c++: error: unable to execute command: Abort trap (core dumped)

Then you need to increase the data ulimit size to 2GB and try again: ulimit -d 2000000

On Solaris:

The default Solaris linker can't be used, you have to install GNU ld, then run cmake manually with the path to your copy of GNU ld:

mkdir -p build/release
cd build/release
cmake -DCMAKE_LINKER=/path/to/ld -D CMAKE_BUILD_TYPE=Release ../..
cd ../..

Then you can run make as usual.

On Linux for Android (using docker):

# Build image (for ARM 32-bit)
docker build -f utils/build_scripts/android32.Dockerfile -t oxen-android .
# Build image (for ARM 64-bit)
docker build -f utils/build_scripts/android64.Dockerfile -t oxen-android .
# Create container
docker create -it --name oxen-android oxen-android bash
# Get binaries
docker cp oxen-android:/src/build/release/bin .

Building portable statically linked binaries

By default, in either dynamically or statically linked builds, binaries target the specific host processor on which the build happens and are not portable to other processors. Portable binaries can be built using the following targets:

Cross Compiling

You can also cross-compile static binaries on Linux for Windows and macOS with the depends system.

The required packages are the names for each toolchain on apt. Depending on your distro, they may have different names.

Using depends might also be easier to compile Oxen on Windows than using MSYS. Activate Windows Subsystem for Linux (WSL) with a distro (for example Ubuntu), install the apt build-essentials and follow the depends steps as depicted above.

The produced binaries still link libc dynamically. If the binary is compiled on a current distribution, it might not run on an older distribution with an older installation of libc. Passing -DBACKCOMPAT=ON to cmake will make sure that the binary will run on systems having at least libc version 2.17.

Installing Oxen from a package

Pre-built packages are available for recent Debian and Ubuntu systems (and are often usable on Debian or Ubuntu-derived Linux distributions). For more details see https://deb.imaginary.stream

You can also build a docker package using:

```bash
# Build using all available cores
docker build -t oxen-daemon-image .

# or build using a specific number of cores (reduce RAM requirement)
docker build --build-arg NPROC=1 -t oxen .

# either run in foreground
docker run -it -v /oxen/chain:/root/.oxen -v /oxen/wallet:/wallet -p 22022:22022 oxen 

# or in background
docker run -it -d -v /oxen/chain:/root/.oxen -v /oxen/wallet:/wallet -p 22022:22022 oxen 
```

Running oxend

The build places the binary in bin/ sub-directory within the build directory from which cmake was invoked (repository root by default). To run in foreground:

./bin/oxend

To list all available options, run ./bin/oxend --help. Options can be specified either on the command line or in a configuration file passed by the --config-file argument. To specify an option in the configuration file, add a line with the syntax argumentname=value, where argumentname is the name of the argument without the leading dashes, for example log-level=1.

To run in background:

./bin/oxend --log-file oxend.log --detach

To run as a systemd service, copy oxend.service to /etc/systemd/system/ and oxend.conf to /etc/. The example service assumes that the user oxen exists and its home is the data directory specified in the example config.

If you're on Mac, you may need to add the --max-concurrency 1 option to oxen-wallet-cli, and possibly oxend, if you get crashes refreshing.

Internationalization

See README.i18n.md.

Debugging

This section contains general instructions for debugging failed installs or problems encountered with Oxen. First ensure you are running the latest version built from the Github repo.

Obtaining stack traces and core dumps on Unix systems

We generally use the tool gdb (GNU debugger) to provide stack trace functionality, and ulimit to provide core dumps in builds which crash or segfault.

Run the build.

Once it stalls, enter the following command:

gdb /path/to/oxend `pidof oxend`

Type thread apply all bt within gdb in order to obtain the stack trace

Enter ulimit -c unlimited on the command line to enable unlimited filesizes for core dumps

Enter echo core | sudo tee /proc/sys/kernel/core_pattern to stop cores from being hijacked by other tools

Run the build.

When it terminates with an output along the lines of "Segmentation fault (core dumped)", there should be a core dump file in the same directory as oxend. It may be named just core, or core.xxxx with numbers appended.

You can now analyse this core dump with gdb as follows:

gdb /path/to/oxend /path/to/dumpfile`

Print the stack trace with bt

coredumpctl -1 gdb

To run Oxen within gdb:

Type gdb /path/to/oxend

Pass command-line options with --args followed by the relevant arguments

Type run to run oxend

Analysing memory corruption

There are two tools available:

ASAN

Configure Oxen with the -D SANITIZE=ON cmake flag, eg:

cd build/debug && cmake -D SANITIZE=ON -D CMAKE_BUILD_TYPE=Debug ../..

You can then run the oxen tools normally. Performance will typically halve.

valgrind

Install valgrind and run as valgrind /path/to/oxend. It will be very slow.

LMDB

Instructions for debugging suspected blockchain corruption as per @HYC

There is an mdb_stat command in the LMDB source that can print statistics about the database but it's not routinely built. This can be built with the following command:

cd ~/oxen/external/db_drivers/liblmdb && make

The output of mdb_stat -ea <path to blockchain dir> will indicate inconsistencies in the blocks, block_heights and block_info table.

The output of mdb_dump -s blocks <path to blockchain dir> and mdb_dump -s block_info <path to blockchain dir> is useful for indicating whether blocks and block_info contain the same keys.

These records are dumped as hex data, where the first line is the key and the second line is the data.

Testing tools

Local Devnet

The local devnet script in utils/local-devnet/service_node_network.py will spin up a series of service nodes that can be interacted with locally for testing. This script requires that:

Thereafter the script can be invoked to launch the local network.

Known Issues

Protocols

Socket-based

Because of the nature of the socket-based protocols that drive Oxen, certain protocol weaknesses are somewhat unavoidable at this time. While these weaknesses can theoretically be fully mitigated, the effort required (the means) may not justify the ends. As such, please consider taking the following precautions if you are a Oxen node operator:

Blockchain-based

Certain blockchain "features" can be considered "bugs" if misused correctly. Consequently, please consider the following: