Hunter: Build and setup a Reticulum instance on a virtual machine.

[Voxelghiest]

Mozilla Hubs Installation Procedure

1. Reticulum

Reticulum is the core service that coordinates all of the other Hubs modules (PostgREST, Dialog, etc.). To compile the Reticulum source code successfully, we will first install a number of necessary dependencies (sect. 1.1 & 1.2) before downloading Reticulum itself (sect. 1.3).

1.1. Configure Postgresql

Postgresql is a database management service used by Hubs. To begin, install postgresql and start it:

$ sudo apt install postgresql
$ sudo systemctl start postgresql.service

In order for the Reticulum server to properly hook into the database locally, you must change some of the configurations for the postgres user. Switch to the postgres user and enter the psql command line:

<yourUsername>@<yourServer>$ sudo -i -u postgres
postgres@<yourServer>$ psql

You will be entered into the psql command line. Run the following psql commands (the semicolons are important). You will see ALTER ROLE display if you did it correctly.

postgres=# ALTER USER postgres WITH PASSWORD 'postgres';
ALTER ROLE
postgres=# ALTER USER postgres WITH SUPERUSER;
ALTER ROLE
postgres=# \q

postgres@<yourServer>$ exit

1.2. Install Erlang and Elixir

Reticulum is dependent on specific versions of Erlang and Elixir. In order to install the correct versions, we will use a software management tool called ASDF, because the usual package management software on Ubuntu (apt-get) lacks access to the versions we need. The following instructions are adapted from the ASDF website and this tutorial.

To get started, install ASDF by downloading the repo from GitHub:

$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf

This will place ASDF into the .asdf directory in your home folder.

Then, to give your shell access to the ASDF commands, add the following lines to the file ~/.bashrc (the periods are important):

. $HOME/.asdf/asdf.sh
. $HOME/.asdf/completions/asdf.bash

Before continuing, log out of your user with the exit command and log back in to refresh your shell.

Once you've logged in again, you can use ASDF to install Erlang and Elixir as plugins. (Be sure to pay careful attention to the specified versions!) Start by installing Erlang:

$ asdf plugin add erlang
$ asdf install erlang 23.3

The installation will take a while, during which Erlang will skip installing certain modules if you do not have the relevant services on your system. If you see warning messages about omitted modules, do not worry; it won't affect this tutorial.

After that finishes, install Elixir similarly:

$ asdf plugin add elixir
$ asdf install elixir 1.12.3-otp-23

The version of Elixir being installed is designed for maximum compatibility with Erlang 23.

Finally, you have to setup Erlang and Elixir for global access. Run the following:

$ asdf global erlang 23.3
$ asdf global elixir 1.12.3-otp-23

With that, we are finally ready to download the Reticulum source code.

1.3. Setup Reticulum

Before beginning, create a folder in your home directory to work out of. The files for the different Hubs modules will end up here. Inside that folder, clone Reticulum from the Mozilla GitHub:

$ git clone https://github.com/mozilla/reticulum.git
$ cd reticulum

Inside the Reticulum directory you cloned from GitHub earlier, run the following:

$ mix deps.get

This installs a number of Reticulum-specific dependencies. As part of the process, it may prompt you to install one or more packages:

Respond yes to all of them. This should be the final output:

Then run the command to compile the Reticulum scripts:

$ mix ecto.create

The compilation process will take a long time, and you may see a number of warning messages about deprecated or incorrect code. You should be fine as long as you see this message at the end:

Finally, create a storage folder in the Reticulum directory:

$ mkdir -p storage/dev

Now Reticulum should be able to operate on its own! To test it, run iex -S mix phx.server from the Reticulum folder. If everything worked, you'll see output like this:

1.4. Configure Reticulum to Run Against a Local Dialog Instance

There are a few configuration changes that have to be made for Reticulum to properly hook into the local Dialog instance, which we will be creating in the next section.

To start, from the Reticulum directory, go to ./config/dev.exs. Modify the file dev.exs as follows (line numbers are provided for convenience):

Line 13: dev_janus_host = "localhost"
Line 205: config :ret, Ret.JanusLoadStatus, default_janus_host: dev_janus_host, janus_port: 4443

These changes update the Janus host and port for later.

Then return to the base Reticulum directory and open up the file ./lib/ret_web/plugs/add_csp.ex. Edit the code block starting at line 102 to add the Dialog meta endpoint to the CSP rules:

default_janus_csp_rule =
   if default_janus_host,
      do: "wss://#{default_janus_host}:#{janus_port} https://#{default_janus_host}:#{janus_port} https://#{default_janus_host}:#{janus_port}/meta",
      else: ""

The last change we need to make requires us to install Coturn, a server management extension.

$ sudo apt-get install coturn

Once the package is installed, open the file /etc/turnserver.conf and add the following line at the top:

psql-userdb="host=localhost dbname=ret_dev user=postgres password=postgres options='-c search_path=coturn' connect_timeout=30"

2. Dialog

Before doing anything else, be sure to install NodeJS:

$ sudo apt install npm

Clone the Mozilla Dialog repo into the same directory as the Reticulum folder, then enter the Dialog folder and install any necessary dependencies:

$ git clone https://github.com/mozilla/dialog.git
$ cd dialog
$ npm install

This tutorial is incomplete. I will continue to add onto it as I make progress in figuring out the procedure.

[Voxelghiest]

I moved the documentation I started from #19 after speaking with Yuri. From now on, all my work on the Reticulum server will be documented here.

[Voxelghiest]

I am using a tutorial for setting up Reticulum someone made, which details additional setup steps not mentioned in the tutorial provided by Mozilla. The tutorial I found even has a linked YouTube video, demonstrating the steps.

Unfortunately, both the written and video portions of the tutorial are not of the most high quality, so I still have to do a lot of work on my own to make it all work.

Currently, I am figuring out how to install the Erlang and Elixir dependencies necessary for Reticulum to run correctly.

[Voxelghiest]

I installed Erlang and Elixir using the latest versions first, since that was the easiest thing to do. The installation commands were as follows:

$ wget https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
$ sudo dpkg -i erlang-solutions_2.0_all.deb
$ sudo apt-get update
$ sudo apt-get install erlang
$ sudo apt-get install elixir

And then I compiled the Reticulum code:

$ mix deps.get
$ mix ecto.create

The compilation seemed to run alright until the very end, which I ran into some sort of error:

I believe the error may have something to do with the versions of Erlang and Elixir I installed. It seems difficult for me to install any other versions, but that will be my next approach. Perhaps the compilation is dependent on features that only existed in earlier versions.

[Voxelghiest]

After a lot of confusing research regarding Ubuntu and apt-get versioning practices, I have finally figured out how to make Reticulum compile correctly.

I tried to use the exact versions of the Erlang and Elixir that the tutorial I'm following used (Erlang 23.3 & Elixir 1.12.3), but I could not get the earlier version of Erlang to compile, and I was only able to obtain version 1.12.2 of Elixir. Thankfully, though, that seemed to do the trick. I believe that the Elixir version was the most important aspect. Once everything compiled correctly, this is what I saw:

[Voxelghiest]

I have updated my procedure with the newest steps I was able to figure out.

[Voxelghiest]

I have continued working through the tutorial for setting everything up. I'm done with the Reticulum configuration, as far as I can tell, but I don't really know a way to test if Reticulum alone works, because the Hubs system requires the other components (Dialog, Hubs, etc.) to run. When I started working through the Dialog installation, and ran npm install to install the necessary dependencies, I came upon some sort of error:

[Voxelghiest]

I've decided to set aside working on Dialog for now and spend more time on Reticulum. As part of that, I began working on #28, learning more about the inner workings of Reticulum and the Phoenix server system it is built in.

When I ran the server with mix phx.server after completing the steps I've outlined so far, I encountered an error:

After some digging, I found this post, which suggests the problem is the version of Erlang I'm using. I will see about changing the version to pre-24.

[Voxelghiest]

Alright, so I was able to find a workaround to install an earlier version of Erlang:

$ wget https://packages.erlang-solutions.com/erlang/debian/pool/esl-erlang_23.3.4.5-1~ubuntu~bionic_amd64.deb
$ sudo dpkg -i esl-erlang_23.3.4.5-1~ubuntu~bionic_amd64.deb

Hopefully this one works. I'm going to retry setting up the Reticulum server from step 1.3 of my guide.

[Voxelghiest]

Okay, so I was able to get the server up and running by installing Erlang and Elixir using a different service called ASDF. It was a rather frustrating process, as I tried about three different ways of installing the correct versions. I've updated the tutorial with the details, but in short, ASDF is a package management utility (like apt-get).

Once that was done, I was finally able to run the Reticulum server! Here's what that looked like:

The only issue is that I can't seem to access the server from my host laptop. It's definitely some sort of networking issue, so my next objective is to sort that out.

[Voxelghiest]

It only took a little while! I fiddled with the network settings endlessly until I figured out that the server had the same MAC address as my laptop, which was confusing the router. Once I fixed that, I was able to set a static IP address for the server by putting the following into /etc/netplan/00-installer-config.yaml:

network:
    version: 2
    ethernets:
        enp0s17:
            addresses:
              - 10.0.0.200/24
            gateway4: 10.0.0.1
            nameservers:
                addresses: [10.0.0.1, 1.1.1.1, 4.4.4.4, 8.8.8.8]

I'm noting this down in case I have any similar issues in the future.

[Voxelghiest]

So I started the Reticulum server and tried accessing it from my laptop's browser. It seems to work! Sort of. Here's what my browser page looked like: The server also put out an error message in the console:

I don't know if the Reticulum server is intended to be accessed like this, so it might not be a problem. I have to do more research and learn what role it plays in the whole Hubs system. But this does mean that it's accessible from the outside, which is progress!