spinalcordmri.org
This repo contains the source code and documentation for the https://spinalcordmri.org/ website.
The main website is a simple landing page written using the Jekyll static site generator, and is deployed and hosted using GitHub Pages. The website uses a custom domain name purchased from NameCheap, then linked with GitHub Pages. (This way, instead of https://spinalcordmri.github.io, we get to use https://spinalcordmri.org.)
Configuration settings can be found here:
Configuration is relatively simple, and was done using instructions from the following pages:
The page can be built locally after installing Jekyll. See more details here.
forum.spinalcordmri.org
The webforum "forum.spinalcordmri.org" is a subdomain of the spinalcordmri.org website. The forum was originally envisioned as a general web forum and community for discussing MRI processing, acquisition, etc. That being said, the spinalcordmri.org forum is often colloquially referred to as the "SCT Forum", because the most active part of the forum is the "SCT" subsection.
The forum runs using the open-source forum software Discourse, and operates within a VM provided by DigitalOcean. This means that it is an entirely separate site from the main website, and so the setup, configuration, and administration of the forum is a bit more involved than the Jekyll part of the site.
If you need to remake the forum's VM from scratch (e.g. to debug an issue without impacting the production server), then follow the instructions below.
NB: The following instructions are heavily based off of Discourse's official Cloud Installation instructions found here. If anything is unclear, it may be helpful to refer to those instructions for further guidance.
Before you begin, first choose an FQDN (i.e. a domain name). This can be forum.spinalcordmri.org
if you're starting completely from scratch (i.e. there is no currently-running forum instance) or something like forum.dev.spinalcordmri.org
(if you're setting up a separate test server).
NB: For the rest of these instructions, we will use the placeholder
{subdomain}.spinalcordmri.org
, but make sure you substitute in whichever domain name you decided on.
First, make sure you have an account with DigitalOcean. Next, contact someone on the admin team; they will add you to the NeuroPoly DigitalOcean team. This will grant you access to the Spinal Cord MRI project.
From the Spinal Cord MRI project page, you can create a new "droplet", which is DigitalOcean's name for cloud servers.
Next, create a droplet using the following settings:
{subdomain}.spinalcordmri.org
)Once the droplet has finished creating, you should see a public IP address on the droplet's page that looks something like ipv4: 142.93.152.255
. You can then use this IP address to test that you've set the hostname correctly by running the following command on any linux machine:
user@device:~$ dig +short -x 142.93.152.255
{subdomain}.spinalcordmri.org.
Once you've confirmed that it returns the domain name you chose, you're all set to connect to the droplet.
Before we make any changes to the server, though, we first have to configure some DNS settings.
First, make sure you have an account with Namecheap. Again, you will need to contact someone on the admin team; they will grant you permissions for specific domains on a case-by-case basis. In this case, you will want access to the spinalcordmri.org domain, which will grant you access to the Spinalcordmri.org Dashboard.
Next, on the dashboard, navigate to the Advanced DNS tab. Then, modify the following sections:
First you will want to make note of two things:
{subdomain}.spinalcordmri.org
, you will use {subdomain}
to identify your subdomain.{subdomain}.spinalcordmri.org
, the IP address was 142.93.152.255
.You can now use the red "Add New Record" button to add 3 new records in the following form:
Type | Host | Value | TTL |
---|---|---|---|
A Record | {subdomain} | 142.93.152.255 | Automatic |
TXT Record | {subdomain} | v=spf1 a mx ip4:142.93.152.255 ~all | Automatic |
TXT Record | _dmarc.{subdomain} | v=DMARC1; p=none | Automatic |
These entries accomplish the following:
{subdomain}.spinalcordmri.org
subdomain to the DigitalOcean droplet's IP address.You can double-check the current values by running the following dig
commands from an external device, not the server itself:
user@device:~$ dig +short {subdomain}.spinalcordmri.org
142.93.152.255
user@device:~$ dig +short TXT {subdomain}.spinalcordmri.org
"v=spf1 a mx ip4:159.89.119.65 include:spf.efwd.registrar-servers.com ~all"
user@device:~$ dig +short TXT _dmarc.{subdomain}.spinalcordmri.org
"v=DMARC1; p=none"
NB: Please note that it may take up to a few hours for changes to these records to propagate. So, if you're running
dig
and see no change, that's why!
Next, scroll down to the "Mail Settings" section and find the table containing MX records. Then, add the following entry:
Type | Host | Mail Server | Priority | TTL |
---|---|---|---|---|
MX Record | {subdomain} | {subdomain}.spinalcordmri.org. | 0 | Automatic |
Again, this setting will help us later when we set up our mail server. (There's not much here to talk about, since details such as 'priority' are only really relevant for setups with multiple mail servers.)
You can double-check the current value of this record by running the following command:
user@device:~$ dig +short MX {subdomain}.spinalcordmri.org
0 {subdomain}.spinalcordmri.org.
Now that NameCheap is configured, we can set up the server itself, starting with its hostname.
You can connect to the server either through SSH (assuming your local device contains the SSH key you added earlier):
user@device:~$ ssh root@{subdomain}.spinalcordmri.org
Welcome to Ubuntu 23.10 (GNU/Linux 6.5.0-13-generic x86_64)
Last login: Fri Dec 9 23:06:11 2023 from 162.243.188.66
root@forum:~#
Or, you can navigate to your droplet's main page, then click the "Console" button in the top-left corner:
From here, you can make changes to the server itself.
NB: If the initial DNS records haven't propagated yet, you can also SSH directly into the server using its IP address (
ssh root@142.93.152.255
) while you wait.
First, connect to the server using either SSH or the built-in DigitalOcean console. Next, edit two files using the text editor of your choice:
root@forum:~# sudo nano /etc/hostname
{subdomain}.spinalcordmri.org
root@forum:~# sudo nano /etc/mailname
{subdomain}.spinalcordmri.org
Additionally, edit the /etc/hosts
file to ensure that the hostname of the server ({subdomain}.spinalcordmri.org
) maps to the server's permanent IP address instead of 127.0.1.1
.
root@forum:~# sudo nano /etc/hosts
# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.debian.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
# /etc/cloud/cloud.cfg or cloud-config from user-data
#
142.93.152.255 {subdomain}.spinalcordmri.org
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Reboot the server by running reboot
, then reconnect and ensure that the following command also returns the same domain name as above:
root@forum:~# hostname
{subdomain}.spinalcordmri.org
We run a small mail server on the same server as Discourse for it to send notifcations and password resets. Discourse recommends using a cloud service like MailGun or Amazon SES or SendGrid, but our usage is so small that the overhead (and risk) of outsourcing is high. Mail servers are something of an arcane art now, but never fear, these instructions will make it work.
First, we must install our mail server software of choice, opensmtpd
.
# apt-get install -y opensmtpd
Replace the existing contents of /etc/smtpd.conf
with the following:
pki {subdomain}.spinalcordmri.org cert "/var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.cer"
pki {subdomain}.spinalcordmri.org key "/var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.key"
listen on eth0 tls-require pki {subdomain}.spinalcordmri.org
listen on eth0 tls-require pki {subdomain}.spinalcordmri.org auth port 587
table aliases file:/etc/aliases
action "local_mail" maildir "~/.mail" alias <aliases>
match for local action "local_mail"
action "relay_mail" relay helo "{subdomain}.spinalcordmri.org"
match from auth for any action "relay_mail"
Edit the following file:
sudo nano /etc/systemd/system/multi-user.target.wants/opensmtpd.service
And make the following change:
-ExecStart=/usr/sbin/smtpd
+ExecStart=/usr/sbin/smtpd -v
We need an SMTP account that Discourse can send mail via. opensmtpd
simply uses the OS's users by default, so we will make an OS user for outgoing emails.
forum@{subdomain}.spinalcordmri.org
using the following commands:
useradd -s /usr/sbin/nologin forum
: Creates the user account.passwd forum
: Sets the password for the account. Paste the password you generated earlier.NB: This username is not the same as what's on the email headers, because
opensmtpd
allows authenticated users to spoof their identities, and we want to send asnoreply@{subdomain}.spinalcordmri.org
.
Connect to the droplet server provided by Digital Ocean, then do:
wget -qO- https://get.docker.com/ | sh
mkdir /var/discourse
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
Increase timeout values
Before we can install Discourse, we must make a small tweak to the default Discourse Docker container template.
Open the file /var/discourse/samples/standalone.yml
and add the following lines to the SMTP section:
## TODO: The SMTP mail server used to validate new accounts and send notifications
# SMTP ADDRESS, username, and password are required
# WARNING the char '#' in SMTP password can cause problems!
DISCOURSE_SMTP_ADDRESS: smtp.example.com
#DISCOURSE_SMTP_PORT: 587
DISCOURSE_SMTP_USER_NAME: user@example.com
DISCOURSE_SMTP_PASSWORD: pa$$word
+ DISCOURSE_SMTP_OPEN_TIMEOUT: 60
+ DISCOURSE_SMTP_READ_TIMEOUT: 60
#DISCOURSE_SMTP_ENABLE_START_TLS: true # (optional, default true)
#DISCOURSE_SMTP_DOMAIN: discourse.example.com # (required by some providers)
#DISCOURSE_NOTIFICATION_EMAIL: noreply@discourse.example.com # (address to send notifications from)
The default values for these options are 5
seconds, which is too strict for our (slower) internal mail server. So, we must increase the timeout to avoid Net::ReadTimeout
errors.
./discourse-setup
Hostname : {subdomain}.spinalcordmri.org
Email : [initial administrator's email address -- will be used for 1st admin account creation]
SMTP address : {subdomain}.spinalcordmri.org
SMTP port : 587
SMTP username : forum
SMTP password : xxxxxxxxxxxxxxxxxxxxxxx
Notification : noreply@{subdomain}.spinalcordmri.org
Let's Encrypt : neuropoly-admin@liste.polymtl.ca
Maxmind License : [Enter]
This will download the Discourse base image, then build the image and initialize the Docker container.
Before continuing, make sure that Discourse has generated its SSL certs. (They will be generated automatically, so long as you provide an email address for Let's Encrypt.)
The files exist in /var/discourse/shared/standalone/ssl/
:
root@forum:~# ls -l /var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.{cer,key}
-rw-r--r-- 1 root root 3799 Dec 5 08:33 /var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.cer
-rw------- 1 root root 3247 Dec 5 08:33 /var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.key
These files must be present for opensmtpd
to be able to send mail correctly, as per the previous /etc/smtpd.conf
configuration we pasted in during a previous step.
Once you've confirmed that the SSL certs are present, you can start the mail server with the following command:
systemctl enable --now opensmtpd
You should be able to connect to the mail server by running:
root@forum:~# nc {subdomain}.spinalcordmri.org 587
220 {subdomain}.spinalcordmri.org ESMTP OpenSMTPD
To create the first admin account on the newly-installed Discourse forum, email delivery must be working, because Discourse will need to send out a confirmation email.
To test email delivery, rather than sending messages to specific personal addresses, we use https://www.mail-tester.com/. This page is very helpful for finding issues missed during setup, especially around spamminess. Email is very very complicated and this helps a lot.
Go to https://www.mail-tester.com/ and copy the email address it gives you. You can use this address alongside one of three testing methods:
Simple test (mail
)
You will first need to install Mailutils using sudo apt-get install mailutils
. Then, you can run:
echo "Test Message" | mail -s "This is a message" somethingsomething@mail-tester.com
Complex test (swaks
)
You will first need to install the "Swiss Army Knife for SMTP" (swaks
) using sudo apt-get install swaks
. Then, you can run:
# You will need to enter the password that you set during the previous "User Setup" section!!
swaks --to me@example.com --from noreply@{subdomain}.spinalcordmri.org --server {subdomain}.spinalcordmri.org -p 587 --auth-user forum --tls-verify --tls
Discourse-specific test
First, cd
into /var/discourse
. Then, run the following command:
./discourse-doctor
It will run some automated tests, then it will prompt you for an email to send a test message to.
_NB: There is a possibility that you may encounter a
Net::ReadTimeout
error. In that case, edit the/var/discourse/containers/app.yml
file to increase the values ofDISCOURSE_SMTP_OPEN_TIMEOUT
andDISCOURSE_SMTP_READ_TIMEOUT
. Then, restart the docker container by runningcd /var/discourse
,./launcher destroy app
, and./launcher start app
._
With each of these tests, your goal here is to ensure that:
Once you send your test message, you can click "View my results" on the mail-tester webpage. It will give you feedback on things that need to be changed. So, try to iteratively address the feedback, send a new test message, and refresh the page until you get a high score.
Navigate to the page https://{subdomain}.spinalcordmri.org/, and follow the instruction to create an admin email account. Discourse will send you a confirmation email. If you've set up mail correctly, you should receive the email and be able to finish making your account.
Once you log in for the first time, you will be presented with an onboarding flow that will ask for things like "Community name" and "Point of contact". Feel free to put some sensible defaults (e.g. "Spinalcordmri.org", "neuropoly-admin@liste.polymtl.ca", etc.). However, if you will be loading a previous backup of the forum, then these settings will be overwritten anyway, making them moot.
If there is a currently-running instance of the forum, you can download backups from the Backups page.
Next, you need to upload this backup to the forum. You can try using the Backups page on the dev forum, however sometimes uploads will fail inexplicably with (due to the large size of the backups). So, you may need to instead use FileZilla to transfer files to the Digital Ocean Droplet. Specifically, the backup will need to be uploaded to the /var/discourse/shared/standalone/backups/default
folder.
Finally, you will need to enable the "allow restore" setting on the dev server's Settings page. Then, you can return to the Backups page and restore the uploaded backup.
Currently, the only mandatory plugin in use by the forum is discourse-solved
. To install it, please follow the instructions from the Install Plugins in Discourse documentation page.