[Help wanted]: testing Debian Jessie deployment instructions

[it33]

We have a new community guide for deploying Mattermost to Debian in a production configuration. It's difficult for the core team to test a very broad set of configurations, so we're asking for help from the community to test.

If you're following these instructions anyway, please share feedback on any issues or corner cases on this thread, and we can update the guide to reflect that it's working for multiple community deployments.

Instructions here: http://docs.mattermost.com/install/prod-debian.html

[reach3r]

Hi, I just deployed a "soon-to-be trial for future production" Mattermost installation on Debian 7 Wheezy. I basically used the Ubuntu guide and changed things where necessary (opted for the /opt installation). After reading through the Debian guide it ended up basically re-telling what I did, except that I used a different username. Also the init.d script came in handy, as I had not created one so far. After changing the username it worked fine.

To summarize:

• I did as the Debian guide says for the Postgres and Mattermost setup (we have a dedicated Loadbalancer) both on the same machine
• mail was set up as well (again, with a dedicated server already in place) and works fine
• everything went well and it works without any Problems so far

[yuvipanda]

I tested this as well but without the load balancer (there is only one instance, so I don't know why a load balancer was suggested?) nor email. Worked pretty well. I used a systemd unit instead of the init script, shall I send a pull request with the unit file?

[jhaubrich]

These instructions worked for me on Debian 8.1 on Linode. There were a few things I noticed.

• Postgres default IP should probably be 127.0.0.1 in this line: "postgres://mmuser:mmuser_password@10.10.10.1:5432/mattermost?sslmode=disable&connect_timeout=10" It isn't mentioned until the nginx section that 10.10.10.1 was assumed to be the default IP. This causes step 27, running ./platform, to fail.
• The init script doesn't start for me. It starts and exits. Haven't looked into that yet.
• There is a mismatched triple tick (```) at the end of the nginx config file that messes up it's formatting and all subsequent code blocks

[lloeki]

I did an install by myself on Jessie and cooked up the following systemd unit (usable for non-Debian system too, such as Arch Linux):

# cat /etc/systemd/system/mattermost.service 
[Unit]
Description=Mattermost
After=network.target

[Service]
User=mattermost
ExecStart=/home/mattermost/mattermost/bin/platform
WorkingDirectory=/home/mattermost/mattermost
Restart=always
RestartSec=30

[Install]
WantedBy=multi-user.target
# systemctl start mattermost
# systemctl enable mattermost

I also used the following as a postgres data source using unix sockets instead of TCP:

    "SqlSettings": {
        "DriverName": "postgres",
        "DataSource": "host=/var/run/postgresql dbname=mattermost connect_timeout=10",

Here is my nginx config (name changed to example.com):

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name mattermost.adhoc-gti.com;

    ssl on;
    ssl_certificate /etc/ssl/certs/example.com.pem;
    ssl_certificate_key /etc/ssl/private/example.com.key;
    ssl_session_timeout 5m;
    ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
    ssl_prefer_server_ciphers on;

    location / {
        gzip off;
        proxy_set_header X-Forwarded-Ssl on;
        client_max_body_size 50M;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header   X-Frame-Options   SAMEORIGIN;
        proxy_pass http://localhost:8065;
    }
}

I also have the following for redirecting to https:

server {
    listen 80;
    listen [::]:80 default ipv6only=on;
    server_name *.example.com;
    rewrite ^(.*) https://$host$1 permanent;
}

[it33]

Hi @lloeki, @yuvipanda, @jhaubrich, @reach3r, thanks for the feedback and the help!

Hmm... it sounds like the instructions are working, and there's additional suggestions and improvements that could be made.

I'm kind of wondering what a best practice might be to help maintain the growing number of install guides from the community to make sure they're working and to incorporate improvements and updates.

Any projects you've seen that have a process you think we could adopt here?

[AndiHD]

Hi, tried to install this on one single machine. Database server is working but step 7 in setting up mattermost server (using 127.0.0.1 in config.json) causing neither or:

di@cubietruck:/opt/mattermost/bin$ sudo ./platform ./platform: 1: ./platform: ELF: not found ./platform: 1: ./platform: Syntax error: word unexpected (expecting ")")

di@cubietruck:/opt/mattermost/bin$ ./platform -bash: ./platform: cannot execute binary file: Exec format error

Same for Ubuntu and for Debian, tried both! (postgresql 9.4 at Debian, 9.3 at Ubuntu)

Any idea??

Solution: Does not support ARM architecture.......

[it33]

@esethna, could we have your help seeing how we can update the Debian install guide given feedback here so we can close out this issue?

[armouredking]

I've setup a MM instance on a LXC with Debian 8 under Proxmox with the instructions provided, modified to use an in-place setup with HAProxy-keepalived load balancers ( which serve up the MM instance instead of NGINX ) and a MariaDB galera cluster. I'm still hunting down an oddball go error ( mentioned with other appliances, seems to be something with the go driver and MySQL? ) with the MariaDB cluster but I don't think it's install related and the instance works from me and my friends testing of it ( just keeps throwing log entries ) so it's a low priority atm to figure out that one.

What I would contribute to the install is asking what the point of the load balancer is? There is no setup related to it except the NGINX bit, is that supposed to be the load balancer part? I'm kinda assuming that must be it, given the IP address consideration of .1 as DB, .2 as MM, and .3 as NGINX and three virtuals were noted at the beginning of the guide.

I must admit I'm curious, can MM even be used as HA in it's current setup? I mean, I could set it up and just try it out, but my gut is telling me that currently just sharing a DB is not going to be good enough to get two instances talking to one another and the guide doesn't go into anything more with regards to that. It's probably a better idea not to call it a load balancer for the purposes of this guide at any rate given only one server install covered.

[mindvox]

Help!

I get to section 7. Run the MatterMost server by typing ./platform and I get the following.

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x6198b0]

goroutine 1 [running]:
github.com/mattermost/platform/utils.LoadConfig(0xc8200cb680, 0x22)
    /go/src/github.com/mattermost/platform/utils/config.go:152 +0x6a0
main.main()
    /go/src/github.com/mattermost/platform/mattermost.go:54 +0x43

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 5 [runnable]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:20
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 6 [runnable]:
github.com/alecthomas/log4go.(*ConsoleLogWriter).run(0xc82000e600, 0x7f7a5392f1e8, 0xc820026010)
    /go/src/github.com/mattermost/platform/Godeps/_workspace/src/github.com/alecthomas/log4go/termlog.go:32
created by github.com/alecthomas/log4go.NewConsoleLogWriter
    /go/src/github.com/mattermost/platform/Godeps/_workspace/src/github.com/alecthomas/log4go/termlog.go:26 +0xb5

I am running Debian Jessie amd64 with Mattermost 2.0.0 :smile:

EDIT Moved to issue #2308

[TwizzyDizzy]

Just recently installed the latest 2.1 release of mattermost on Debian Jessie - running on a postgres DB because Debian Jessie only has MySQL 5.5, which does not support Fulltext Search Indizes which leads to a failing database initialization on MySQL 5.5 - 5.6 from debian stretch/testing, however, does work though I will not recommend to do that). No flaws detected whatsoever.

Some feedback on the guide itself:

• GRANT ALL PRIVILEGES
  • would be nice to know what is actually needed, than just granting all privileges
• INIT-Script
  • this is NOT, i repeat, NOT the way to go an a vanilla Debian Jessie installation
  • systemd unit file is to be preferred
  • Init-Script for installations without systemd (talking of upgraded systems (from wheezy) without activating systemd or the likes of Devuan)
• it should be stressed, that all salts from [..]/config/config.json (for example AtRestEncryptKey) should be changed after installation. This has grave security implications.

I'm currently running on one of these servers with lots of different stuff running on it. The mattermost installation currently only hosts 10 users but there is no sign of that installation not being capable of hosting 100 or more users.

Cheers Thomas

[esethna]

@TwizzyDizzy thanks for testing the guide and posting your feedback! Would you be interested in submitting a PR to the guide on the docs repo to address the issues you've mentioned?

[ScottMansfield]

I tried using the init.d version and it failed to do anything. I used the systemd unit but the docs show directories inconsistent with the rest of the guide. I'm pretty much in the same boat as @TwizzyDizzy with the init script.

I'd recommend figuring out a single-box guide and a multi-box guide if a user is going that route. Starting "getting started" instructions with procuring three separate VMs is a bit odd. A single-box guide would have unix socket communication with postgres at least and can have a companion "split your install into separate boxes" guide if you're thinking about HA or higher load.

My systemd unit:

# cat mattermost.service 
[Unit]
Description=Mattermost
After=network.target

[Service]
User=mattermost
ExecStart=/opt/mattermost/bin/platform
WorkingDirectory=/opt/mattermost
Restart=always
RestartSec=30

[Install]
WantedBy=multi-user.target

[TwizzyDizzy]

@esethna yeah, sure! From tomorrow I'm on vacation for a week but I guess there is no hurry. Maybe a mattermost DEV could, until then, elaborate on the DB privileges that are needed so I can integrate that :)

@ScottMansfield I guess running a single instance is the usual use case of mattermost. I think the installation guides should stay as they are and a new chapter could be added to cover a load-balanced setup (I guess we are currently speaking of How to scale mattermost frontends, not the DB backend, which may then be yet another chapter). I would find that interesting, that's for sure! People who think about load balancing are usually able to abstract from the general setup guide anyway. See #2466 :)

I think the load balanced setup should rather be covered in a separate issue because the one at hand is for debian jessie.

Cheers so far Thomas

[ilTofa]

I just installed it on a fresh Debian 8 (a linode1GB VPS). I used the guide changing things for a signle machine installation (that should be the default, I agree with other people).

The application works if started by hand. The init.d don't work.

Activating a systemd unit like in @ScottMansfield sample fails at startup. Nothing in mattermost.log in both cases (the application fails before initing the log system probably).

> sudo systemctl status mattermost
● mattermost.service - Mattermost
   Loaded: loaded (/etc/systemd/system/mattermost.service; enabled)
   Active: activating (auto-restart) (Result: exit-code) since Tue 2016-04-05 18:54:08 CEST; 14s ago
  Process: 3651 ExecStart=/opt/mattermost/bin/platform (code=exited, status=2)
 Main PID: 3651 (code=exited, status=2)

Apr 05 18:54:08 mattermost systemd[1]: mattermost.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Apr 05 18:54:08 mattermost systemd[1]: Unit mattermost.service entered failed state.
Apr 05 18:54:08 mattermost platform[3651]: /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/config.go:170 +0xdb5
Apr 05 18:54:08 mattermost platform[3651]: main.main()
Apr 05 18:54:08 mattermost platform[3651]: /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:55 +0x48 

[ilTofa]

Just a further note... starting the application with nohup crashes it, it could be related to the systemd start problem. It happens both as root or as a user in the mattermost group.

The application starts with nohup only if started from /opt/mattermost/bin with ./platform (this should probably be a different bug), make me know if you want me to open another issue (and thank you).

panic: Failed to load system translations for 'en'

goroutine 1 [running]:
panic(0xb2e0c0, 0xc8200eb7e0)
        /usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/mattermost/platform/utils.GetTranslationsBySystemLocale(0xc8200eb693)
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/i18n.go:44 +0x18b
github.com/mattermost/platform/utils.InitTranslations()
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/i18n.go:29 +0x2e9
main.main()
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:54 +0x2c
panic: Failed to load system translations for 'en'

goroutine 1 [running]:
panic(0xb2e0c0, 0xc8200e97d0)
        /usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/mattermost/platform/utils.GetTranslationsBySystemLocale(0xc8200e9683)
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/i18n.go:44 +0x18b
github.com/mattermost/platform/utils.InitTranslations()
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/i18n.go:29 +0x2e9
main.main()
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:54 +0x2c
panic: Failed to load system translations for 'en'

goroutine 1 [running]:
panic(0xb2e0c0, 0xc8200eb800)
        /usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/mattermost/platform/utils.GetTranslationsBySystemLocale(0xc8200eb6b3)
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/i18n.go:44 +0x18b
github.com/mattermost/platform/utils.InitTranslations()
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/i18n.go:29 +0x2e9
main.main()
        /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:54 +0x2c

[yuki]

@ilTofa I have had the same problem you described when starting with the systemd unit. I have solved deleting the log /opt/mattermost/logs/mattermost.log because it was generated by user root (when testing the application). When trying to execute the systemd unit with user mattermost, it can write on it and it failed.

After deleting the log file, I could start using systemctl:

yukim:~# systemctl start mattermost yukim:~# systemctl status mattermost ● mattermost.service - Mattermost Loaded: loaded (/etc/systemd/system/mattermost.service; enabled) Active: active (running) since mié 2016-04-13 13:43:27 CEST; 15min ago Main PID: 5703 (platform) CGroup: /system.slice/mattermost.service └─5703 /opt/mattermost/bin/platform

abr 13 13:57:05 yukim platform[5703]: [2016/04/13 13:57:05 CEST] [DEBG] /api/v1/users/status

Hope you could solve the same way.

[michaelgrilo]

I'm attempting to install Mattermost to a DigitalOcean 512MB virtual machine following the Production Install on Debian Jessie guide. I haven't even been able to get passed the postgresql test in Step 13.

psql: could not connect to server: Connection timed out
    Is the server running on host "10.10.10.1" and accepting
    TCP/IP connections on port 5432?

On a fresh vm, do I need to manually change my IP from 127.0.0.1 to 10.10.10.1 before setting up the database?

netstat -nltp | grep 5432
tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN      422/postgres

If I edit the postgresql.conf to include the second line here:

host    all             all             127.0.0.1/32            md5
host    all             postgres        127.0.0.1/32            md5

After running /etc/init.d/postgresql restart, I now get returned the desired mattermost=>.

Is it more desirable to be on an internal IP of 10.10.10.1?

[obones]

@ilTofa: I have the exact same problem when using the init.d file. I changed MATTERMOST_ROOT and DAEMON so that the /bin is inside MATTERMOST_ROOT but still have no luck starting the script.

Did you manage to work around this issue?

[ilTofa]

No, @obones I didn't have time to test, manually starting the program was enough for my scope (testing).

[obones]

Well, I finally found what the issue was and sure enough, it was my own fault. There was an extra space at the very beginning of the script, right before #!/bin/sh With this, the script is not recognized and does not start. Removing the space led to a perfectly functioning script, without even having to modify MATTERMOST_ROOT.

[TwizzyDizzy]

@esethna I finally got around to creating a pull request for the documentation, concerning my input in https://github.com/mattermost/platform/issues/1185#issuecomment-197070465.

For the database grants I opened an issue to find out what privileges are actually needed (https://github.com/mattermost/platform/issues/2998). As soon as I know, I'll send another pull request.

As for now, here are the changes I made: https://github.com/mattermost/docs/pull/161

Cheers Thomas

[lfbrock]

Thanks @TwizzyDizzy, @esethna is away until next week - in the meantime, if anyone would like to help review the PR that would be great.

[kevinlong206]

Same issue, if you blindly copy and paste the init.d script from the guide there is whitespace in the first line that causes the script to fail when run by systemd and it actually is not obvious from the error messages what the problem is I recommended removing those spaces in #!/bin/bash ASAP as to not wate people's time thank you for the guide though.

[tux-00]

Chapter : Set up Database Server (section10 and 11) Maybe we should indicate that the path of the configuration file could change depending on the version of postgres. In debian 8.4 the version is 9.4.

Chapter : Set up Database Server (section12) systemctl reload postgresql instead ?

Chapter : Set up Mattermost Server (section2) When I tried to download mattermost with wget I get an certificate error. I resolved this issue using --no-check-certificate parameter.

Chapter : Set up Mattermost Server (section 8 / Set up systemd with a unit file)

• Replace : sudo vi /etc/init.d/mattermost with sudo vi /etc/systemd/system/mattermost.service
• Replace : Copy the following lines into /etc/init.d/mattermost with Copy the following lines into /etc/systemd/system/mattermost.service
• as @yuki said, I needed to delete the log file to get mattermost started by systemd.

[Cyrillius]

I have installed mattermost on an OVH VPS with debian jessie. Here is my suggestion:

Setup Database # 10

Indicate that the path of the postgres can change wih the version

Set up Mattermost Server # 5

Indicate after this step that the user need to disconnect and reconnect to apply the change.

Set up NGINX with SSL # 1

https://github.com/letsencrypt/letsencrypt doesn't exist anymore

use instead certbot https://certbot.eff.org/

to install on jessie:

$ wget https://dl.eff.org/certbot-auto $ chmod a+x ./certbot-auto $ ./certbot-auto --help

or:

$ sudo nano /etc/apt/sources.list

add this line: deb http://http.debian.net/debian wheezy-backports main

$ sudo apt-get update $ sudo apt-get install certbot -t jessie-backports $ sudo certbot certonly

[eadmund]

The 'Finish Mattermost Server setup' section states, 'Change Local Directory Location from ./data/ to /mattermost/data'; the directory which was actually created is /opt/mattermost/data.

[dantrevino]

Set Up Mattermost Server

Section #8 all of the systemctl commands need sudo for both the "unit file" and "init script" sections:

• sudo systemctl daemon-reload
• sudo systemctl enable mattermost
• sudo systemctl start mattermost

[dunatotatos]

Same problem as : https://github.com/mattermost/platform/issues/1185#issuecomment-209396385 (solved on the same way) BTW, it makes the init script start, and exit without error. But mattermost does not run after that.

And now, I finished the installation, but the home page is blank. Firebugs gives me this error : TypeError: window.setup_root is not a function

[ghost]

Hi there, i got an issue while i´m setting up Mattermost on Debian 8. i follow the guide till "Set up NGINX Server" without any complications, but at "curl http://localhost" i got an error 502 instead of "Mattermost - Signup". did anyone know how to fix that? i allready post that issue at the mattermost forum: http://forum.mattermost.org/t/debian-8-nginx-502-error/2045/13

[itg-dave]

Hi everyone, with a strange mix of the solutions mentioned by @yuki , howtoforge (yes it's originally written for CentOS but it helped anyways; especially the permissions/root part) and the certbot docs I was able to get my setup to production state.

But now I'm running into the next issue with strange permission errors. Maybe it's nginx-related? When I try to upload a file to a channel I get this error: APP: Encountered an error creating the directory for the new file LOG:

[2016/09/04 20:33:08 UTC] [EROR] /api/v3/teams/qgpahforo3r1insws5kq7rce7e/files/upload:WriteFile code=500 rid=guccbuctyid1iy4z5nt7sjhhyr uid=uxemjqtsr3ndtqn9bnyphwrjho ip=***** Encountered an error creating the directory for the new file [details: directory=/mattermost/data/teams/qgpahforo3r1insws5kq7rce7e/channels/obux1fi4uif9f8pdskin3xbmha/users/uxemjqtsr3ndtqn9bnyphwrjho/bbfgrfmzbjr9m8mmpunruz6g8c, err=mkdir /mattermost: permission denied]

It's kinda strange because the permissions of the mattermost folder are set up correct:

Local storage location is /mattermost/data/

Any solutions or hints are really appreciated!

UPDATE

Well leaving the storage location blank fixes this issue...?

@flix-wedo I have added a reply to your thread. Hope it helps you. If anyone else is interested I can compare the steps to the guide for additions.

[pydubreucq]

Hi, I've made my install with root user, and If I follow the install guide, I can't start mattermost via systemd. I have this error :

sept. 27 13:38:08 debian systemd[1]: Started Mattermost. sept. 27 13:38:08 debian platform[7332]: FileLogWriter("/opt/mattermost/logs/mattermost.log"): open /opt/mattermost/logs/mattermost.log: permission denied

It's possible to don't have this problem if you use the mattermost user to run server like this :

https://github.com/mattermost/docs/commit/6f9fa9ba33fa19e8d3cf8284aa3aa5783b8fea06

Best Regards

[Rudloff]

A quick feedback here: The doc says that we need to install MySQL 5.6+ but as it has already been said the stable Debian version is 5.5 so there is no clean way to install 5.6 on Debian stable. However it seems to work correctly with 5.5 if I switch some of the tables (Channels, Posts and Users, I think?) to MyISAM.

[danielsreichenbach]

Quick tip for those wanting Debian Jessie: you can install MariaDB client and server there instead of the provided MySQL 5.5 client and server, and it works fine. I am running this for a few weeks in production myself.

[ionas]

Raspberrian / Raspberry Pi 3 Model B

I was able to install MySQL 5.7 and even a recent docker version and gcc-6/g++-6 on my Pi but I could not get mattermost installed. Coming from php-world I assumed it was a webserver software and could easily run on my Pi. Now the Pi is sitting there waiting for Mattermost to run on arm (64-Bit-ARMv8-CPU).

Any chance this will happen anytime soon? Else there is little reason to keep it around as I purchased it just to run a mattermost server on a fixed internet connection to be able to manage team collaboration in software development.

[jasonblais]

We have new install guides available here: https://docs.mattermost.com/install/install-debian-88.html

If there is any feedback, let us know in the forums here: https://forum.mattermost.org/t/help-testing-debian-jessie-deployment-instructions/3674

Thanks all for the wonderful comments, it made help write up the docs. Also a big thanks to @acgustafson for the effort.