search

mattermost / mattermost-docker

Deprecated
Apache License 2.0
964 stars 575 forks source link

Upgrade Postgres to 10 for 5.33.0 #529

Closed EugenMayer closed 3 years ago

EugenMayer commented 3 years ago

fixes #528 #489

For anybody in need and avoiding the complexity, you can use the DB images from https://hub.docker.com/repository/docker/eugenmayer/mattermost-db build by https://github.com/EugenMayer/mattermost-boilerplate - you will find versions prior 5.33.0 based on pg 9.6 and versions above base on pg 10.

Those images are unmodified variants of this repo - just publish to docker-hub for obvious convenience reasons i guess :)

mattermod commented 3 years ago

Hello @EugenMayer,

Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

davidbartonau commented 3 years ago

@EugenMayer you're a ninja :-) Worked perfectly for me on Ubuntu 18.04, although wow it's a slow build!

Is it worth adding a warning to the documentation? Upgrading the postgres version is not a "safe" operation and requires the user to do some work to backup and restore their files.

This is the procedure I followed:

# We need to delete the database files, start a fresh postgres and then restore our backup.
# Backup your postgres database.  Go to the docker-compose root.
export MM_DOCKER_DB=`/usr/bin/docker ps --filter ancestor=mattermost-docker_db --format "{{.Names}}"`
echo Docker container is ${MM_DOCKER_DB}
docker container exec -i ${MM_DOCKER_DB} /usr/local/bin/pg_dump -c -U mmuser -h localhost mattermost > /tmp/mattermost.sql

# Take down docker
docker-compose down

# Move the files ... just to be sure
mv volumes/db volumes/db.old
mkdir volumes/db

# Start just the database
docker-compose up -d db 

# Check it started
export MM_DOCKER_DB=`/usr/bin/docker ps --filter ancestor=mattermost-docker_db --format "{{.Names}}"`
docker logs ${MM_DOCKER_DB}

# Restore our database
docker container exec -i ${MM_DOCKER_DB} /usr/local/bin/psql -U mmuser -h localhost mattermost < /tmp/mattermost.sql

# Check the restore has some data
docker container exec -i ${MM_DOCKER_DB} /usr/local/bin/psql -U mmuser -h localhost mattermost -c "SELECT count(*) FROM audits;" 

# Bring up the other services
docker-compose up -d
EugenMayer commented 3 years ago

You can test the image from https://github.com/EugenMayer/mattermost-boilerplate which i use to build the mattermost images and publish them, since i cannot understand why mattermost is not doing exactly that.

Using a pg 10 image is using eugenmayer/mattermost-db:team-5.33.2 if you do not want to build it.

I used https://github.com/tianon/docker-postgres-upgrade to migrate from 9.6 to 10.

It works for me and i already run it production now

craph commented 3 years ago

Any news on the availability of Postgres 10 ?

davidbartonau commented 3 years ago

@craph manually edit the dockerfile per the pull request and then do a build. That should have you sorted until the powers that be merge it.

I did that and it works, EXCEPT you must upgrade your container manually. My instructions above work, or you can follow @EugenMayer instructions

mattermod commented 3 years ago

This PR has been automatically labelled "stale" because it hasn't had recent activity. A core team member will check in on the status of the PR to help with questions. Thank you for your contribution!

/cc @jasonblais @jfrerich @emilyacook

Stasmo commented 3 years ago

In its current state, the docker deployment in this project is not usable until this PR is merged. This project is currently broken.

Stasmo commented 3 years ago

Probably a different topic, but it looks like the tests that travis is running aren't capturing this issue.

craph commented 3 years ago

When this PR will be merged ? it seems there is no automatic way to upgrade MM to latest version .. @stylianosrigas ? @emilyacook do you have any idea ?

cobenash commented 3 years ago

I faced the same situation too and look forward on this pull request. :)

craph commented 3 years ago

@hanzei @cpanato @metanerd is it possible to have a status about this PR ? The community needs an official and automatic way to upgrade Mattermost docker.

Thank you very much for all your work.

Best regards,

cpanato commented 3 years ago

@craph sorry for the delay, I'm preparing some announcements for this repository and my personal opinion is we should not have a custom DB build to run MM the normal image should do the work

but those changes looks good to me will merge it

EugenMayer commented 3 years ago

@craph sorry for the delay, I'm preparing some announcements for this repository and my personal opinion is we should not have a custom DB build to run MM the normal image should do the work

but those changes looks good to me will merge it

Thanks for merging.

I hope you do not mind the honesty, the current repository is really far of the standard an best practices. Not only the packaged Postgres DB without any reason, but even without publishing the actual image anywhere but letting people build them themself over and over again.

Happy to join a discussion though, maybe this can be ironed out! :)

cpanato commented 3 years ago

it is, and I know, but this repo is was a community effort and we don't officially support those things here. I will make an announcement and make this official again because in the past was like this but got lost :/

I recommend not use the mattermost-prod-app and instead of that use mattermost/mattermost-team-edition or mattermost/mattermost-enterprise-edition those ones we support and you can find it in the mattermost-server repo.

for the other images like DB and proxy, we don't use those custom ones and we use the default images or managed DB from AWS or any other cloud provider.

If you want to help to maintain this repository I'm glad to help you to onboard and discuss next steps

cpanato commented 3 years ago

the images for custom DB and others as I said we don't officially give support and that is the reason we don't push those images, if you want to use you can build them and push them to your own registry.

craph commented 3 years ago

@craph sorry for the delay, I'm preparing some announcements for this repository and my personal opinion is we should not have a custom DB build to run MM the normal image should do the work

but those changes looks good to me will merge it

Hello @cpanato the repository mattermost/mattermost-docker is not officially maintained ?

Is it possible to use mattermost/mattermost-team-edition with docker or k8s and do you have any documentation for that ?

It is possible to help to maintain this repo ?

Best regards,

cpanato commented 3 years ago

no, and was maintained by me and others as you can see https://github.com/mattermost/mattermost-docker/blob/master/MAINTENANCE.md in the past when I was not an employee.

our mattermost-operator and mattermost-helm repos uses the image from mattermost/mattermost-team-edition.

I'm open to chat with in any time we can make a zoom meeting if you want :)

EugenMayer commented 3 years ago

the images for custom DB and others as I said we don't officially give support and that is the reason we don't push those images, if you want to use you can build them and push them to your own registry.

@cpanato as you can see here https://github.com/EugenMayer/mattermost-boilerplate I'am already doing this https://hub.docker.com/r/eugenmayer/mattermost-app for quiet a while now.

I also maintain a rancher catalog https://github.com/EugenMayer/docker-rancher-extra-catalogs/tree/master/templates/mattermost for mattermost.

What is mattermost/mattermost-team-edition? It seems neither to be a docker hub image nor a repository, or did I fail to find it?

cpanato commented 3 years ago

@EugenMayer thanks for maintaining those projects!

you can find the repositories here:

for example

docker pull mattermost/mattermost-team-edition:5.35.1

craph commented 3 years ago

@cpanato sorry for the delay but I have a question.

You said that this repository is not officially maintained but in the official documentation, I see this repository mentionned when we want to install Mattermost in Production with Docker. Here is the link :

  1. https://github.com/mattermost/mattermost-server#install-on-your-own-machine
  2. https://docs.mattermost.com/install/prod-docker.html#docker-setup-on-ubuntu

Is that normal ?

Moreover, you mentionned https://hub.docker.com/repository/docker/mattermost/mattermost-team-edition and I can see that this image is based on mattermost-server.

So my question is : what is the best way to install Mattermost with Docker ?

And why the dockerfile mentionned in mattermost-team-edition in dockerhub is the same for mattermost-enterprise-edition ? https://github.com/mattermost/mattermost-server/blob/master/build/Dockerfile

cpanato commented 3 years ago

@craph thanks the links above is interesting because I thought that was already removed (cc @svelle @mrckndt @cwarnermm @jasonblais ) will work with my colleagues to remove this references

mattermost-team-edition and mattermost-enterprise-edition use the same dockerfile, but the only difference is the package we use to download, if is the team edition we download the team edition package.

My colleagues are working to publish a new repository that will show how to run MM with docker and other services as well

craph commented 3 years ago

@craph thanks the links above is interesting because I thought that was already removed (cc @svelle @mrckndt @cwarnermm @jasonblais ) will work with my colleagues to remove this references

mattermost-team-edition and mattermost-enterprise-edition use the same dockerfile, but the only difference is the package we use to download, if is the team edition we download the team edition package.

My colleagues are working to publish a new repository that will show how to run MM with docker and other services as well

Thank you very much @cpanato for your reply.

But why your colleagues are creating a new repository ? This repository is called mattermost-docker, the name is not correct ? will you be able to give us the new repository ?

Moreover about migrating from pg 9 to pg 10 I was thinking it could be a great idea to do the same as https://github.com/mattermost/mattermost-docker#upgrading-to-team-edition-30x-from-2x but I'm unable to see what is done by the option -upgrade_db_30 where I can see what is done by this argument ? I say that because of this new issue : https://github.com/mattermost/mattermost-docker/issues/540

About mattermost-team-edition and mattermost-enterprise-edition, I can't see the difference about the package you use to download. The link in the Dockerfile are no difference that's why I mention that.

craph commented 3 years ago

@cpanato here also you have the old link regarding your comment : https://docs.mattermost.com/install/ee-install.html#installing-enterprise-edition

craph commented 3 years ago

My colleagues are working to publish a new repository that will show how to run MM with docker and other services as well

@cpanato, @svelle , @mrckndt , @cwarnermm , @jasonblais do you have the link of the new repository ? And does this repository will explain how to have pg10 or higher with the latest version of MM ?

Thank you very much

jasonblais commented 3 years ago

@craph Yes, we are in the process of publishing the new repository soon. We're conducting internal reviews (including security reviews) after which we'll publish it and open it for the community to try and test it out. It will support postgres 10. We also plan to have migration guides for those who are using the Docker image in this repository.

As for the Docker instructions in our documentation (https://docs.mattermost.com/install/prod-docker.html#docker-setup-on-ubuntu), it is an unofficial guide maintained by the community.

We definitely acknowledge the confusion here, and hope to resolve this confusion soon.

@cpanato Regarding the different packages downloaded for TE and EE docker images, can we add a link or description to the DockerHub pages that explains this? Since I agree with @craph that these currently look identical.

craph commented 3 years ago

@jasonblais Thank you very much for your reply.

Please, will you be able to give the link of the new repo here when it will be available ?

We also plan to have migration guides for those who are using the Docker image in this repository.

It's a good news. I'm using this repository for my Mattermost in production :)

If you want, I'll be available to test the migration guides ;)

Best regards,

jasonblais commented 3 years ago

@craph

Please, will you be able to give the link of the new repo here when it will be available ?

Absolutely. We're hoping to open the new repository soon (in a few weeks) for initial testing and feedback from the community, stay tuned!

craph commented 3 years ago

@jasonblais any news about the new repositories ?

Thank you very much.

jasonblais commented 3 years ago

@craph we're actively working on it, we're doing some internal testing and security reviews at the moment after which we'll open the new repo for community testing

EugenMayer commented 3 years ago

Sorry for bumping this, but since we talked a bunch about the official images, some quick notes from my side:

It seems fairly easy beside:

All in one migrating to pg10 and the official image works just fine, i just wished it would all be a lot better documented :)

Thanks for giving us the team-edition official builds, very appritiated!

jasonblais commented 3 years ago

@EugenMayer glad to hear it helped!

@craph the new repository is now public! https://github.com/mattermost/docker. It is still a work-in-progress, and we recommend testing a fresh set up first. If you decide to migrate to the new image, please make sure to have full data backups as we haven't fully tested the migration process yet, so there may be unforeseen issues until those tests have been completed. If any questions or feedback, let us know!

wiersgallak commented 3 years ago

For anyone else following along, please see also see this note for more information and context of the solution.

craph commented 3 years ago

@jasonblais thank you very much for the update. I will test it soon and keep you posted if I have some issues.

craph commented 3 years ago

Hello @jasonblais ,

Sorry for the time without no news about my tests. After reviewing the code in the docker-compose.yml I think there is something missing. The 13 may 2020 I fix an issue about self-signed certificate : https://github.com/mattermost/mattermost-docker/pull/474 for this issue : https://github.com/mattermost/mattermost-server/issues/13059

I think, this is missing in the new repository. Can I reopen a PR for this ?

Thank you very much.