Docker Compose Based Development Environment

conatus commented 7 years ago

This allows you to run: docker-compose up and have a working development environment on your machine. Or follow the instructions in the README.md for a bit more detail.

@chriscroome have a go with this and see if it works for you.

I'll then add in the piece which updates the images programmatically.

conatus commented 7 years ago

@nicksellen

Would be cool if you took a look too.

chriscroome commented 7 years ago

This is great :-) There were a few small issues like I needed to do a recursive chown, but it seems to work really well!

conatus commented 7 years ago

@chriscroome What recursive chown is needed mate? Need to add this in.

chriscroome commented 7 years ago

This is the error I had:

~/cotech/website/web/app/themes/coop-tech-oowp-theme$ npm install
npm ERR! Linux 4.9.0-3-amd64
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v6.11.0
npm ERR! npm  v3.10.10
npm ERR! path /home/chris/cotech/website/web/app/themes/coop-tech-oowp-theme/node_modules/.staging
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall mkdir

npm ERR! Error: EACCES: permission denied, mkdir '/home/chris/cotech/website/web/app/themes/coop-tech-oowp-theme/node_modules/.staging'
npm ERR!     at Error (native)
npm ERR!  { Error: EACCES: permission denied, mkdir '/home/chris/cotech/website/web/app/themes/coop-tech-oowp-theme/node_modules/.staging'
npm ERR!     at Error (native)
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/home/chris/cotech/website/web/app/themes/coop-tech-oowp-theme/node_modules/.staging' }
npm ERR! 
npm ERR! Please try running this command again as root/Administrator.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/chris/cotech/website/web/app/themes/coop-tech-oowp-theme/npm-debug.log

I'm sorry I don't recall exactly which directory was www-data:www-data -- I was trying to get it all up and running quickly, I can repeat the process from scratch on another machine when I get a chance -- I simply did a chown chris:chris -R ~/cotech/website/ to fix it.

chriscroome commented 7 years ago

I have installed everything from scratch on a Debian Stretch machine and documented what I did on the wiki, when I got to the tasks in the post-merge git hooks script I hit the permissions issue:

composer update
Loading composer repositories with package information

Could not fetch https://api.github.com/repos/outlandishideas/routemaster/contents/composer.json?ref=master, please create a GitHub OAuth token to go over the API rate limit
Head to https://github.com/settings/tokens/new?scopes=repo&description=Composer+on+XXX+2017-06-28+2138
to retrieve a token. It will be stored in "/home/chris/.composer/auth.json" for future use by Composer.
Token (hidden): 
Token stored successfully.
Updating dependencies (including require-dev)                        
Nothing to install or update

  [ErrorException]                                                                                                        
  file_put_contents(/home/chris/cotech/website/vendor/composer/installed.json): failed to open stream: Permission denied  

update [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [--with-dependencies] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [-i|--interactive] [--root-reqs] [--] [<packages>]...

It is the website/vendor directory that is owned by www-data:

ls -lah /home/chris/cotech/website
total 208K
drwxr-xr-x  8 chris    chris    4.0K Jun 28 21:20 .
drwxr-xr-x  4 chris    chris    4.0K Jun 22 07:55 ..
-rw-r--r--  1 chris    chris    3.8K Jun 22 07:55 CHANGELOG.md
-rw-r--r--  1 chris    chris    2.9K Jun 28 21:40 composer.json
-rw-r--r--  1 chris    chris    107K Jun 28 21:03 composer.lock
drwxr-xr-x  3 chris    chris    4.0K Jun 22 07:55 config
drwxr-xr-x  5 chris    chris    4.0K Jun 28 21:03 docker
-rw-r--r--  1 chris    chris    1.2K Jun 28 21:03 docker-compose.yml
-rw-r--r--  1 chris    chris     663 Jun 22 07:55 .env.example
drwxr-xr-x  8 chris    chris    4.0K Jun 28 21:31 .git
-rw-r--r--  1 chris    chris     127 Jun 28 21:03 .gitattributes
drwxr-xr-x  2 chris    chris    4.0K Jun 28 21:30 git-hooks
-rw-r--r--  1 chris    chris     325 Jun 22 07:55 .gitignore
-rw-r--r--  1 chris    chris     35K Jun 22 07:55 LICENSE.md
-rw-r--r--  1 chris    chris    1.5K Jun 28 21:03 README.md
drwxr-xr-x 22 www-data www-data 4.0K Jun 28 21:19 vendor
drwxr-xr-x  5 chris    chris    4.0K Jun 28 21:18 web
-rw-r--r--  1 chris    chris      13 Jun 22 07:55 wp-cli.yml

So a sudo chown -R ${SUDO_USER}:${SUDO_USER} solves it... but perhaps there is another way that would be cross platform compatible?

conatus commented 7 years ago

@chriscroome

I haven't changed anything about the Git hooks stuff here, see the diff, so if you are having problems its not the result of this merge request which is only about Docker Compose and doesn't touch these files.

chriscroome commented 7 years ago

if you are having problems its not the result of this merge request

The only issue I have raised is the ownership of the vendor directory and it was you who suggested that something needed to be done about this, I don't believe that running composer update caused the ownership issue, it simply exposed it. I have documented how to reproduce this on Debian Stretch.

chriscroome commented 7 years ago

Just to add that without running the following tasks after running docker-compose up -d you have a site with no CSS or images, so I think we should document that these things need to be run, together with a chown, or am I misunderstanding something here?

composer update
cd web/app/themes/coop-tech-oowp-theme
composer update
npm install
nodejs ./node_modules/gulp/bin/gulp.js

These are the things that are run when the live or dev site are updated.

nicksellen commented 7 years ago

@conatus so, a summary of my changes:

move image downloading to assets container (as they are assets)
download images only if uploads/2017 does not exist (+ option to force a new download, see readme)
hopefully fix yarn/node-sass issue (make it explicit dep in package.json and run yarn add --force node-sass)
remove instructions from readme about composer/gulp outside of container
make node image use explicit version so it is deterministic (:latest will change over time)

nicksellen commented 7 years ago

And final change:

the wordpress entrypoint script will now check if there is composer.json in the theme dir and run composer install there if so

nicksellen commented 7 years ago

@chriscroome all this should be addressed now, just to make it clear where they each happen:

composer update

This is to install composer deps from root dir, runs in wordpress docker entrypoint script each time container is started. Uses composer install to use versions pinned from composer.lock.

cd web/app/themes/coop-tech-oowp-theme composer update

This is also run in wordpress docker entrypoint by checking for a composer.json in the theme directory (as described in previous comment). Also runs each time container is started.

npm install

This is run (although using yarn) in the assets container now, and runs each time the container is started. Defined in docker-compose.yml.

nodejs ./node_modules/gulp/bin/gulp.js

This also runs in the assets container and is the command for that container. Defined in docker-compose.yml (as gulp watch though, so gets compiled on change).

nicksellen commented 7 years ago

@chriscroome and to clarify - all this docker stuff only applies for development environments, perhaps production deployment using docker could be looked into in the future, but this is not addressed here at all.

nicksellen commented 7 years ago

@chriscroome I just read https://wiki.coops.tech/wiki/CoTech_WordPress#Debian_Stretch_dev_env - no need to install anything other than docker on your dev machine. Everything else (composer, nodejs, php, php extensions, etc) is installed inside docker (this is why it's so useful!).

So, if you feel like it you can do a fresh install and skip the Install Nodejs, Install Composer and PHP modules, and Run Composer steps.

conatus commented 7 years ago

@chriscroome

If you could look to @nicksellen's instructions - i.e. you don't need to install anything apart from Docker - it would be really cool to get this reviewed as soon as possible. Thanks a lot mate!

chriscroome commented 7 years ago

Sorry not to have had the time and energy to looks at this sooner.

So, removing the www-data:www-data owned vendor directory and the unneeded packages and restarting Docker and updating the code:

sudo rm -rf vendor/
sudo aptitude remove nodejs docker-compose php7.0-mbstring php7.0-xml php7.0-curl
sudo service docker restart
git pull

Checking with docker ps and there are no containers running, so tab completion suggestions for docker run have some options, none of which seen to work?

docker run website_wordpress
Starting up entrypoint script
Composer could not find a composer.json file in /var/www/html
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section

docker run adamyeats/docker-nginx-hhvm-bedrock
error: missing DB_HOST and MYSQL_PORT_3306_TCP environment variables
  Did you forget to --link some_mysql_container:mysql or set an external db
  with -e DB_HOST=hostname:port?

Sorry I'm unclear how to start the docker container in order to be able to do things like running composer in the container? I guess I'm missing something obvious?

chriscroome commented 7 years ago

I seem to be getting somewhere:

docker run wordpress

Results in a WordPress web install running at http://172.17.0.2/wp-admin/setup-config.php

chriscroome commented 7 years ago

Is docker-compose installed in a container or do I need to install that locally in addition to docker-ce?

conatus commented 7 years ago

Install locally. Then things should just work. Its the only command you need, normal use of Docker isn't required.

On Mon, 3 Jul 2017 22:13 Chris Croome, notifications@github.com wrote:

Is docker-compose installed in a container https://docs.docker.com/compose/install/#install-as-a-container or so I need to install that locally in addition to docker-ce?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cotech/website/pull/41#issuecomment-312734691, or mute the thread https://github.com/notifications/unsubscribe-auth/AATZJgDikpuXvSvum1bdCdzb6XXg-65vks5sKVl2gaJpZM4OFhom .

nicksellen commented 7 years ago

@chriscroome

docker run wordpress

This command downloads and starts the official wordpress docker image from docker hub [1]. It doesn't relate to the cotech docker-compose setup.

docker-compose often comes bundled with docker setups as it's an official tool now, but you might need to install it separately. Anything version >=1.6 should be sufficient.

To update an existing docker-compose installation you may need to rebuild the images, you can add the ---build args to the up command, so:

docker-compose up -d --build

If nothing changed it won't actually do much (you'll see lots of ---> Using cache) so it doesn't hurt to run it like that.

The README should hopefully contain enough instructions anyway :) Let me know if I can help more, feel free to ping me on cotech slack too.

[1] https://hub.docker.com/_/wordpress/

chriscroome commented 7 years ago

Thanks for you patience, it is up and running now and I have updated the wiki, Debian Stretch has docker-compose (1.8.0-2) and that seems to work OK.

cotech / website

Docker Compose Based Development Environment #41