alphagov/static fails to start when starting alphagov/publisher

alphagov / govuk-docker

GOV.UK development environment using Docker 🐳

MIT License

82 stars 22 forks source link

alphagov/static fails to start when starting alphagov/publisher #237

Closed connorgurney closed 5 years ago

connorgurney commented 5 years ago

When trying to start alphagov/publisher using the command govuk-docker-up, the start-up process proceeds until recreating alphagov/static. It fails with the following output:

Recreating govuk-docker_static-app_1 ... error

ERROR: for govuk-docker_static-app_1  Cannot start service static-app: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"bin/rails\": stat bin/rails: no such file or directory": unknown

ERROR: for static-app  Cannot start service static-app: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"bin/rails\": stat bin/rails: no such file or directory": unknown
ERROR: Encountered errors while bringing up the project.

Any idea where to start with resolving this?

benthorner commented 5 years ago

Have you tried running make publisher in govuk-docker (after pulling)? It sounds like the static repo hasn't been setup.

connorgurney commented 5 years ago

Hi Ben,

Many thanks, Ben—just tried to run it now. It succeeds down to setting up alphagov/calendars tries to connect to the E2E alphagov/publishing-api stack:

/Users/Connor/govuk/govuk-docker/exe/govuk-docker run calendars-setup bin/rake publishing_api:publish
Starting govuk-docker_postgres_1        ... done
Starting govuk-docker_rabbitmq_1        ... done
Starting govuk-docker_mongo_1           ... done
Starting govuk-docker_nginx-proxy-app_1 ... done
Starting govuk-docker_router-app_1       ... done
Starting govuk-docker_router-app-draft_1 ... done
Starting govuk-docker_router-api-app-draft_1 ... done
Starting govuk-docker_router-api-app_1       ... done
Starting govuk-docker_content-store-app-draft_1 ... done
Starting govuk-docker_content-store-app_1       ... done
Starting govuk-docker_publishing-api-worker_1   ... done
Starting govuk-docker_publishing-api-app-e2e_1  ... done
rake aborted!
GdsApi::HTTPInternalServerError: URL: http://publishing-api.dev.gov.uk/v2/content/58f79dbd-e57f-4ab2-ae96-96df5767d1b2
Response body:
Puma caught this error: File exists @ dir_s_mkdir - /govuk (Errno::EEXIST)
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:239:in `mkdir'
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:239:in `fu_mkdir'
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:217:in `block (2 levels) in mkdir_p'
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:215:in `reverse_each'
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:215:in `block in mkdir_p'
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:200:in `each'
/home/build/.rbenv/versions/2.6.5/lib/ruby/2.6.0/fileutils.rb:200:in `mkdir_p'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/application.rb:597:in `generate_development_secret'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/application.rb:430:in `secret_key_base'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/application.rb:253:in `env_config'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:706:in `build_request'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/application.rb:608:in `build_request'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:523:in `call'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/puma-4.2.1/lib/puma/configuration.rb:228:in `call'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/puma-4.2.1/lib/puma/server.rb:667:in `handle_request'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/puma-4.2.1/lib/puma/server.rb:470:in `process_client'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/puma-4.2.1/lib/puma/server.rb:328:in `block in run'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/puma-4.2.1/lib/puma/thread_pool.rb:134:in `block in spawn_thread'

Request body:
{:title=>"UK bank holidays", :description=>"Find out when bank holidays are in England, Wales, Scotland and Northern Ireland - including past and future bank holidays", :base_path=>"/bank-holidays", :document_type=>"calendar", :schema_name=>"calendar", :publishing_app=>"calendars", :rendering_app=>"calendars", :locale=>"en", :details=>{:body=>""}, :public_updated_at=>"2019-10-30T15:59:54+00:00", :routes=>[{:type=>"prefix", :path=>"/bank-holidays"}], :update_type=>"minor"}
/govuk/calendars/app/services/calendar_publisher.rb:7:in `publish'
/govuk/calendars/lib/tasks/publishing_api.rake:6:in `block (3 levels) in <top (required)>'
/govuk/calendars/lib/tasks/publishing_api.rake:4:in `each'
/govuk/calendars/lib/tasks/publishing_api.rake:4:in `block (2 levels) in <top (required)>'

Caused by:
RestClient::InternalServerError: 500 Internal Server Error
/govuk/calendars/app/services/calendar_publisher.rb:7:in `publish'
/govuk/calendars/lib/tasks/publishing_api.rake:6:in `block (3 levels) in <top (required)>'
/govuk/calendars/lib/tasks/publishing_api.rake:4:in `each'
/govuk/calendars/lib/tasks/publishing_api.rake:4:in `block (2 levels) in <top (required)>'
Tasks: TOP => publishing_api:publish
(See full trace by running task with --trace)
make: *** [calendars] Error 1

Really quite confused as to why ~/govuk existing is an issue... Any ideas?

connorgurney commented 5 years ago

Just as a heads-up, I've also tried to run build calendars and receive an identical error. However, the Publishing API URL works when accessed via a browser, so the 500 error seems weird.

benthorner commented 5 years ago

I'm afraid I can't reproduce the error you're having, although it's possible they're related. Can you post the output govuk-docker run static-lite ls /govuk and govuk-docker run calendars-setup ls /govuk?

connorgurney commented 5 years ago

Thanks, Ben. Per your request:

govuk-docker run static-lite ls /govuk:

docker-compose -f [...] run static-lite ls /govuk
calendars      govuk-content-schemas  publisher       router      static
content-store  govuk-docker       publishing-api  router-api

govuk-docker run calendars-setup ls /govuk

Starting govuk-docker_postgres_1        ... done
Starting govuk-docker_postgres_1 ... 
Starting govuk-docker_rabbitmq_1 ... done
Starting govuk-docker_redis_1           ... done
Starting govuk-docker_nginx-proxy-app_1 ... done
Starting govuk-docker_router-app-draft_1 ... done
Starting govuk-docker_router-app_1       ... done
Starting govuk-docker_router-api-app-draft_1 ... done
Starting govuk-docker_router-api-app_1       ... done
Starting govuk-docker_content-store-app-draft_1 ... done
Starting govuk-docker_content-store-app_1       ... done
Starting govuk-docker_publishing-api-worker_1   ... done
Starting govuk-docker_publishing-api-app-e2e_1  ... done
calendars      govuk-content-schemas  publisher       router      static
content-store  govuk-docker       publishing-api  router-api

benthorner commented 5 years ago

Hmmm...that seems OK. Can you try a few more things:

govuk-docker run static-lite ls
govuk-docker run publishing-api-e2e ls /govuk
govuk-docker run publishing-api-e2e ls

It might also be worth retrying your original command to start publisher, as the build issue you're getting might be completely unrelated and less valuable to fix.

connorgurney commented 5 years ago

static-lite returns nothing, while both publishing-api-e2e commands return ERROR: No such service: publishing-api-e2e. The Publishing API has been cloned and seems to all be in order within ~/govuk.

benthorner commented 5 years ago

Aha, so that suggests that ~/govuk isn't getting mounted properly inside the containers. I expect you'll get the same answer with (corrected) govuk-docker run publishing-api-app-e2e ls.

I'm not sure what's going wrong here. By default, everything in ~/govuk should be available under /govuk in containers. Have you got GOVUK_ROOT_DIR set in your environment? If so, that might be the cause of the problem.

connorgurney commented 5 years ago

Surprisingly, the corrected command works just fine:

docker-compose -f [...] run publishing-api-app-e2e ls
Starting govuk-docker_rabbitmq_1        ... done
Starting govuk-docker_postgres_1        ... done
Starting govuk-docker_redis_1           ... done
Starting govuk-docker_mongo_1           ... done
Starting govuk-docker_nginx-proxy-app_1 ... done
Starting govuk-docker_router-app-draft_1 ... done
Starting govuk-docker_router-app_1           ... done
Starting govuk-docker_router-api-app-draft_1 ... done
Starting govuk-docker_router-api-app_1       ... done
Starting govuk-docker_content-store-app_1       ... done
Starting govuk-docker_content-store-app-draft_1 ... done
Starting govuk-docker_publishing-api-worker_1   ... done
CONTRIBUTING.md  Jenkinsfile  Rakefile  config     docker_tests.sh  startup.sh
Dockerfile   LICENSE      app   config.ru  lib          tmp
Gemfile      Procfile     bench db     log
Gemfile.lock     README.md    bin   doc    spec

Glad I'm not the only one who's baffled! I don't have GOVUK_ROOT_DIR set, so it should default to $HOME/govuk, I believe?

connorgurney commented 5 years ago

Recloned everything and tried again—make publisher works fine, but govuk-docker-up outputs:

docker-compose -f [...] up publisher-app
govuk-docker_postgres_1 is up-to-date
govuk-docker_nginx-proxy-app_1 is up-to-date
govuk-docker_redis_1 is up-to-date
govuk-docker_mongo_1 is up-to-date
Starting govuk-docker_link-checker-api-worker_1 ... 
Starting govuk-docker_static-app_1              ... 
govuk-docker_publishing-api-app_1 is up-to-date
Starting govuk-docker_link-checker-api-worker_1 ... done
Starting govuk-docker_static-app_1              ... error
Starting govuk-docker_content-store-app_1       ... done
Starting govuk-docker_link-checker-api-app_1    ... 

ERROR: for govuk-docker_static-app_1  Cannot start service static-app: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"bin/rails\": stat bin/rails: no sStarting govuk-docker_link-checker-api-app_1    ... done

ERROR: for static-app  Cannot start service static-app: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"bin/rails\": stat bin/rails: no such file or directory": unknown
ERROR: Encountered errors while bringing up the project.

I've tried running govuk-docker run static-lite ls /govuk which returns the same as ls ~/govuk, while govuk-docker run static-lite ls returns nothing.

connorgurney commented 5 years ago

Following that, I've tried make static which returns the following output:

docker-compose -f [...] build static-lite
Building static-lite
Step 1/16 : FROM buildpack-deps
 ---> 2389ea5f6378
Step 2/16 : RUN apt-get update -qq && apt-get install -y libxss1 libappindicator1 libindicator7
 ---> Using cache
 ---> 72de72db6215
Step 3/16 : RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb 2>&1 &&    apt install -y ./google-chrome*.deb &&     rm ./google-chrome*.deb
 ---> Using cache
 ---> d86403cfe641
Step 4/16 : RUN curl -sL https://deb.nodesource.com/setup_12.x | bash -
 ---> Using cache
 ---> 800f63ad8c7f
Step 5/16 : RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
 ---> Using cache
 ---> b25481c04b90
Step 6/16 : RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
 ---> Using cache
 ---> bb9fefcd1ae2
Step 7/16 : RUN apt-get update -qq && apt-get install -y yarn nodejs
 ---> Using cache
 ---> 189ab172f977
Step 8/16 : RUN export VERSION=2.1.1 &&     wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-$VERSION-linux-x86_64.tar.bz2 2>&1 &&     tar -vxf phantomjs-$VERSION-linux-x86_64.tar.bz2 &&     cp -v phantomjs-$VERSION-linux-x86_64/bin/phantomjs /usr/local/bin/phantomjs &&     rm -vr phantomjs-$VERSION-linux-x86_64
 ---> Using cache
 ---> 9a1a2c0abc82
Step 9/16 : RUN git clone https://github.com/sstephenson/rbenv.git /rbenv
 ---> Using cache
 ---> 30da94ca8039
Step 10/16 : RUN git clone https://github.com/sstephenson/ruby-build.git /rbenv/plugins/ruby-build
 ---> Using cache
 ---> c81bc28cf3a7
Step 11/16 : RUN /rbenv/plugins/ruby-build/install.sh
 ---> Using cache
 ---> aae07079b27a
Step 12/16 : ENV PATH /rbenv/bin:$PATH
 ---> Using cache
 ---> b5f1bd7519c5
Step 13/16 : RUN apt-get update -qq && apt-get install -y postgresql-client
 ---> Using cache
 ---> 9441a5718efa
Step 14/16 : RUN useradd -m build
 ---> Using cache
 ---> a0aefcbb5a01
Step 15/16 : ENV PATH /home/build/.rbenv/shims:${PATH}
 ---> Using cache
 ---> eb4b2f24dedb
Step 16/16 : USER build
 ---> Using cache
 ---> 40bcb70543ad

Successfully built 40bcb70543ad
Successfully tagged static:latest
/Users/Connor/govuk/govuk-docker/exe/govuk-docker run static-lite rbenv install -s
docker-compose -f [...] run static-lite rbenv install -s
Usage: rbenv install [-f|-s] [-kpv] <version>
       rbenv install [-f|-s] [-kpv] <definition-file>
       rbenv install -l|--list
       rbenv install --version

  -l/--list          List all available versions
  -f/--force         Install even if the version appears to be installed already
  -s/--skip-existing Skip if the version appears to be installed already

  ruby-build options:

  -k/--keep          Keep source tree in $RBENV_BUILD_ROOT after installation
                     (defaults to $RBENV_ROOT/sources)
  -p/--patch         Apply a patch from stdin before building
  -v/--verbose       Verbose mode: print compilation status to stdout
  --version          Show version of ruby-build

For detailed information on installing Ruby versions with
ruby-build, including a list of environment variables for adjusting
compilation, see: https://github.com/rbenv/ruby-build#usage

make: *** [bundle-static] Error 1

benthorner commented 5 years ago

OK. I'm glad make publisher works OK. I was really struggling to work out what was going on there, but it seems unrelated to the static issue, for which I think I know the cause.

I think some previous build of static failed on your machine. Normally running make would fix the issue, but a few small bugs in the repo mean that hasn't happened.

I'll look at raising PRs to fix the bugs, but in the meantime I think you can make progress by doing rm -r ~/govuk/static and then make static in govuk-docker.

connorgurney commented 5 years ago

make static seems to now run fine, but I'm now confronted with an error about MongoDB which I suspect is unrelated to this.

Starting govuk-docker_mongo_1 ... done
Traceback (most recent call last):
    21: from bin/rails:4:in `<main>'
    20: from bin/rails:4:in `require'
    19: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands.rb:18:in `<top (required)>'
    18: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/command.rb:46:in `invoke'
    17: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/command/base.rb:65:in `perform'
    16: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
    15: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
    14: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
    13: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands/runner/runner_command.rb:41:in `perform'
    12: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands/runner/runner_command.rb:41:in `eval'
    11: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands/runner/runner_command.rb:41:in `<main>'
    10: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/findable.rb:148:in `first'
     9: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual.rb:20:in `first'
     8: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:255:in `first'
     7: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:522:in `try_cache'
     6: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:257:in `block in first'
     5: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:257:in `first'
     4: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/query_cache.rb:222:in `each'
     3: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/collection/view/iterable.rb:39:in `each'
     2: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/retryable.rb:43:in `read_with_retry'
     1: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/collection/view/iterable.rb:40:in `block in each'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/server_selector/selectable.rb:115:in `select_server': No server is available matching preference: #<Mongo::ServerSelector::Primary:0x46936051702920 tag_sets=[] max_staleness=nil> using server_selection_timeout=30 and local_threshold=0.015 (Mongo::Error::NoServerAvailable)

Tried recloning alphagov/publisher and restarting Docker with no luck. Looks like Mongo is failing to start within the container, but unsure why and not sure how to debug further. Any ideas?

benthorner commented 5 years ago

Glad to hear static is better! Could you clarify what command you're running when you get this error?

connorgurney commented 5 years ago

Sorry, thought I'd mentioned it in my comment—I'm running make publisher following make static.

benthorner commented 5 years ago

Thanks @gurneyconnor. make runs quite a few things. Can you give a bit more of the terminal output before the error?

connorgurney commented 5 years ago

Sure thing, @benthorner — I think this explains in more detail:

/Users/Connor/govuk/govuk-docker/exe/govuk-docker run content-store-lite bin/rake db:create
docker-compose -f [...] run content-store-lite bin/rake db:create
Starting govuk-docker_mongo_1 ... done
/Users/Connor/govuk/govuk-docker/exe/govuk-docker run content-store-lite bin/rails runner 'User.first || User.create'
docker-compose -f [...] run content-store-lite bin/rails runner User.first || User.create
Starting govuk-docker_mongo_1 ... done
Traceback (most recent call last):
    21: from bin/rails:4:in `<main>'
    20: from bin/rails:4:in `require'
    19: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands.rb:18:in `<top (required)>'
    18: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/command.rb:46:in `invoke'
    17: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/command/base.rb:65:in `perform'
    16: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
    15: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
    14: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
    13: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands/runner/runner_command.rb:41:in `perform'
    12: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands/runner/runner_command.rb:41:in `eval'
    11: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-5.2.3/lib/rails/commands/runner/runner_command.rb:41:in `<main>'
    10: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/findable.rb:148:in `first'
     9: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual.rb:20:in `first'
     8: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:255:in `first'
     7: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:522:in `try_cache'
     6: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:257:in `block in first'
     5: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/contextual/mongo.rb:257:in `first'
     4: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongoid-6.2.1/lib/mongoid/query_cache.rb:222:in `each'
     3: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/collection/view/iterable.rb:39:in `each'
     2: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/retryable.rb:43:in `read_with_retry'
     1: from /home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/collection/view/iterable.rb:40:in `block in each'
/home/build/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/mongo-2.4.3/lib/mongo/server_selector/selectable.rb:115:in `select_server': No server is available matching preference: #<Mongo::ServerSelector::Primary:0x47425298430440 tag_sets=[] max_staleness=nil> using server_selection_timeout=30 and local_threshold=0.015 (Mongo::Error::NoServerAvailable)
make: *** [content-store] Error 1

benthorner commented 5 years ago

This looks like the content-store container is having trouble communicating with the mongo container. One reason for this could be the mongo container isn't starting properly: can you check its status with govuk-docker ps and govuk-docker logs mongo after running this command?

Another reason could be that you're using an outdated or modified version of the content-store repo - I'm not sure if you have had content-store cloned previously. Can you check that your local content-store repo is up-to-date with respect to the origin?

connorgurney commented 5 years ago

Hi Ben,

Sorry for the delay in responding to you—been away! I reset Docker to factory settings and all seems to work well.

However, if anyone comes across this in the future, it seemed to be an issue with files in /data/db of the MongoDB container being corrupted—presumably through force-closing the container. Documentation on how to repair the files from MongoDB.org might be helpful.

Thanks for all your help—you've been a star!

benthorner commented 5 years ago

No problem @gxrneyme! This issue identified a few things to fix, which will benefit all the developers on GOV.UK 🎉