search

RefugeRestrooms / refugerestrooms

REFUGE restrooms indexes and maps safe restroom locations for trans, intersex, and gender nonconforming individuals.
http://www.refugerestrooms.org
GNU Affero General Public License v3.0
891 stars 263 forks source link

Use a volume to deal with bundled gems #640

Closed brunoocasali closed 3 years ago

brunoocasali commented 3 years ago

Context

Summary of Changes

Checklist

DeeDeeG commented 3 years ago

I looked at this, and I can confirm there are cases where this makes docker-compose up work (when running with a Gemfile and Gemfile.lock that have been modified after the last time the user ran docker-compose build). So there is certainly some value to this approach.

I'm still trying to make sure there are no downsides, but it's difficult to find time to thoroughly test this. I'm trying to prioritize some other pull requests that impact the quality or features of the site itself, although this could be an improvement to the developer experience if it works well.

Particularly, I want to investigate if this has any chance of making docker-compose up not work when it would have without this volume. (And to a lesser extent, I want to be sure this volume isn't likely to become extremely large on disk, though this seems unlikely and the size should be reasonable. Maybe even smaller than having multiple "throwaway" volumes from doing docker-compose run, which are frequently created and then not used again.)

brunoocasali commented 3 years ago

Hi @DeeDeeG I've created an image that shows what this PR does:

image

In the first terminal I've ran a down command just to clean up everything, then I open a docker-compose run, as you can see no volumes where created. In the second terminal, the same command was executed. And the last terminal I've check if there are multiple volumes but just one was found!

This kind of approach is nice because we can just ran a bundle install in any open terminal of the app and everything will work, without this every change we must to regenerate the docker image. Even if we ran multiple times the docker-compose up, everytime will ask about the gems, so you are obligated to regenerate :D.

About the size of the volume, excluding the possibility of the docker image having a better compression than the volume option, the size must be the same in both ;)

DeeDeeG commented 3 years ago

@brunoocasali I need to test whether the volume with the gems can be made inadequate to launch the server.

For example:

  1. Build the Docker container docker-compose build
  2. Run the Docker container docker-compose up
    1. Now the volume with gems in it has been created.
  3. Edit the Gemfile/Gemfile.lock manually, outside of Docker
    1. Example: Update Rails, or update Puma version
  4. Question: Does Docker now fail to start when running docker-compose up/docker-compose run web [command]?
  5. Do the fix that would have worked before: docker-compose build
  6. Same question again: Does Docker fail to start? (docker-compose up/docker-compose run web [command])

I haven't gotten around to testing this yet, sorry. But I mean to test that first before merging this.

If step 6 that I described above gives errors, then this improves one scenario but makes another scenario worse, and I will have to weigh the benefits in one case against the regression in the other case.

DeeDeeG commented 3 years ago

Ah, okay. I just tested this...

The change in setup/entry is a good idea. Much more flexible than what we had before. And takes good advantage of the caching this Pull Request adds.

The following output is what happened when I rebuilt with a newer Ruby version (docker-compose build) and then ran an instance of bash in the container (docker-compose run --rm web bash):

Long command-line output, click to expand: ```console % docker-compose run --rm web bash Creating refugerestrooms_web_run ... done Warning: the running version of Bundler (1.17.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`. The following gems are missing * nokogiri (1.10.10) * nio4r (2.5.4) * websocket-driver (0.7.3) * ffi (1.12.2) * sassc (2.2.1) * bcrypt (3.1.13) * debug_inspector (0.0.3) * binding_of_caller (0.8.0) * msgpack (1.3.3) * bootsnap (1.4.6) * json (2.3.0) * pg (1.2.3) * puma (5.0.4) Install missing gems with `bundle install` Ignoring bcrypt-3.1.13 because its extensions are not built. Try: gem pristine bcrypt --version 3.1.13 Ignoring binding_of_caller-0.8.0 because its extensions are not built. Try: gem pristine binding_of_caller --version 0.8.0 Ignoring bootsnap-1.4.6 because its extensions are not built. Try: gem pristine bootsnap --version 1.4.6 Ignoring debug_inspector-0.0.3 because its extensions are not built. Try: gem pristine debug_inspector --version 0.0.3 Ignoring ffi-1.12.2 because its extensions are not built. Try: gem pristine ffi --version 1.12.2 Ignoring json-2.3.0 because its extensions are not built. Try: gem pristine json --version 2.3.0 Ignoring msgpack-1.3.3 because its extensions are not built. Try: gem pristine msgpack --version 1.3.3 Ignoring nio4r-2.5.4 because its extensions are not built. Try: gem pristine nio4r --version 2.5.4 Ignoring nokogiri-1.10.10 because its extensions are not built. Try: gem pristine nokogiri --version 1.10.10 Ignoring pg-1.2.3 because its extensions are not built. Try: gem pristine pg --version 1.2.3 Ignoring puma-5.0.4 because its extensions are not built. Try: gem pristine puma --version 5.0.4 Ignoring puma-5.0.2 because its extensions are not built. Try: gem pristine puma --version 5.0.2 Ignoring sassc-2.2.1 because its extensions are not built. Try: gem pristine sassc --version 2.2.1 Ignoring websocket-driver-0.7.3 because its extensions are not built. Try: gem pristine websocket-driver --version 0.7.3 Warning: the running version of Bundler (1.17.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`. Ignoring bcrypt-3.1.13 because its extensions are not built. Try: gem pristine bcrypt --version 3.1.13 Ignoring binding_of_caller-0.8.0 because its extensions are not built. Try: gem pristine binding_of_caller --version 0.8.0 Ignoring bootsnap-1.4.6 because its extensions are not built. Try: gem pristine bootsnap --version 1.4.6 Ignoring debug_inspector-0.0.3 because its extensions are not built. Try: gem pristine debug_inspector --version 0.0.3 Ignoring ffi-1.12.2 because its extensions are not built. Try: gem pristine ffi --version 1.12.2 Ignoring json-2.3.0 because its extensions are not built. Try: gem pristine json --version 2.3.0 Ignoring msgpack-1.3.3 because its extensions are not built. Try: gem pristine msgpack --version 1.3.3 Ignoring nio4r-2.5.4 because its extensions are not built. Try: gem pristine nio4r --version 2.5.4 Ignoring nokogiri-1.10.10 because its extensions are not built. Try: gem pristine nokogiri --version 1.10.10 Ignoring pg-1.2.3 because its extensions are not built. Try: gem pristine pg --version 1.2.3 Ignoring puma-5.0.4 because its extensions are not built. Try: gem pristine puma --version 5.0.4 Ignoring puma-5.0.2 because its extensions are not built. Try: gem pristine puma --version 5.0.2 Ignoring sassc-2.2.1 because its extensions are not built. Try: gem pristine sassc --version 2.2.1 Ignoring websocket-driver-0.7.3 because its extensions are not built. Try: gem pristine websocket-driver --version 0.7.3 Fetching gem metadata from https://rubygems.org/......... Using rake 13.0.1 Using concurrent-ruby 1.1.7 Using builder 3.2.4 Using erubi 1.9.0 Using mini_portile2 2.4.0 Using crass 1.0.6 Using rack 2.2.3 Fetching nio4r 2.5.4 Using websocket-extensions 0.1.5 Using mini_mime 1.0.2 Using formtastic_i18n 0.6.0 Using method_source 1.0.0 Using arel 9.0.0 Fetching ffi 1.12.2 Using tilt 2.0.10 Using public_suffix 4.0.3 Using ast 2.4.1 Using execjs 2.7.0 Fetching bcrypt 3.1.13 Using coderay 1.1.2 Fetching msgpack 1.3.3 Using bundler 1.17.2 Using orm_adapter 0.5.0 Using regexp_parser 1.7.0 Using diff-lcs 1.3 Using dotenv 2.2.2 Using dry-equalizer 0.3.0 Using dry-inflector 0.2.0 Using geocoder 1.6.3 Using ruby2_keywords 0.0.2 Using temple 0.8.2 Using hashdiff 1.0.1 Using high_voltage 3.0.0 Using http_accept_language 2.1.1 Fetching json 2.3.0 Using mimemagic 0.3.5 Using rb-fsevent 0.10.3 Using ruby_dep 1.5.0 Using parallel 1.19.2 Fetching pg 1.2.3 Using docile 1.3.2 Using rainbow 3.0.0 Using rakismet 1.5.4 Using rdoc 6.2.1 Using rexml 3.2.4 Using rspec-support 3.9.2 Using ruby-progressbar 1.10.1 Using unicode-display_width 1.7.0 Using semantic_range 2.3.0 Using simplecov-html 0.10.2 Using turbolinks-source 5.2.0 Using i18n 1.8.5 Fetching nokogiri 1.10.10 Using cliver 0.3.2 Using rack-test 1.1.0 Fetching websocket-driver 0.7.3 Using coffee-script-source 1.12.2 Using mail 2.7.1 Using sprockets 3.7.2 Using addressable 2.7.0 Using autoprefixer-rails 9.7.5 Using better_errors 2.4.0 Using bugsnag 6.13.0 Using warden 1.2.8 Using dry-core 0.4.9 Using rack-accept 0.4.5 Using parser 2.7.2.0 Using pry 0.13.0 Using rack-cors 1.1.1 Using rack-jsonp 1.3.1 Using rack-proxy 0.6.5 Using uglifier 4.2.0 Using mustermann 1.1.1 Using haml 5.2.0 Using marcel 0.3.3 Using i18n-debug 1.2.0 Using rspec-core 3.9.1 Using thread_safe 0.3.6 Using database_cleaner 1.8.3 Using rspec-mocks 3.9.1 Using sdoc 1.1.0 Using turbolinks 5.2.1 Using coffee-script 2.4.1 Using dry-configurable 0.11.5 Using dry-logic 1.0.6 Using rubocop-ast 0.7.1 Using mustermann-grape 1.0.1 Using rspec-expectations 3.9.1 Installing nio4r 2.5.4 with native extensions Using kaminari-core 1.2.1 Using dry-container 0.7.2 Using tzinfo 1.2.7 Installing msgpack 1.3.3 with native extensions Using dry-types 1.4.0 Using i18n_data 0.10.0 Installing bcrypt 3.1.13 with native extensions Using safe_yaml 1.0.5 Using minitest 5.14.2 Using crack 0.4.3 Using rubocop 0.92.0 Using activesupport 5.2.4.4 Using rubocop-rspec 1.43.2 Using globalid 0.4.2 Using rubocop-rails 2.8.1 Using activejob 5.2.4.4 Installing pg 1.2.3 with native extensions Using webmock 3.2.1 Fetching debug_inspector 0.0.3 Using thor 1.0.1 Using arbre 1.2.1 Using activemodel 5.2.4.4 Using factory_bot 4.8.2 Using grape 1.3.1 Using activerecord 5.2.4.4 Installing websocket-driver 0.7.3 with native extensions Installing debug_inspector 0.0.3 with native extensions Using unicode_utils 1.4.0 Using kaminari-activerecord 1.2.1 Using polyamorous 2.3.2 Using grape-swagger 1.0.0 Using kaminari-grape 1.0.1 Using sort_alphabetical 1.1.0 Using pg_search 2.3.2 Using sixarm_ruby_unaccent 1.2.0 Installing json 2.3.0 with native extensions Using jbuilder 2.10.0 Using countries 3.0.1 Using ransack 2.3.2 Using country_select 4.0.0 Installing ffi 1.12.2 with native extensions Installing nokogiri 1.10.10 with native extensions Fetching binding_of_caller 0.8.0 Installing binding_of_caller 0.8.0 with native extensions Fetching puma 5.0.4 Installing puma 5.0.4 with native extensions Fetching bootsnap 1.4.6 Installing bootsnap 1.4.6 with native extensions Fetching sassc 2.2.1 Using rb-inotify 0.10.1 Using listen 3.1.5 Installing sassc 2.2.1 with native extensions Using loofah 2.7.0 Using xpath 3.2.0 Using rails-dom-testing 2.0.3 Using capybara 3.32.0 Using rails-html-sanitizer 1.3.0 Using actionview 5.2.4.4 Using poltergeist 1.18.1 Using actionpack 5.2.4.4 Using kaminari-actionview 1.2.1 Using actioncable 5.2.4.4 Using kaminari 1.2.1 Using formtastic 3.1.5 Using has_scope 0.7.2 Using railties 5.2.4.4 Using sprockets-rails 3.2.2 Using coffee-rails 4.2.2 Using activestorage 5.2.4.4 Using simple_form 5.0.2 Using actionmailer 5.2.4.4 Using grape-kaminari 0.1.9 Using jquery-rails 4.3.5 Using dotenv-rails 2.2.2 Using factory_bot_rails 4.8.2 Using rspec-rails 4.0.0 Using rails 5.2.4.4 Using webpacker 5.2.1 Using mail_form 1.8.0 Using responders 3.0.0 Using inherited_resources 1.11.0 Using devise 4.7.1 Using sassc-rails 2.1.2 Using bootstrap-sass 3.4.1 Using activeadmin 2.7.0 Using simplecov 0.17.1 Bundle complete! 49 Gemfile dependencies, 162 gems now installed. Use `bundle info [gemname]` to see where a bundled gem is installed. Database 'bathrooms_development' already exists Database 'bathrooms_test' already exists -- enable_extension("plpgsql") -> 0.0435s -- enable_extension("unaccent") -> 0.0162s -- create_table("active_admin_comments", {:id=>:serial, :force=>:cascade}) -> 0.0889s -- create_table("admin_users", {:id=>:serial, :force=>:cascade}) -> 0.0667s -- create_table("restrooms", {:id=>:serial, :force=>:cascade}) -> 0.0728s -- enable_extension("plpgsql") -> 0.0304s -- enable_extension("unaccent") -> 0.0094s -- create_table("active_admin_comments", {:id=>:serial, :force=>:cascade}) -> 0.0728s -- create_table("admin_users", {:id=>:serial, :force=>:cascade}) -> 0.0413s -- create_table("restrooms", {:id=>:serial, :force=>:cascade}) -> 0.0231s root@264d809dcd0b:/refugerestrooms# ```

It mostly reused packages from the volume. It only had to reinstall gems that have native extensions. It works great. 👍

At first glance, the number of jobs for bundler seems high (bundle install --jobs=20). But In practice, this did not seem to run slowly on my computer.

Overall, I can see that this doesn't cause any new situations where Docker can't start. And it properly ensures that docker-compose run will pretty much always work. Also saves on network usage. I think this is worth merging.