"You must use Bundler 2"

Hi!

I'm trying to install the sync server following the steps in Self Hosting with Docker.

Starting from a pristine Ubuntu Server 18.04, fully updated and with Docker installed via apt install docker...

sudo apt-get install -y git
git clone https://github.com/standardnotes/syncing-server.git
cd syncing-server
cp env.sample .env.production
bundle exec rake secret

Here I get an error:

Command 'bundle' not found, but can be installed with:

sudo snap install ruby          # version 2.7.1, or
sudo apt  install ruby-bundler

See 'snap info ruby' for additional versions.

I'm guessing at this point that the instructions left out a small detail such as "install ruby." So I tried both suggestions, but after doing so and again trying to bundle exec rake secret again I get the error:

You must use Bundler 2 or greater with this lockfile.

I'm sure if I were a ruby dev then the error messages might make some sense, but since apt installed ruby 2.7.1 but Bundler is apparently still not "2 or greater" I am assuming that there's some secret ruby trick to getting the appropriate version installed.

Any suggestions?

You just need to make sure Bundler >= 2 is installed. You can check your current version with bundle -v.

Hi, Mo!

So I should amend this issue to be that the instructions are wrong, in that they state that all one needs is a standard install of Ubuntu plus Docker. Anyone who relies on that information will get stuck as I did, and will further find multiple roadblocks facing them when they try to resolve the issue by "mak[ing] sure Bundler >= 2 is installed."

I realize that you likely live and breathe the Ruby universe and that it's hard to understand why "mak[ing] sure Bundler >= 2 is installed" isn't a complete and useful solution to this problem, but judging from the many responses I see when I search for "install bundle 2" this is completely opaque to many folks outside the Ruby ecosystem. I do know of at least two other experienced users who have given up installing the SN sync server (and so given up on SN altogether, as their use case required local storage) because "following the instructions didn't work" and they didn't feel that it was worth their time to troubleshoot the issue.

I should point out that I have almost half a century of experience in software development and systems administration. I won't bore you (further) with the reasons why following your instructions created a persistent problem that wasn't solved by just "mak[ing] sure Bundler >= 2 is installed." Suffice to say that had the instructions simply been amended to say "...just finished setting up a Linux server (say, Ubuntu 16.04 64-bit) and have installed Docker and Bundler > 2.0 on it" it would have saved me a few hours of grief.

Put another way, when you're asking someone to buy into your solution (and as you may know, I'm a big fan of Standard Notes) one question they might well have is "how well will it be supported?" When you send the message that you obviously have never even tried the instructions that you post, it diminishes people's confidence that you'll be able to support them in the future.

At any rate, having dutifully "ma[de] sure Bundler >= 2 is installed," I now get this:

[emily:1742][~/syncing-server]$ /usr/local/bin/bundle --version
Bundler version 2.1.4
[emily:1742][~/syncing-server]$ /usr/local/bin/bundle exec rake secret
Could not find rake-13.0.1 in any of the sources
Run `bundle install` to install missing gems.
[emily:1801][~/syncing-server]$ /usr/local/bin/bundle install
Following files may not be writable, so sudo is needed:
  /usr/local/bin
  /var/lib/gems/2.5.0
  /var/lib/gems/2.5.0/build_info
  /var/lib/gems/2.5.0/cache
  /var/lib/gems/2.5.0/doc
  /var/lib/gems/2.5.0/extensions
  /var/lib/gems/2.5.0/gems
  /var/lib/gems/2.5.0/specifications
Fetching gem metadata from https://rubygems.org/.........
Fetching https://github.com/mobitar/capistrano-shoryuken.git
Fetching rake 13.0.1

Your user account isn't allowed to install to the system RubyGems.
  You can cancel this installation and run:

      bundle install --path vendor/bundle

  to install the gems into ./vendor/bundle/, or you can enter your password
  and install the bundled gems to RubyGems using sudo.

  Password: 

Your user account isn't allowed to install to the system RubyGems.
  You can cancel this installation and run:

      bundle install --path vendor/bundle

  to install the gems into ./vendor/bundle/, or you can enter your password
  and install the bundled gems to RubyGems using sudo.

  Password: 
Installing rake 13.0.1
Fetching concurrent-ruby 1.1.6
Installing concurrent-ruby 1.1.6
Fetching i18n 1.8.2
Installing i18n 1.8.2
Fetching minitest 5.14.0
Installing minitest 5.14.0
Fetching thread_safe 0.3.6
Installing thread_safe 0.3.6
Fetching tzinfo 1.2.6
Installing tzinfo 1.2.6
Fetching activesupport 5.1.7
Installing activesupport 5.1.7
Fetching builder 3.2.4
Installing builder 3.2.4
Fetching erubi 1.9.0
Installing erubi 1.9.0
Fetching mini_portile2 2.4.0
Installing mini_portile2 2.4.0
Fetching nokogiri 1.10.9
Installing nokogiri 1.10.9 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /tmp/bundler20200420-25177-rmzw9qnokogiri-1.10.9/gems/nokogiri-1.10.9/ext/nokogiri
/usr/bin/ruby2.5 -r ./siteconf20200420-25177-1tftxvs.rb extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.4.0
checking for gzdopen() in -lz... no
zlib is missing; necessary for building libxml2
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/usr/bin/$(RUBY_BASE_NAME)2.5
    --help
    --clean
    --use-system-libraries
    --enable-static
    --disable-static
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --enable-cross-build
    --disable-cross-build

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /tmp/bundler20200420-25177-rmzw9qnokogiri-1.10.9/extensions/x86_64-linux/2.5.0/nokogiri-1.10.9/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /tmp/bundler20200420-25177-rmzw9qnokogiri-1.10.9/gems/nokogiri-1.10.9 for inspection.
Results logged to /tmp/bundler20200420-25177-rmzw9qnokogiri-1.10.9/extensions/x86_64-linux/2.5.0/nokogiri-1.10.9/gem_make.out

An error occurred while installing nokogiri (1.10.9), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.10.9' --source 'https://rubygems.org/'` succeeds before bundling.

In Gemfile:
  rails was resolved to 5.1.7, which depends on
    actioncable was resolved to 5.1.7, which depends on
      actionpack was resolved to 5.1.7, which depends on
        actionview was resolved to 5.1.7, which depends on
          rails-dom-testing was resolved to 2.0.3, which depends on
            nokogiri
[emily:1802][~/syncing-server]$

So I suspect this is another case where the bugs in the installation instructions were excused with "BUT IT WORKS ON MY MACHINE!!!" I already own a couple of long-term Standard Notes licenses and would like to make SN the, uh, standard at several other sites. It can't happen, though, without local storage and that can't happen as long as the installation is buggy. Obviously, nobody actually tested these instructions on a freshly-configured machine or it would have been apparent that they wouldn't work.

I realize that letting folks install their own servers isn't your highest development priority, but I think you'd have happier customers if you just said "sorry, if you want to do that then you're on your own" than you have by publishing buggy instructions. If I weren't dealing with a pandemic right now (I'm a physician), then I'd probably just take the day or two I would need to debug your instructions and write my own, but I'm afraid there's no time for that right now.

Thanks yet again for a great product. Let me know if/when you have some working instructions for installing the server and I'll see if it's not too late to get the other sites to at least try SN.

Peace...

--Ron

Hi @twoprops!

Thank you for creating this issue!

...
[emily:1801][~/syncing-server]$ /usr/local/bin/bundle install
_Following files may not be writable, so sudo is needed:_
  /usr/local/bin
  /var/lib/gems/2.5.0
  /var/lib/gems/2.5.0/build_info
  /var/lib/gems/2.5.0/cache
  /var/lib/gems/2.5.0/doc
  /var/lib/gems/2.5.0/extensions
  /var/lib/gems/2.5.0/gems
  /var/lib/gems/2.5.0/specifications
Fetching gem metadata from https://rubygems.org/.........
Fetching https://github.com/mobitar/capistrano-shoryuken.git
Fetching rake 13.0.1

Your user account isn't allowed to install to the system RubyGems.
  You can cancel this installation and run:

      bundle install --path vendor/bundle

  to install the gems into ./vendor/bundle/, or you can enter your password
  and install the bundled gems to RubyGems using sudo.
...

Looks like you are using the ruby installed by Ubuntu. To use bundle exec rake secret, it is assumed that a working ruby installation is present and bundler install has installed all gems. We recommend using RVM for a smoother experience.

In short, I would suggest you to uninstall your current version of ruby and use RVM instead.

Having said that, I have updated the docs with some missing steps. You can find the updated version in this commit.

The steps have been reviewed and tested to work on Ubuntu +16.04 (fresh install).

Thanks again for bringing this issue to our attention! We’re awaiting your response. :smile:

Awesome! I'll be able to check it out on Wednesday. Thanks!

So...

I decided to slavishly follow your directions. They call for Ubuntu 16.04 which is now in its last year of supported life, but I didn't want to introduce variables (though today's shiny new Ubuntu 20.04 release was calling my name). I downloaded a fresh .iso of Ubuntu Server 16.04 and installed it on a standalone server. I made no modifications, but proceeded straight to the install directions.

I followed the Docker link on your instructions, which took me to the Docker for Linux page, which linked to the Install Docker Engine on Ubuntu page. I followed directions to the letter. Everything seemed to be fine.
I followed the RVM link on your instructions, which directed me to the Ubuntu RVM GitHub page, from where I could open the README.md file and then follow the install instructions to the letter.
I followed the rest of your instructions to the letter. Things seemed to go as expected, even through the bundle exec rake secret command.
When I tried to "simply" docker-compose up -d, I got:

The program 'docker-compose' is currently not installed. You can install it by typing:
sudo apt install docker-compose

What?!? So a Docker install following the instructions on the Docker website doesn't include docker-compose? Okay, weird, but if that's the case, your instructions really ought to call that out. What to do now? Actually follow the instructions in the error message? Alas, trying

sudo apt install docker-compose

succeeded, but then trying docker-compose up -d I got:

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

So I'm guessing that when you say "You've just finished setting up a Linux server (say, Ubuntu 16.04 64-bit) and have installed Docker on it" you mean something somewhat more involved than following your link to the Docker web site, following the subsequent links, and following the Docker install instructions there. Perhaps you might elaborate a bit more?

So what did you have to do on your scratch Ubuntu 16.04 machine to make these instructions work? They certainly fail as presented.

BTW, I did try installing Ubuntu 20.04 which gives the option to include a Docker install at OS install time. Again, when I tried to "simply" docker-compose up -d I got:

>> ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.

docker-machine start default

>> Docker machine "default" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.
docker-machine create default

If this is too much of a distraction from the main work you're doing, I can dig out the answers and post back here when I have the time. I just didn't anticipate that this would involve that much more than just following the instructions and configuring the server.

At any rate, thanks for your help. The revised instructions certainly seem to fill in a lot of previously missing pieces, so I'm confident that I can figure out the rest.

--2p

Sorry for the late response!

sudo apt install docker-compose

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1. For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

You will run into issues if you use the package from Ubuntu. The installed package by Ubuntu can handle docker-compose files up to version 2. Our docker-compose.yml file is version 3.

Use the guide from Docker at https://docs.docker.com/compose/install/ to install docker-compose.

I guess we should also make it clear that both the docker engine and docker compose are needed.

BTW, I did try installing Ubuntu 20.04 which gives the option to include a Docker install at OS install time.

Haven't tested it up to Ubuntu 20.04, but it should work fine. Of course, it is not guaranteed to work with the exact instructions, but it should be straightforward to figure it out.

ERROR: Couldn't connect to Docker daemon - you might need to run docker-machine start default.

Looks like the daemon isn't running. Please check the status with:

sudo systemctl status docker

If not running, start it with:

sudo systemctl start docker

If it's already running, then you should probably use sudo docker-compose up -d

This happens because you need root-user privileges to manage docker.

There's a post-install guide (optional) that address this matter: https://docs.docker.com/engine/install/linux-postinstall/

But in general, you should be fine by adding sudo in front of docker-compose up -d

So we should also make this clear :+1:

Thanks for letting us know your experience with our guide! It helps us to better improve it :smile:

standardnotes / forum

"You must use Bundler 2" #945