jcs / rubywarden

An unofficial, mostly Bitwarden-compatible API server written in Ruby (Sinatra and ActiveRecord)
ISC License
593 stars 49 forks source link

Docker support #7

Closed rtfpessoa closed 5 years ago

rtfpessoa commented 6 years ago

Hi,

Great initiative you have here. I started to look and could not find how to run this with a local installation of bitwarden. Do you have any guide?

Atm I made a Dockerfile to run this and a script to change the docker-compose setup to use that docker. Would you be interested in having an automated script like https://raw.githubusercontent.com/bitwarden/core/master/scripts/bitwarden.sh to install this project? If yes any suggestions?

kendokan commented 6 years ago

Another take at a Dockerfile, if it's of interest.

FROM ruby:alpine

ENV APP_HOME=/opt/bitwarden-ruby RACK_ENV=production PORT=4567
WORKDIR $APP_HOME

# Needed for build but can be discarded.
RUN apk add --no-cache --virtual .build-deps build-base gcc abuild binutils linux-headers git \
 && apk add --no-cache sqlite-dev \
 && git clone https://github.com/jcs/bitwarden-ruby.git ${APP_HOME} \
 && bundle install \
 && apk del .build-deps

ARG UID=1001
ARG GID=1001

RUN addgroup -g ${GID} abc \
 && adduser -D -u ${UID} -G abc abc \
 && chown -R ${UID}:${GID} ${APP_HOME}/db

EXPOSE ${PORT}
VOLUME ["${APP_HOME}/db"]
USER abc

ENTRYPOINT ["bundle","exec"]
CMD rackup -p ${PORT} config.ru
rtfpessoa commented 6 years ago

Ups, closed by mistake. About your Dockerfile, I have a couple of questions/suggestions:

As soon as I can I will try your option locally

kendokan commented 6 years ago

Bundle install requires make, gcc, etc. I remove them all as soon as build install completes, though. By the time the image is built the only additional apk package still installed is sqlite-dev.

Point taken on the downsides of doing a git clone from the master. You could also fork in github and change the repository path in the Dockerfile to your own repo. I do it this way because I generally prefer a single step to build a working image.

I use abc instead of nobody because I want the container writing files as the same UID as the docker user on my host. Makes dealing with permissions easier when I map the volume elsewhere.

jcs commented 6 years ago

I haven't worked with Docker but I'd like to make it easier for others to deploy as long as it's secure. I'd prefer that one user is created to own the base directory and a separate user created that will own db, log, and tmp.

Also, it would be great if you could figure out why bundle install does not work, rather than manually installing gem dependencies that have to be synced if Gemfile ever changes.

Lastly, ALLOW_SIGNUPS should probably not be in the main file since most users won't want new users creating accounts once their own is made. I'm not sure how best to handle toggling that variable through Docker.

kendokan commented 6 years ago

ALLOW_SIGNUPS can be left out of the Dockerfile entirely and set as an environment variable by the user at the time of container creation.

rtfpessoa commented 6 years ago

@jcs ok nice. I will look into the details you suggested and will do a pr with the setup script similar to the official one.

rtfpessoa commented 6 years ago

I think I have most of the modifications in this commit https://github.com/rtfpessoa/bitwarden-ruby/commit/0c3d28194f1fb3250f75324ae24dbf2c704388c0 but for some reason the local installation seems to be connecting to the bitwarden.com vault.

If you want to give it a try you can run:

curl -s -o bitwarden.sh https://raw.githubusercontent.com/rtfpessoa/bitwarden-ruby/master/scripts/bitwarden.sh && chmod +x bitwarden.sh

Do you have any ideas?

EDIT: Made some changes and misc fixes to some vars and replacements

rtfpessoa commented 6 years ago

@jcs I think I misunderstood this project. Before I got the full picture I tough this was only an API replacement, but this should replace all the back-end components (db, icons, identity, API) and be used with the Mobile and Browsers clients. Can you confirm this?

jcs commented 6 years ago

That is correct, it is a complete standalone backend for the mobile apps and browser extensions.

rtfpessoa commented 6 years ago

@jcs https://github.com/rtfpessoa/bitwarden-ruby/commit/328392128af2ef13920f7a2efa18a6e2e1b96aac should have a working docker with a proper setup script and nginx.

Questions:

Notes:

To test this docker you can run:

curl -s -o bitwarden.sh \
    https://raw.githubusercontent.com/rtfpessoa/bitwarden-ruby/master/scripts/bitwarden.sh \
    && chmod u+x bitwarden.sh

Please let me know if you found something wrong and what you think.

Pschittt commented 6 years ago

@rtfpessoa I've tested your setup.rb file, and for me your settings for the web vault are OK. (They are the same that I used for my testing).

But, currently @jcs API are compatible with the web vault. I've made some change to make them compatible. You can check my pull request here : #14

One more thing about your docker. You should put the nginx installation as optional. If a user already have a web server running or a reverse proxy, it doesn't nginx. Is it possible for you to add it as optional ?

Edit : last thing. On your install.sh script, you should add an option to build the docker image from the dockerfile, or to pull it from docker hub. Currently you only do the pull from docker hub.

rtfpessoa commented 6 years ago

I will try to do this soon.

Edit: Thanks for testing đź‘Ť

Pschittt commented 6 years ago

On your setup.rb you are creating a "web" directory which contains a settings.js file. That's the file used for the web vault parameter, where the backend servers url are set.

On the official bitwarden repository, the file is name settings.Production.json

rtfpessoa commented 6 years ago

Let me see if I understood. The file I have settings.js has the correct addresses and it is still using the remote vault. Do I need to change the file to be named settings.Production.json?

Pschittt commented 6 years ago

2 options for that :

It si explained here : https://github.com/bitwarden/web/blob/master/README.md

Regarding your question about nginx, It will be to use the setup.rb for only the web vault and the api backend. I think that « attachment » is not needed too. I didn’t figure what it is use for.

rtfpessoa commented 6 years ago

I think I fixed everything related to the webvault. And I added an option to run without nginx, can you test it?

Also I think building the docker might be a future option since it might take cloning and to make it right we would have to build the webvault too.

Pschittt commented 6 years ago

I’ve checked quickly, some comments :

rtfpessoa commented 6 years ago
  1. I tough about that but to do it I would need to change a lot of code since I directly implemented what bitwarden had. If I have some more time I can look into it but the functionality is there and at least works for the case.

  2. atm the dockers have no ports exposed, nginx is the one redirecting to their internal names without port since they are always using 80. We could customize the ports and expose them but that would need a bit more work.

scantem commented 6 years ago

I've been using the Dockerfile @kendokan posted it and it works well, my personal preference is for a container that doesn't require any changes to the project it's hosting. That container simply fires up a basic container that exposes the application on a port and can store the sqlite database on a volume, I'm not sure why we need much more than that.

Pschittt commented 6 years ago

You’re right, maybe we should keep it simple.

The docker of @rtfpessoa is a script for full installation, including the web vault.

rtfpessoa commented 6 years ago

@scantem not sure what you mean, but my Dockerfile provides the same.

What the rest offers, is a script to install the full bitwarden suite like the official one with full support for SSL, out of the box and effortless.

Pschittt commented 6 years ago

@kendokan Does your Dockerfile work ? I've tested it, and it doesn't seem to work for me.

From what I see, if it runs only "rackup -p ${PORT} config.ru" it doesn't work. I need to run the full command to make it works : "env RACK_ENV=production bundle exec rackup -p ${PORT} config.ru"

Pschittt commented 6 years ago

@kendokan Edit : the dockerfile works. What was not working was the access to my local sqlite database.

I explain myself : Your dockerfile set permission to the db folder on the container (group abc, only known on the container). I've mounted my db folder (from the host) to the db folder on the container. My host db folder ACL doesn't match the one set on the container during the build. So, when the container is running, the API cannot write on the sqlite file (permissions denied).

Do you know how can I do to match the ACL on the newly group on the container with the file on the host ?

I'd like to avoid to set a 777 permission on the db file on the host.

kendokan commented 6 years ago

@Pschittt You can set the UID and GID environment variables to match the uid/gid of the host user you want to own the db folder. The abc user/group will then be created using that uid/gid and permissions will work.

Pschittt commented 6 years ago

@kendokan Is there a simple way to do this ? I’ve read some article, and it seems complicated to do this.

scantem commented 6 years ago

@Pschittt you just need to pass them through as environment variables like so: docker run -d --name bitwarden -e UID=1000 -e GID=100 -v /docker/bitwarden:/opt/bitwarden-ruby/db -p 4567:4567 imagename

This will use the UID 1000 and GID 100 for local filesystem ownership of the db directory. If /docker/bitwarden doesn't exist on your local filesystem it will be created automatically and the correct permissions applied. You can pass through any environment variabled you like using the -e option, for example -e ALLOW_SIGNUPS=1 to allow signups.

kendokan commented 6 years ago

As @scantem said, or if using docker-compose with the Dockerfile, something like:

  bitwarden-ruby:
    build:
      context: ~/dockerfiles/bitwarden-ruby
      args:
       - UID=1000
       - GID=100
    environment:
      - PORT=4567
      - ALLOW_SIGNUPS=1
    volumes:
      - ~/volumes/bitwarden-ruby:/opt/bitwarden-ruby/db
joelmccracken commented 6 years ago

I started working on porting to the cloudron platform here: https://github.com/joelmccracken/bitwarden-ruby/commit/5edc4a6e48924139f4cb9f75c723f2c7680f24fd

(cloudron apps all run on docker)

Pschittt commented 6 years ago

I’m sorry, but what the interest to get Bitwarden-ruby running on cloudron ? It already runs on docker.

kendokan commented 6 years ago

It's for people that want to run self-hosted software but at the same time don't want to learn how to use and manage something like docker. I'd be reluctant to choose self-hosting for a critical security service that was still being managed by a third party. You get none of the advantages of self-hosting, but also all of the disadvantages of self-hosting (ultimately I chose not to self-host Bitwarden at all).

Pschittt commented 6 years ago

Well, I don’t find using docker complicated if the projet is available on the docker hub.

joelmccracken commented 6 years ago

@kendokan I don't necessarily disagree with your assessment, but cloudron supports self-hosting. I think for those who are worried about it, this is the route they would take.

Pschittt commented 6 years ago

@scantem I've tried to run your command : docker run -d --name bitwarden -e UID=1000 -e GID=100 -v /docker/bitwarden:/opt/bitwarden-ruby/db -p 4567:4567 imagename

And nothing changed. The db folder (inside the docker) still has the host permission UID and GID. And the user ABC (created when the image has been created), still has the permission 1001 and 1001, set when the image has been created.

scantem commented 6 years ago

@Pschittt My apologies, the UID & GID are ARGS, not ENV variables. Probably the easiest way is to use the docker-compose file kendokan supplied so that the ARGS are supplied when the image is built (or you can supply them if you build the image manually).

docker-compose is probably the way to go, it's good to have your original config settings if you ever need to create the container again.

mbirth commented 6 years ago

I just installed rtfpessoa/bitwarden-ruby on my Synology DS415+'s internal Docker host. No problems at all. Thanks a lot!

TMHBOFH commented 6 years ago

@mbirth did you test the web vault too? On my test installation the web vault ist not work.

Thanks for the great project

rtfpessoa commented 6 years ago

@tmhbofh latest version should have web vault working. Previous versions didn't. Otherwise I can take a look and eventually pr the whole thing. On Fri, 5 Jan 2018 at 19:40, TMHBOFH notifications@github.com wrote:

@mbirth https://github.com/mbirth did you test the web vault too? On my test installation the web vault ist not work.

Thanks for the great project

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jcs/bitwarden-ruby/issues/7#issuecomment-355646455, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3E8EhH1UcqOIGV_t5W3tOGsNqX2D49ks5tHnqtgaJpZM4Qk5Ma .

-- Best, Rodrigo

mbirth commented 6 years ago

I only tested it with the Chrome extension and the Android app.

TMHBOFH commented 6 years ago

@rtfpessoa

tanks for the fast feedback?

I used this script for installation https://raw.githubusercontent.com/rtfpessoa/bitwarden-ruby/master/scripts/bitwarden.sh

Thanks for your help

rtfpessoa commented 6 years ago

I am assuming you executed it with the command in here ( https://github.com/jcs/bitwarden-ruby/issues/7#issuecomment-348823514) right?

That should work.

Can you report what were the options you selected/inserted?

On Fri, Jan 5, 2018 at 8:40 PM TMHBOFH notifications@github.com wrote:

@rtfpessoa https://github.com/rtfpessoa

tanks for the fast replease?

I used this script for installation https://raw.githubusercontent.com/rtfpessoa/bitwarden-ruby/master/scripts/bitwarden.sh

Thanks for your help

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jcs/bitwarden-ruby/issues/7#issuecomment-355659835, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3E8GhvL-UKvPkEsBYyKMQWjgAownx9ks5tHoi7gaJpZM4Qk5Ma .

-- Best, Rodrigo

TMHBOFH commented 6 years ago

yes you are right

a short install descript

./bitwarden.sh install

(!) Enter the domain name for your bitwarden instance (ex. bitwarden.company.com): pw.test.***.de (!) Do you want to use Let's Encrypt to generate a free SSL certificate? (y/n): n dev: Pulling from rtfpessoa/bitwarden-ruby Digest: sha256:dcc4fbef73acde046982d49b36b50f3681b4e9f4c7052b4206201685f0d9905b Status: Image is up to date for rtfpessoa/bitwarden-ruby:dev Bitwarden setup... (!) Enter your installation id (get it at https://bitwarden.com/host): (!) Enter your installation key: (!) Do you need nginx for SSL configuration? [Y/n] y (!) Do you have a SSL certificate to use? [Y/n] y Make sure 'certificate.crt' and 'private.key' are provided in the appropriate directory (see setup instructions). (!) Do you want to use the default ports for HTTP (80) and HTTPS (443)? [Y/n] y (!) Use Diffie Hellman ephemeral parameters for SSL (requires dhparam.pem)? [Y/n] y (!) Is this a trusted SSL certificate (requires ca.crt)? [Y/n] y Building nginx config. (!) Notifications email:

(!) Allow user registration? [Y/n] y Building docker environment files. Building docker environment override files. Building app settings. Building FIDO U2F app id. Building docker-compose.yml. Bitwarden setup complete!

after that i copy the certificate files to folde ./bwdata/ssl/pw.test.***.de/*cert,key,dh,ca

and i start with ./bitwarden.sh start

Open source password management solutions Copyright 2015-2018, 8bit Solutions LLC https://bitwarden.com, https://github.com/bitwarden Ruby: https://github.com/rtfpessoa/bitwarden-ruby

===================================================

Docker version 17.12.0-ce, build c97c6d6 docker-compose version 1.18.0, build 8dd22a9

Removing network docker_default WARNING: Network docker_default not found. Pulling web (bitwarden/web:1.20.1)... 1.20.1: Pulling from bitwarden/web Digest: sha256:06ca14ce33da430d27769d561d2608bd56f1fd8034ff52664357b1a2ef7d3e97 Status: Image is up to date for bitwarden/web:1.20.1 Pulling api (rtfpessoa/bitwarden-ruby:dev)... dev: Pulling from rtfpessoa/bitwarden-ruby Digest: sha256:dcc4fbef73acde046982d49b36b50f3681b4e9f4c7052b4206201685f0d9905b Status: Image is up to date for rtfpessoa/bitwarden-ruby:dev Pulling attachments (bitwarden/attachments:1.14.1)... 1.14.1: Pulling from bitwarden/attachments Digest: sha256:9d571382660a7861c241b1fa2ea7e62e10c160521a55babf36b92d11c9135055 Status: Image is up to date for bitwarden/attachments:1.14.1 Pulling nginx (bitwarden/nginx:1.14.1)... 1.14.1: Pulling from bitwarden/nginx Digest: sha256:7fa99f8764efe1bed8bd496cab3046fb87a17b7524eb2e4cc7ca220bfc8ce32b Status: Image is up to date for bitwarden/nginx:1.14.1 Creating network "docker_default" with the default driver Creating nginx ... done Creating api ... Creating nginx ... Creating attachments ... Total reclaimed space: 0B dev: Pulling from rtfpessoa/bitwarden-ruby Digest: sha256:dcc4fbef73acde046982d49b36b50f3681b4e9f4c7052b4206201685f0d9905b Status: Image is up to date for rtfpessoa/bitwarden-ruby:dev

If I wants to create a account over web i get the error "master password hash" (i think that's a known problem?)

With Android-APP i can create a account and add conntent.

Now a get the error "deviceidentifier cannot be blank" if i will login via web.

rtfpessoa commented 6 years ago

Ahhh, ok. That is normal, we need https://github.com/jcs/bitwarden-ruby/pull/14 and also something from bitwarden. But should not take long to be fixed.

On Fri, Jan 5, 2018 at 9:07 PM TMHBOFH notifications@github.com wrote:

yes you are right

a short install descript

./bitwarden.sh install

(!) Enter the domain name for your bitwarden instance (ex. bitwarden.company.com): pw.test.***.de (!) Do you want to use Let's Encrypt to generate a free SSL certificate? (y/n): n dev: Pulling from rtfpessoa/bitwarden-ruby Digest: sha256:dcc4fbef73acde046982d49b36b50f3681b4e9f4c7052b4206201685f0d9905b Status: Image is up to date for rtfpessoa/bitwarden-ruby:dev Bitwarden setup... (!) Enter your installation id (get it at https://bitwarden.com/host): (!) Enter your installation key: (!) Do you need nginx for SSL configuration? [Y/n] y (!) Do you have a SSL certificate to use? [Y/n] y Make sure 'certificate.crt' and 'private.key' are provided in the appropriate directory (see setup instructions). (!) Do you want to use the default ports for HTTP (80) and HTTPS (443)? [Y/n] y (!) Use Diffie Hellman ephemeral parameters for SSL (requires dhparam.pem)? [Y/n] y (!) Is this a trusted SSL certificate (requires ca.crt)? [Y/n] y Building nginx config. (!) Notifications email:

(!) Allow user registration? [Y/n] y Building docker environment files. Building docker environment override files. Building app settings. Building FIDO U2F app id. Building docker-compose.yml. Bitwarden setup complete!

after that i copy the certificate files to folde ./bwdata/ssl/pw.test.***.de/*cert,key,dh,ca

and i start with ./bitwarden.sh start

Open source password management solutions Copyright 2015-2018, 8bit Solutions LLC https://bitwarden.com, https://github.com/bitwarden Ruby: https://github.com/rtfpessoa/bitwarden-ruby

===================================================

Docker version 17.12.0-ce, build c97c6d6 docker-compose version 1.18.0, build 8dd22a9

Removing network docker_default WARNING: Network docker_default not found. Pulling web (bitwarden/web:1.20.1)... 1.20.1: Pulling from bitwarden/web Digest: sha256:06ca14ce33da430d27769d561d2608bd56f1fd8034ff52664357b1a2ef7d3e97 Status: Image is up to date for bitwarden/web:1.20.1 Pulling api (rtfpessoa/bitwarden-ruby:dev)... dev: Pulling from rtfpessoa/bitwarden-ruby Digest: sha256:dcc4fbef73acde046982d49b36b50f3681b4e9f4c7052b4206201685f0d9905b Status: Image is up to date for rtfpessoa/bitwarden-ruby:dev Pulling attachments (bitwarden/attachments:1.14.1)... 1.14.1: Pulling from bitwarden/attachments Digest: sha256:9d571382660a7861c241b1fa2ea7e62e10c160521a55babf36b92d11c9135055 Status: Image is up to date for bitwarden/attachments:1.14.1 Pulling nginx (bitwarden/nginx:1.14.1)... 1.14.1: Pulling from bitwarden/nginx Digest: sha256:7fa99f8764efe1bed8bd496cab3046fb87a17b7524eb2e4cc7ca220bfc8ce32b Status: Image is up to date for bitwarden/nginx:1.14.1 Creating network "docker_default" with the default driver Creating nginx ... done Creating api ... Creating nginx ... Creating attachments ... Total reclaimed space: 0B dev: Pulling from rtfpessoa/bitwarden-ruby Digest: sha256:dcc4fbef73acde046982d49b36b50f3681b4e9f4c7052b4206201685f0d9905b Status: Image is up to date for rtfpessoa/bitwarden-ruby:dev

If I wants to create a account over web i get the error "master password hash" (i think that's a known problem?)

With Android-APP i can create a account and add conntent.

Now a get the error "deviceidentifier cannot be blank" if i will login via web.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jcs/bitwarden-ruby/issues/7#issuecomment-355665605, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3E8Kk1RLlA8U9RQCUinpl4KyoqwV1gks5tHo8BgaJpZM4Qk5Ma .

-- Best, Rodrigo

TMHBOFH commented 6 years ago

Ahh ok i understand

Thank you very much for the fast answer and all other.

jh74 commented 6 years ago

Following this project since December and looking forward to when the login via web vault will function properly :-) In the meantime I wanted to try out the bitwarden Conversion tool but have no clue how to use this from within the docker container. How can I use the command env RACK_ENV=production bundle exec ruby tools/bitwarden_import.rb -f /path/to/data.csv -u you@example.com in the docker container? Maybe it is not possible at the moment and we have to wait for the web vault? Thanks a lot in advance!

rtfpessoa commented 6 years ago

@jh74 you can try docker run -v /path/outside/data.csv:/path/inside/data.csv -e RACK_ENV=production --entrypoint bundle <docker-image-name> exec ruby tools/bitwarden_import.rb -f /path/inside/data.csv -u you@example.com

Edit: fixed spaces and better file paths

jh74 commented 6 years ago

@rtfpessoa Thank you that seems to be the correct way but: The error ruby: No such file or directory -- tools/bitwarden_import.rb (LoadError) is because you dont have the bitwarden_import.rb in your folder tools of the api docker I switched to lastpass_import.rb to see if it works but then i get a tools/lastpass_import.rb:66:in <main>: can't find existing User record for "here.my@mailadress.com" (RuntimeError) Seems it doesnt work with the sqlite db. I will wait for the vault access i think. But thanks a lot for this explanation of yours.

mbirth commented 6 years ago

@rtfpessoa Do you plan on updating your Docker listing when this Repo is updated or is there some updater in your image?

sebastian-ruiz commented 6 years ago

@kendokan I use your Dockerfile from https://github.com/jcs/bitwarden-ruby/issues/7#issuecomment-345851417 and I use your docker-compose.yml from https://github.com/jcs/bitwarden-ruby/issues/7#issuecomment-350887855 like this:

version: "3"

services:  
  bitwarden-ruby:
    build:
      context: .
      dockerfile: Dockerfile
      args:
       - UID=1000
       - GID=100
    environment:
      - PORT=4567
      - ALLOW_SIGNUPS=1
    volumes:
      - ./data/db:/opt/bitwarden-ruby/db
    ports:
      - "4567:4567"

however when I run docker-compose up -d I get this error:

...
Cloning into '/opt/bitwarden-ruby'...
Your Ruby version is 2.5.1, but your Gemfile specified >= 2.2.8, < 2.5.0
ERROR: Service 'bitwarden-ruby' failed to build: The command '/bin/sh -c apk add --no-cache --virtual .build-deps build-base gcc abuild binutils linux-headers git  && apk add --no-cache sqlite-dev  && git clone https://github.com/jcs/bitwarden-ruby.git ${APP_HOME}  && bundle install  && apk del .build-deps' returned a non-zero code: 18

What am I doing wrong? Thanks!

kendokan commented 6 years ago

My Dockerfile uses base image ruby:2.4.2-alpine3.6, but it looks like you're using a newer version or Ruby than I am. Based on the error, bitwarden-ruby will work up to 2.5.0.

mannp commented 6 years ago

@rtfpessoa Hi, I am giving your docker a try and can't seem to disable account creation.

I've added the docker variable ALLOW_SIGNUPS=false but signups are still allowed?

Edit: So I need to edit the .env file in the docker image to disable logins it seems...

Also, does your docker use the official bitwarden web vault, potentially exposing the microsoft and .net issues I am concerned about? (no specific evidence here, but still concerned)

@kendokan I use an nginx proxy with LE and wondered if your docker was the preferred one for this scenario, without web access?

@jcs is only one port ever exposed with your backend, so allowing access through a proxy to either docker will give access to the api, web etc or should I limited access to just the api?

And is there a particular docker you prefer from a security perspective or either would be fine?

Thanks in advance & apologies for the noob questions, but having choices of docker is thought provoking :)