search

hypothesis / h

Annotate with anyone, anywhere.
https://hypothes.is/
BSD 2-Clause "Simplified" License
2.94k stars 427 forks source link

Installation setup for "Standard Dev Installation" using Docker Toolbox not possible #5410

Closed Afrowave closed 5 years ago

Afrowave commented 5 years ago

I have a pre-2010 Mac Pro (old CPUs) so I cannot use the Docker on Mac app. I use the Docker Toolbox to spin my containers.

Possibly because I am not using the Docker on the Mac app, I am found that if there is a default PostgreSQL installation on the system that is bound to the localhost, binding the docker container ports to the localhost ports does not work for me on Mac OS X 10.11.6. In the process, one cannot make dev successfully.

Steps to reproduce

I have installed and used the Installing the system dependencies on macOS instructions. Basically here is my containers.

... postgres:9.4-alpine     "docker-entrypoint..."   ...    127.0.0.1:5432->5432/tcp     h_postgres_1
... hypothesis/elasticsearch:latest   "/usr/local/bin/do..."   ...   127.0.0.1:9200->9200/tcp, 9300/tcp   h_elasticsearch_1
...

Expected behaviour

At this point I expect to make dev and go to my localhost:5000 and localhost:9200 and see the installation and an Elasticsearch JSON object .

Actual behaviour

On make dev, I get a postgresql error:

...
File "h/cli/commands/init.py", line 38, in _init_db
    engine.execute('select 1 from alembic_version')
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2054, in execute
    connection = self.contextual_connect(close_with_result=True)
 ...
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2138, in _wrap_pool_connect
    return fn()
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect
    return _ConnectionFairy._checkout(self)
 ...
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect
    connection = pool._invoke_creator(self)
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect
    return dialect.connect(*cargs, **cparams)
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 392, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/Users/jimmygitonga/devzone/my_repos/hypo-h/.tox/py27-dev/lib/python2.7/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) fe_sendauth: no password supplied

ERROR: InvocationError for command '/bin/sh bin/hypothesis --dev init' (exited with code 1)

No password supplied.

But on the postgres logs:

$ docker-compose logs postgres
...
postgres_1       | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1       | creating subdirectories ... ok
postgres_1       | selecting default max_connections ... 100
postgres_1       | selecting default shared_buffers ... 128MB
postgres_1       | selecting dynamic shared memory implementation ... posix
postgres_1       | creating configuration files ... ok
postgres_1       | creating template1 database in /var/lib/postgresql/data/base/1 ... ok
postgres_1       | initializing pg_authid ... ok
postgres_1       | setting password ... ok
postgres_1       | initializing dependencies ... ok
postgres_1       | creating system views ... ok
postgres_1       | loading system objects' descriptions ... ok
postgres_1       | sh: locale: not found
postgres_1       | creating collations ... ok
postgres_1       | No usable system locales were found.
postgres_1       | Use the option "--debug" to see details.
postgres_1       | creating conversions ... ok
postgres_1       | creating dictionaries ... ok
postgres_1       | setting privileges on built-in objects ... ok
postgres_1       | creating information schema ... ok
postgres_1       | loading PL/pgSQL server-side language ... ok
postgres_1       | vacuuming database template1 ... ok
postgres_1       | copying template1 to template0 ... ok
postgres_1       | copying template1 to postgres ... ok
postgres_1       | syncing data to disk ... ok
...

docker-compose exec postgres psql -U postgres
psql (9.4.19)
Type "help" for help.

postgres=# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges
-----------+----------+----------+------------+------------+-----------------------
 h         | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(4 rows)

Everything looks okay.

Additional details

I decided to see which postgres I am looking at. I have an old PostgreSQL 9.5 on my local machine. This was meant to be replaced by the Hypothesis Docker postres container.

The localhost postgres is:

$ psql postgresql://postgres@localhost/postgres
psql (10.5, server 9.5.0)
Type "help" for help.

postgres=# \l
                             List of databases
   Name    |  Owner   | Encoding | Collate | Ctype |   Access privileges
-----------+----------+----------+---------+-------+-----------------------
 postgres  | postgres | UTF8     | C       | C     |
 template0 | postgres | UTF8     | C       | C     | =c/postgres          +
           |          |          |         |       | postgres=CTc/postgres
 template1 | postgres | UTF8     | C       | C     | =c/postgres          +
           |          |          |         |       | postgres=CTc/postgres
(3 rows)

I created the htest database, to make sure I was on the right track. The Docker postgres is:

$ docker-compose exec postgres psql -U postgres
psql (9.4.19)
Type "help" for help.

postgres=# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges
-----------+----------+----------+------------+------------+-----------------------
 h         | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 htest     | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(5 rows)

From the installation part of the docs: "You should also be able to visit your Elasticsearch service by opening http://localhost:9200/ in a browser, and connect to your PostgreSQL by running psql postgresql://postgres@localhost/postgres (if you have psql installed)."

At this point, the ElasticSearch container is not showing up on the localhost though I have nothing else on port 9200:

... hypothesis/elasticsearch:latest   "/usr/local/bin/do..."   ...   127.0.0.1:9200->9200/tcp, 9300/tcp   h_elasticsearch_1

$ curl -XGET 127.0.0.1:9200
curl: (7) Failed to connect to 127.0.0.1 port 9200: Connection refused

When I change the binding of the ports in docker-compose.yml and free the Docker containers from the localhost, I get:

$ docker ps
... postgres:9.4-alpine      "docker-entrypoint..."   ...   0.0.0.0:5432->5432/tcp                                                                       h_postgres_1
... hypothesis/elasticsearch:latest   "/usr/local/bin/do..."    0.0.0.0:9200->9200/tcp, 9300/tcp                                                             h_elasticsearch_1

$ curl -XGET 192.168.99.100:9200
{
  "name" : "FkjsWfv",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "mwDSmdaPRyufw424LIPdRg",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

So the services are running and docker-compose logs rabbit works as expected as well. After looking around, I found localhost is tied up to many things.

Browser/system information

Mac Pro 2,1 Mac OS X 10.11.6 Docker 17.07.0-ce, build 8784753

@robertknight is there anything I am missing here?

robertknight commented 5 years ago

I'm not familiar with Docker Toolbox, but https://stackoverflow.com/questions/35496273/port-mapping-in-docker-on-mac-osx-installed-with-docker-toolbox suggests that mapping of container ports to host ports does not work the same way. From what I understand, Docker Toolbox creates a Linux virtual machine using VirtualBox and then runs the containers inside that. This means that any references to services being on localhost:<port> will instead need to be changed to <IP of Docker VM>:<port> and you'll need to set appropriate environment variables (DATABASE_URL, ELASTICSEARCH_URL, BROKER_URL) to point to these IPs.

An alternative option is to set up a Linux VM and run h inside that, following the setup instructions for a Linux development environment.

If your Postgres instance needs a password, you can configure that in the DATABASE_URL env var. See https://stackoverflow.com/questions/3582552/postgresql-connection-url

Afrowave commented 5 years ago

Thanks @robertknight . Let me have a look at that and I will add something in the docs for users of Docker Toolbox.