Open jbothma opened 5 years ago
I think what's happening is the setup instructions don't wait for solr and postgres to be ready (which the normal running command does), but try to connect to them.
The first command in setup instructions after cloning is setting up the DB. This is what it looks like for me after cleaning up my setup (incl deleting volumes)
docker-compose run --rm ckan paster --plugin=ckan db init -c /ckan.ini
Creating volume "ckan-docker_solr-data" with default driver
Creating volume "ckan-docker_ckan-filestore" with default driver
Creating volume "ckan-docker_db-data" with default driver
Creating ckan-docker_solr_1 ... done
Creating ckan-docker_db_1 ... done
Creating ckan-docker_ckan-datapusher_1 ... done
Creating ckan-docker_redis_1 ... done
2019-05-23 07:34:57,980 ERROR [pysolr] Failed to connect to server at 'http://solr:8983/solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json', are you sure that URL is correct? Checking it in a browser might help: HTTPConnectionPool(host='solr', port=8983): Max retries exceeded with url: /solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc174c9ebd0>: Failed to establish a new connection: [Errno 111] Connection refused',))
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 366, in _send_request
timeout=self.timeout)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 537, in get
return self.request('GET', url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 524, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 637, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='solr', port=8983): Max retries exceeded with url: /solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc174c9ebd0>: Failed to establish a new connection: [Errno 111] Connection refused',))
2019-05-23 07:34:57,983 ERROR [ckan.lib.search.common] Failed to connect to server at 'http://solr:8983/solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json', are you sure that URL is correct? Checking it in a browser might help: HTTPConnectionPool(host='solr', port=8983): Max retries exceeded with url: /solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc174c9ebd0>: Failed to establish a new connection: [Errno 111] Connection refused',))
Traceback (most recent call last):
File "/src/ckan/ckan/lib/search/common.py", line 57, in is_available
conn.search(q="*:*", rows=1)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 720, in search
response = self._select(params, handler=search_handler)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 418, in _select
return self._send_request('get', path)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 375, in _send_request
raise SolrError(error_message % params)
SolrError: Failed to connect to server at 'http://solr:8983/solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json', are you sure that URL is correct? Checking it in a browser might help: HTTPConnectionPool(host='solr', port=8983): Max retries exceeded with url: /solr/ckan/select/?q=%2A%3A%2A&rows=1&wt=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc174c9ebd0>: Failed to establish a new connection: [Errno 111] Connection refused',))
2019-05-23 07:34:57,984 WARNI [ckan.lib.search] Problems were found while connecting to the SOLR server
Traceback (most recent call last):
File "/usr/local/bin/paster", line 11, in <module>
sys.exit(run())
File "/usr/local/lib/python2.7/dist-packages/paste/script/command.py", line 102, in run
invoke(command, command_name, options, args[1:])
File "/usr/local/lib/python2.7/dist-packages/paste/script/command.py", line 141, in invoke
exit_code = runner.run(args)
File "/usr/local/lib/python2.7/dist-packages/paste/script/command.py", line 236, in run
result = self.command()
File "/src/ckan/ckan/lib/cli.py", line 333, in command
self._load_config(cmd!='upgrade')
File "/src/ckan/ckan/lib/cli.py", line 306, in _load_config
self.site_user = load_config(self.options.config, load_site_user)
File "/src/ckan/ckan/lib/cli.py", line 221, in load_config
load_environment(conf.global_conf, conf.local_conf)
File "/src/ckan/ckan/config/environment.py", line 99, in load_environment
p.load_all()
File "/src/ckan/ckan/plugins/core.py", line 129, in load_all
unload_all()
File "/src/ckan/ckan/plugins/core.py", line 182, in unload_all
unload(*reversed(_PLUGINS))
File "/src/ckan/ckan/plugins/core.py", line 210, in unload
plugins_update()
File "/src/ckan/ckan/plugins/core.py", line 121, in plugins_update
environment.update_config()
File "/src/ckan/ckan/config/environment.py", line 268, in update_config
model.init_model(engine)
File "/src/ckan/ckan/model/__init__.py", line 156, in init_model
version_table = Table('migrate_version', meta.metadata, autoload=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/schema.py", line 352, in __new__
table._init(name, metadata, *args, **kw)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/schema.py", line 425, in _init
self._autoload(metadata, autoload_with, include_columns)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/schema.py", line 448, in _autoload
self, include_columns, exclude_columns
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1670, in run_callable
with self.contextual_connect() as conn:
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1737, in contextual_connect
self.pool.connect(),
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 332, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 630, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 433, in checkout
rec = pool._do_get()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 949, in _do_get
return self._create_connection()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 278, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 404, in __init__
self.connection = self.__connect()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 530, in __connect
connection = self.__pool._creator()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 95, in connect
connection_invalidated=invalidated
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 89, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 376, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py", line 179, in connect
connection_factory=connection_factory, async=async)
sqlalchemy.exc.OperationalError: (OperationalError) could not connect to server: Connection refused
Is the server running on host "db" (172.21.0.2) and accepting
TCP/IP connections on port 5432?
None None
After running that, I checked what actually got up and running:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fc22087de6ed ckan-docker_solr "docker-entrypoint.s…" 47 seconds ago Up 44 seconds 0.0.0.0:8983->8983/tcp ckan-docker_solr_1
dd6cca1940c0 redis:latest "docker-entrypoint.s…" 47 seconds ago Up 44 seconds 6379/tcp ckan-docker_redis_1
513e1a2ceb77 ckan-docker_ckan-datapusher "python /src/datapus…" 47 seconds ago Up 44 seconds 0.0.0.0:8800->8800/tcp ckan-docker_ckan-datapusher_1
e00a72353999 postgres:9.4 "docker-entrypoint.s…" 47 seconds ago Up 44 seconds 0.0.0.0:5433->5432/tcp ckan-docker_db_1
Since solr and postgres as running, I tried the DB setup command again:
docker-compose run --rm ckan paster --plugin=ckan db init -c /ckan.ini
Starting ckan-docker_solr_1 ...
Starting ckan-docker_ckan-datapusher_1 ... done
Starting ckan-docker_solr_1 ... done
2019-05-23 07:35:55,456 DEBUG [ckanext.extractor.model] Defining resource metadata table
2019-05-23 07:35:55,460 DEBUG [ckanext.extractor.model] Defining resource metadatum table
Initialising DB: SUCCESS
What I get on a new docker-compose up
$ docker-compose up [9:54:42]
Creating network "treasury-ckan_default" with the default driver
Creating treasury-ckan_ckan-datapusher_1 ... done
Creating treasury-ckan_db_1 ... done
Creating treasury-ckan_redis_1 ... done
Creating treasury-ckan_solr_1 ... done
Creating treasury-ckan_ckan-worker_1 ...
Creating treasury-ckan_ckan_1 ... error
ERROR: for treasury-ckan_ckan_1 Cannot start service ckan: OCI runtime create failed:
container_linux.go:345: starting container process caused "process_linux.go:424: container init caused
\"rootfs_linux.go:58: mounting \\\"/home/foiuglkj/Workspace/openup/treasury-ckan/ckan.ini\\\" to rootfs \\\"/mnt
/sda1/var/lib/docker/overlay2/9fd51096f4b9d0bb26134f838e9e24527b3a73c49525a6dd16e10a932c71d6a4
/merged\\\" at \\\"/mnt/sda1/var/lib/docker/overlay2
/9fd51096f4b9d0bb26134f838e9e24527b3a73c49525a6dd16e10a932c71d6a4/merged/ckan.ini\\\"
causedCreating treasury-ckan_ckan-worker_1 ... error
exists and is the expected type
ERROR: for treasury-ckan_ckan-worker_1 Cannot start service ckan-worker: OCI runtime create failed:
container_linux.go:345: starting container process caused "process_linux.go:424: container init caused
\"rootfs_linux.go:58: mounting \\\"/home/foiuglkj/Workspace/openup/treasury-ckan/ckan.ini\\\" to rootfs \\\"/mnt
/sda1/var/lib/docker/overlay2/8f79ec531babbd490a82504d3346b41643207e00a6d95af9c4aa2e13fc2afaec
/merged\\\" at \\\"/mnt/sda1/var/lib/docker/overlay2
/8f79ec531babbd490a82504d3346b41643207e00a6d95af9c4aa2e13fc2afaec/merged/ckan.ini\\\" caused
\\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the
specified host path exists and is the expected type
ERROR: for ckan Cannot start service ckan: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/home/foiuglkj/Workspace/openup/treasury-ckan/ckan.ini\\\" to rootfs \\\"/mnt/sda1/var/lib/docker/overlay2/9fd51096f4b9d0bb26134f838e9e24527b3a73c49525a6dd16e10a932c71d6a4/merged\\\" at \\\"/mnt/sda1/var/lib/docker/overlay2/9fd51096f4b9d0bb26134f838e9e24527b3a73c49525a6dd16e10a932c71d6a4/merged/ckan.ini\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: for ckan-worker Cannot start service ckan-worker: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/home/foiuglkj/Workspace/openup/treasury-ckan/ckan.ini\\\" to rootfs \\\"/mnt/sda1/var/lib/docker/overlay2/8f79ec531babbd490a82504d3346b41643207e00a6d95af9c4aa2e13fc2afaec/merged\\\" at \\\"/mnt/sda1/var/lib/docker/overlay2/8f79ec531babbd490a82504d3346b41643207e00a6d95af9c4aa2e13fc2afaec/merged/ckan.ini\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
I can confirm all oft those files exist on my host machine.
docker version
Client:
Version: 18.09.6-ce
API version: 1.39
Go version: go1.12.4
Git commit: 481bc77156
Built: Sat May 11 06:11:03 2019
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.09.6-ce
API version: 1.39 (minimum version 1.12)
Go version: go1.12.4
Git commit: 481bc77156
Built: Sat May 11 06:10:35 2019
OS/Arch: linux/amd64
Experimental: false
docker-compose version
docker-compose version 1.24.0, build unknown
docker-py version: 3.7.2
CPython version: 3.7.3
OpenSSL version: OpenSSL 1.1.1b 26 Feb 2019
I managed to get this working by following the setup instructions in the readme inside Docker-in-Docker
I don't think that's a sensible way to have to use this (probably) but it suggests that it works at least on alpine linux (dind base image) and arch linux and some ubuntu (that jaron was using) so we need to figure out what it is about Delena's environment that makes it not work.
docker exec -ti dind /bin/ash
Delena said she had trouble setting up - I'll let her fill in details if needed