build/test errors

[scalefree]

So I'm finally at the build & test steps. I believe I have everything except SMTP properly configured & I'm working on getting email creds. Build output looked clean but came up with a few errors in the test scripts & when I try the URLs I get nothing.

From initialize-data-stores.sh I get this:

Starting wsdockercommunity_redis_1 ... Starting wsdockercommunity_frontend_1 ... Starting wsdockercommunity_redis_1 Starting wsdockercommunity_api_1 ... Starting wsdockercommunity_frontend_1 Starting wsdockercommunity_rabbitmq_1 ... Starting wsdockercommunity_api_1 Starting wsdockercommunity_rabbitmq_1 Starting wsdockercommunity_tasknode_1 ... Starting wsdockercommunity_tasknode_1 Starting wsdockercommunity_frontend_1 ... done Starting wsdockercommunity_rabbitmq_1 ... done Starting wsdockercommunity_tasknode_1 ... done Starting wsdockercommunity_api_1 ... done Starting wsdockercommunity_redis_1 ... done Error response from daemon: Container 0f4583251eeec7b561ebee1a9fd5a0e983de379eec280d70a00431e0c9c313ce is not running Stopping wsdockercommunity_rabbitmq_1 ... Stopping wsdockercommunity_frontend_1 ... Stopping wsdockercommunity_redis_1 ... 1A2KStopping wsdockercommunity_redis_1 ... done Stopping wsdockercommunity_frontend_1 ... done Stopping wsdockercommunity_rabbitmq_1 ... done Going to remove wsdockercommunity_rabbitmq_1, wsdockercommunity_frontend_1, wsdockercommunity_api_1, wsdockercommunity_tasknode_1, wsdockercommunity_redis_1 Are you sure? [yN] y Removing wsdockercommunity_rabbitmq_1 ... Removing wsdockercommunity_frontend_1 ... Removing wsdockercommunity_api_1 ... Removing wsdockercommunity_tasknode_1 ... Removing wsdockercommunity_redis_1 ... Removing wsdockercommunity_frontend_1 ... done Removing wsdockercommunity_tasknode_1 ... done Removing wsdockercommunity_rabbitmq_1 ... done Removing wsdockercommunity_api_1 ... done Removing wsdockercommunity_redis_1 ... done

From run-tasknode-tests.sh I get this:

Creating wsdockercommunity_redis_1 ... Creating wsdockercommunity_frontend_1 ... Creating wsdockercommunity_rabbitmq_1 ... Creating wsdockercommunity_api_1 ... Creating wsdockercommunity_tasknode_1 ... Creating wsdockercommunity_rabbitmq_1 Creating wsdockercommunity_tasknode_1 Creating wsdockercommunity_redis_1 Creating wsdockercommunity_frontend_1 Creating wsdockercommunity_api_1 Creating wsdockercommunity_tasknode_1 ... done Creating wsdockercommunity_redis_1 ... done Creating wsdockercommunity_frontend_1 ... done Creating wsdockercommunity_rabbitmq_1 ... done Creating wsdockercommunity_api_1 ... done "docker exec" requires at least 2 arguments. See 'docker exec --help'.

Usage: docker exec [OPTIONS] CONTAINER COMMAND [ARG...]

Run a command in a running container Stopping wsdockercommunity_frontend_1 ... Stopping wsdockercommunity_rabbitmq_1 ... Stopping wsdockercommunity_redis_1 ... Stopping wsdockercommunity_redis_1 ... done Stopping wsdockercommunity_frontend_1 ... done Stopping wsdockercommunity_rabbitmq_1 ... done Going to remove wsdockercommunity_frontend_1, wsdockercommunity_api_1, wsdockercommunity_rabbitmq_1, wsdockercommunity_redis_1, wsdockercommunity_tasknode_1 Are you sure? [yN] y Removing wsdockercommunity_frontend_1 ... Removing wsdockercommunity_api_1 ... Removing wsdockercommunity_rabbitmq_1 ... Removing wsdockercommunity_redis_1 ... Removing wsdockercommunity_tasknode_1 ... Removing wsdockercommunity_frontend_1 ... done Removing wsdockercommunity_tasknode_1 ... done Removing wsdockercommunity_api_1 ... done Removing wsdockercommunity_redis_1 ... done Removing wsdockercommunity_rabbitmq_1 ... done

From run-api-tests.sh:

Creating wsdockercommunity_redis_1 ... Creating wsdockercommunity_frontend_1 ... Creating wsdockercommunity_api_1 ... Creating wsdockercommunity_tasknode_1 ... Creating wsdockercommunity_rabbitmq_1 ... Creating wsdockercommunity_redis_1 Creating wsdockercommunity_api_1 Creating wsdockercommunity_tasknode_1 Creating wsdockercommunity_frontend_1 Creating wsdockercommunity_rabbitmq_1 Creating wsdockercommunity_frontend_1 ... done Creating wsdockercommunity_api_1 ... done Creating wsdockercommunity_redis_1 ... done Creating wsdockercommunity_rabbitmq_1 ... done Creating wsdockercommunity_tasknode_1 ... done "docker exec" requires at least 2 arguments. See 'docker exec --help'.

Usage: docker exec [OPTIONS] CONTAINER COMMAND [ARG...]

Run a command in a running container Stopping wsdockercommunity_redis_1 ... Stopping wsdockercommunity_rabbitmq_1 ... Stopping wsdockercommunity_tasknode_1 ... Stopping wsdockercommunity_frontend_1 ... Stopping wsdockercommunity_tasknode_1 ... done Stopping wsdockercommunity_redis_1 ... done Stopping wsdockercommunity_frontend_1 ... done Stopping wsdockercommunity_rabbitmq_1 ... done Going to remove wsdockercommunity_redis_1, wsdockercommunity_rabbitmq_1, wsdockercommunity_api_1, wsdockercommunity_tasknode_1, wsdockercommunity_frontend_1 Are you sure? [yN] y Removing wsdockercommunity_redis_1 ... Removing wsdockercommunity_rabbitmq_1 ... Removing wsdockercommunity_api_1 ... Removing wsdockercommunity_tasknode_1 ... Removing wsdockercommunity_frontend_1 ... Removing wsdockercommunity_redis_1 ... done Removing wsdockercommunity_api_1 ... done Removing wsdockercommunity_tasknode_1 ... done Removing wsdockercommunity_rabbitmq_1 ... done Removing wsdockercommunity_frontend_1 ... done

From run-tasknode-deploy-checker.sh:

Creating wsdockercommunity_redis_1 ... Creating wsdockercommunity_api_1 ... Creating wsdockercommunity_rabbitmq_1 ... Creating wsdockercommunity_frontend_1 ... Creating wsdockercommunity_tasknode_1 ... Creating wsdockercommunity_redis_1 Creating wsdockercommunity_api_1 Creating wsdockercommunity_rabbitmq_1 Creating wsdockercommunity_frontend_1 Creating wsdockercommunity_tasknode_1 Creating wsdockercommunity_frontend_1 ... done Creating wsdockercommunity_tasknode_1 ... done Creating wsdockercommunity_api_1 ... done Creating wsdockercommunity_rabbitmq_1 ... done Creating wsdockercommunity_redis_1 ... done Error response from daemon: Container 258fb498b332b523057565a928c34e3e66cce9c8a59fe2619a561ccf909bf0b3 is not running Stopping wsdockercommunity_rabbitmq_1 ... Stopping wsdockercommunity_frontend_1 ... Stopping wsdockercommunity_redis_1 ... Stopping wsdockercommunity_redis_1 ... done Stopping wsdockercommunity_frontend_1 ... done Stopping wsdockercommunity_rabbitmq_1 ... done Going to remove wsdockercommunity_api_1, wsdockercommunity_rabbitmq_1, wsdockercommunity_frontend_1, wsdockercommunity_tasknode_1, wsdockercommunity_redis_1 Are you sure? [yN] y Removing wsdockercommunity_api_1 ... Removing wsdockercommunity_rabbitmq_1 ... Removing wsdockercommunity_frontend_1 ... Removing wsdockercommunity_tasknode_1 ... Removing wsdockercommunity_redis_1 ... Removing wsdockercommunity_frontend_1 ... done Removing wsdockercommunity_tasknode_1 ... done Removing wsdockercommunity_redis_1 ... done Removing wsdockercommunity_api_1 ... done Removing wsdockercommunity_rabbitmq_1 ... done

[scalefree]

Bump. Chris, there's potentially money on the table if you can help get this working for me. Reach out?

[lavalamp-]

Hey @scalefree - sorry this slipped through the cracks, and no worries re: the money. I'm not sure exactly what's going on here - in some cases it appears the output from the Docker commands is not lining up with what the scripts were built for.

First, what version of Docker are you running?

Let's start with debugging the first issue you've listed - running initialize-data-stores.sh. What is the output of running the following commands from the ws-docker-community repo directory:

docker-compose down
docker-compose rm
docker-compose up -d
docker ps | grep wsbackend-tasknode

[scalefree]

Lot of done's & the grep comes up empty.

[lavalamp-]

How about just docker ps then?

[scalefree]

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9dab4842aa24 redis:latest "docker-entrypoint..." 14 minutes ago Up 14 minutes 6379/tcp wsdockercommunity_redis_1 e3a74bdee8d9 rabbitmq:3-management "docker-entrypoint..." 14 minutes ago Up 14 minutes 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp wsdockercommunity_rabbitmq_1 5dd9fc0a5c2b wsfrontend:latest "nginx -g 'daemon ..." 14 minutes ago Up 14 minutes 443/tcp, 0.0.0.0:8001->80/tcp wsdockercommunity_frontend_1

[scalefree]

Forgot the version, here it is: Docker version 17.09.0-ce, build afdb6d4 Running on macOS Sierra

[lavalamp-]

Hey @scalefree - it appears that Docker has changed their automatic naming convention for their containers in docker-compose.

I've pushed a change to the scripts that should address this (https://github.com/lavalamp-/ws-docker-community/commit/a8500c86183b974031347094e31c10d7b9edd319). Please pull from master and see if that fixed everything for you.

[scalefree]

Pulled latest build. Running initialize-data-stores.sh I get this:

Recreating wsdockercommunity_tasknode_1 ... done Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 307, in execute settings.INSTALLED_APPS File "/usr/local/lib/python2.7/site-packages/django/conf/init.py", line 56, in getattr self._setup(name) File "/usr/local/lib/python2.7/site-packages/django/conf/init.py", line 41, in _setup self._wrapped = Settings(settings_module) File "/usr/local/lib/python2.7/site-packages/django/conf/init.py", line 110, in init mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/local/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/ws-backend/wsbackend/settings.py", line 162, in [[CORS_ORIGINS]] NameError: name 'CORS_ORIGINS' is not defined Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 307, in execute settings.INSTALLED_APPS File "/usr/local/lib/python2.7/site-packages/django/conf/init.py", line 56, in getattr self._setup(name) File "/usr/local/lib/python2.7/site-packages/django/conf/init.py", line 41, in _setup self._wrapped = Settings(settings_module) File "/usr/local/lib/python2.7/site-packages/django/conf/init.py", line 110, in init mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/local/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/ws-backend/wsbackend/settings.py", line 162, in [[CORS_ORIGINS]] NameError: name 'CORS_ORIGINS' is not defined

[lavalamp-]

Hey @scalefree - it appears that the contents of your settings.py file has not been fully updated.

Update the configs/settings.py file to reflect the following:

CORS_ORIGIN_WHITELIST = (
    'localhost:8001',
    '127.0.0.1:8001'
)

From there you will want to delete all of the existing dependent images and rebuild them:

docker rmi wsbackend-tasknode
docker rmi wsbackend-api
docker rmi wsbackend-base
./scripts/build-all-images.sh

Once the images are rebuilt try running the above again.

[scalefree]

Wiped & rebuilt from scratch. Only error I got was for ALLOWED_HOSTS. Verified postgres, elasticsearch & kibana all running. I noticed latest tasknode.cfg has es_host = elasticsearch, mine's on localhost. Aside from that, all green but still no joy.

[lavalamp-]

In the most recent commit to master I've dockerized Elasticsearch. If you've already set up Elasticsearch locally and want to use it outside of Docker you're still fine but you can also change that tasknode.cfg value to es_host = elasticsearch and even more of the deployment is abstracted away into Docker.

In reference to the ALLOWED_HOSTS issue, adjust your configs/settings.py file to reflect the following:

ALLOWED_HOSTS = [
    '127.0.0.1'
]

Once you've done that, wipe and rebuild again with the following:

docker rmi wsbackend-tasknode
docker rmi wsbackend-api
docker rmi wsbackend-base
./scripts/build-all-images.sh

Hopefully that should finish it!

[scalefree]

So here's a new one. Running run-api-tests.sh:

Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 337, in execute django.setup() File "/usr/local/lib/python2.7/site-packages/django/init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate app_config.import_models() File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models self.models_module = import_module(models_module_name) File "/usr/local/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/ws-backend/rest/models/init.py", line 12, in from .dns import ( File "/ws-backend/rest/models/dns.py", line 6, in from lib import JsonSerializableMixin File "/ws-backend/lib/init.py", line 102, in from .pubsub import File "/ws-backend/lib/pubsub/init.py", line 4, in from .connector import ( File "/ws-backend/lib/pubsub/connector.py", line 25, in class BasePubSubConnector(object): File "/ws-backend/lib/pubsub/connector.py", line 37, in BasePubSubConnector def publish_message(self, topic=config.pubsub_publish_topic, message=None): File "/ws-backend/lib/config.py", line 1043, in pubsub_publish_topic return self.get_string("PubSub", "pubsub_publish_topic") File "/ws-backend/lib/config.py", line 104, in __get_string return str(self.get_value(section, name)) File "/ws-backend/lib/config.py", line 89, in get_value return self.config.get(section, name) File "/usr/local/lib/python2.7/ConfigParser.py", line 607, in get raise NoSectionError(section) ConfigParser.NoSectionError: No section: 'PubSub' Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 337, in execute django.setup() File "/usr/local/lib/python2.7/site-packages/django/init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate app_config.import_models() File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models self.models_module = import_module(models_module_name) File "/usr/local/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/ws-backend/rest/models/init.py", line 12, in from .dns import ( File "/ws-backend/rest/models/dns.py", line 6, in from lib import JsonSerializableMixin File "/ws-backend/lib/init__.py", line 102, in from .pubsub import File "/ws-backend/lib/pubsub/init.py", line 4, in from .connector import ( File "/ws-backend/lib/pubsub/connector.py", line 25, in class BasePubSubConnector(object): File "/ws-backend/lib/pubsub/connector.py", line 37, in BasePubSubConnector def publish_message(self, topic=config.pubsub_publish_topic, message=None): File "/ws-backend/lib/config.py", line 1043, in pubsub_publish_topic return self.get_string("PubSub", "pubsub_publish_topic") File "/ws-backend/lib/config.py", line 104, in __get_string return str(self.get_value(section, name)) File "/ws-backend/lib/config.py", line 89, in __get_value return self.config.get(section, name) File "/usr/local/lib/python2.7/ConfigParser.py", line 607, in get raise NoSectionError(section) ConfigParser.NoSectionError: No section: 'PubSub'

[lavalamp-]

The tasknode.cfg.example config file has been updated since you last pulled the ws-docker-community repository. Please take a look at your tasknode.cfg file and ensure that it has all of the configuration values found in tasknode.cfg.example, rebuild, and try again.

[scalefree]

Wiped, fresh download & install again just in case, got the latest changes into tasknode.cfg & settings.py (used the .example files & copied values into them then renamed). Still get the PubSub error, this time from initialize-data-stores.sh.

Didn't seem to affect the build but I did notice these while running build-all-images.sh:

W: Size of file /var/lib/apt/lists/security.debian.org_dists_jessie_updates_main_binary-amd64_Packages.gz is not what the server reported 587960 609758 [...] debconf: delaying package configuration, since apt-utils is not installed

[lavalamp-]

That error is specifically stating that the PubSub section within tasknode.cfg is not present. To confirm, you can shell into a Docker container of the image:

docker run -it wsbackend-api:latest /bin/bash

Once in the container at the bash prompt you can look at the contents of the tasknode.cfg file as follows:

cat tasknode/tasknode.cfg

I have a feeling you will find that the PubSub section is not present. As of right now the configs are (unfortunately) baked into the images instead of loaded in as a volume as I ran into an annoying Docker bug when I was putting this repo together. Ergo, if PubSub is not present in the config file, you will need to re-bake the image:

docker rmi wsbackend-tasknode
docker rmi wsbackend-api
docker rmi wsbackend-base
./scripts/build-all-images.sh

Give that a try and if it's still complaining about PubSub then I can take a deeper dive.

[scalefree]

Time for a deeper dive. I've done two clean builds (docker rmi, freshly downloaded source, copy/paste data into config files then rename) & both times got hit with PubSub. Sorry.

[lavalamp-]

And when you go into the container and cat tasknode/tasknode.cfg do you see a section entitled [PubSub]?

[lavalamp-]

Better yet - try running this and paste the output here:

docker run -it wsbackend-api:latest grep -i pubsub /ws-backend/tasknode/tasknode.cfg

[scalefree]

Did not see any such section, no.

[scalefree]

Came up empty.

[lavalamp-]

As I stated previously then, either the contents of your configs/tasknode.cfg file does not include the PubSub section or that file has not been baked into your images. Perhaps you are getting this mixed up with configs/tasknode.cfg.example? Either that or your images aren't being deleted with the docker rmi invocations.

[lavalamp-]

Try building the backend base image without using a cache using the following command:

docker build --no-cache -f dockerfiles/backend-base.dockerfile -t wsbackend-base:latest .

Once the image is built run this and see if the PubSub section is now found in your image:

docker run -it wsbackend-base:latest grep -i pubsub /ws-backend/tasknode/tasknode.cfg

Please confirm that the contents of your configs/tasknode.cfg file are correct as well.

[scalefree]

No-cache build came up empty too. Each build is as clean as I can make it. I'm doing docker rmi -f on all 3, deleting the whole ws-docker-community file structure & getting a fresh one from GitHub. I use the latest tasknode.cfg & settings.py & copy/paste data into them leaving the rest alone.

[lavalamp-]

And what about this:

grep -i pubsub configs/tasknode.cfg

[scalefree]

In tasknode.cfg I replace AWS, DB & SMTP. In settings.py I replace Django SECRET_KEY, ALLOWED_HOSTS, DATABASES, CORS_ORIGIN_WHITELIST & EMAIL. All else stays intact.

[scalefree]

Checking GitHub I don't see any mention of pubsub in ws-docker-community/configs/tasknode.cfg.example of Master branch.

[lavalamp-]

Everything that needs to be replaced in the configuration files is surrounded by square brackets (with the exception of dns_dnsdb_api_key if you're not using dnsdb and gcp_project_name if you're not using Google Cloud Platform). If you've replaced all the necessary fields, then the following commands should return empty (minus the exemptions mentioned previously):

grep "\[\[" configs/tasknode.cfg
grep "\[\[" configs/settings.py

[lavalamp-]

Aha - the tasknode.cfg.example file in this repo hasn't been updated to reflect the state of ws-backend-community. Use this file instead:

https://github.com/lavalamp-/ws-backend-community/blob/master/tasknode/tasknode.cfg.example

I'd push a change but I've got some other staged commits and don't want to break anything in master right now.

[scalefree]

Taking a break, will pick it back up tonight or tomorrow. Thanks for all your help.

[lavalamp-]

Of course - I'm checking to make sure that things are working with Dockerizing all of Postgres as well and will hopefully have it pushed up soon.

[scalefree]

Once we get this working I'm gonna be back on it with the enterprise edition, building that from scratch; cause Docker can't hold a couple hundred thousand hosts. Probably after Christmas break.

[scalefree]

I have postrgres running on 5432, verified listening on localhost using nmap -sV. But this is the error I get.

Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 355, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, *cmd_options) File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(args, options) File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/makemigrations.py", line 110, in handle loader.check_consistent_history(connection) File "/usr/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 282, in check_consistent_history applied = recorder.applied_migrations() File "/usr/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations self.ensure_schema() File "/usr/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()): File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor return self._cursor() File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor self.ensure_connection() File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection self.connect() File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection self.connect() File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 189, in connect self.connection = self.get_new_connection(conn_params) File "/usr/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection connection = Database.connect(conn_params) File "/usr/local/lib/python2.7/site-packages/psycopg2/init.py", line 130, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) django.db.utils.OperationalError: could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432? could not connect to server: Cannot assign requested address Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5432?

[zeoculls]

First of all thank you @lavalamp- for the contribution. I would like to test your project in the docker deployment but I having the same that @scalefree.

@scalefree are you getting the 8001 port open? I thing webSight is listening in that port at least in my case.

Starting wsdockercommunity_frontend_1 ... done Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 337, in execute django.setup() File "/usr/local/lib/python2.7/site-packages/django/init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate app_config.import_models() File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models self.models_module = import_module(models_module_name) File "/usr/local/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/ws-backend/rest/models/init.py", line 12, in from .dns import ( File "/ws-backend/rest/models/dns.py", line 6, in from lib import JsonSerializableMixin File "/ws-backend/lib/init.py", line 102, in from .pubsub import * File "/ws-backend/lib/pubsub/init.py", line 4, in from .connector import ( File "/ws-backend/lib/pubsub/connector.py", line 25, in class BasePubSubConnector(object): File "/ws-backend/lib/pubsub/connector.py", line 37, in BasePubSubConnector def publish_message(self, topic=config.pubsub_publish_topic, message=None): File "/ws-backend/lib/config.py", line 1043, in pubsub_publish_topic return self.get_string("PubSub", "pubsub_publish_topic") File "/ws-backend/lib/config.py", line 104, in __get_string return str(self.get_value(section, name)) File "/ws-backend/lib/config.py", line 89, in __get_value return self.config.get(section, name) File "/usr/local/lib/python2.7/ConfigParser.py", line 607, in get raise NoSectionError(section) ConfigParser.NoSectionError: No section: 'PubSub'

[zeoculls]

In my case during the first user sign up, I am not getting any response from the POST to the API (port 8000) path /users/. So I am not getting any responde from the backend.

Any suggestion on how I can debug what is happening?

[scalefree]

@zeoculls check above for solution on Pubsub. Need to get tasknode.cfg.example that's not been pushed to Master yet.

[lavalamp-]

Hey all - I'll do what I can to get the most recent pushes out to master tonight. I'm in the process of Dockerizing all of PostgreSQL so that it's even easier to set up without dependencies and have to figure out this one bug...

[lavalamp-]

@scalefree - so you have your tasknode.cfg and settings.py files configured to point to localhost for your database?

[scalefree]

Yes on both. localhost:5432. Verified listening using nmap.

[scalefree]

Chris, what's your expected timeframe on dockerizing postgres? Is your time best spent on that than my bug? If it's just a couple days I can wait.

[lavalamp-]

@scalefree no worries - this one is fairly easy.

So "localhost" in the context of the Docker containers is not the same as "localhost" on your local system. Every container is effectively its own VM with its own IP address. As such when your Docker container is attempting to connect to "localhost" it's attempting to connect to itself. Instead, do the following:

• Bind your PostgreSQL database to an interface other than localhost
• Reconfigure your Web Sight deployment to connect to the IP address associated with this new interface

This should address your issue.

[scalefree]

Yes! We're moving forward. Postgres is populating. On to the next hurdle, elastic.

Traceback (most recent call last): File "manage.py", line 23, in execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/init.py", line 355, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, *cmd_options) File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(args, **options) File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/shell.py", line 95, in handle exec(options['command']) File "", line 1, in File "/ws-backend/lib/bootstrap.py", line 32, in bootstrap_data_stores bootstrap_elasticsearch() File "/ws-backend/lib/bootstrap.py", line 52, in bootstrap_elasticsearch update_model_mappings() File "/ws-backend/wselasticsearch/bootstrap.py", line 50, in update_model_mappings bootstrap_index_model_mappings(index=config.es_default_index, delete_first=delete_first) File "/ws-backend/wselasticsearch/bootstrap.py", line 31, in bootstrap_index_model_mappings indices = es_helper.get_indices() File "/ws-backend/wselasticsearch/helper.py", line 242, in get_indices return self.connection.indices.get_alias().keys() File "/ws-backend/wselasticsearch/helper.py", line 423, in connection self._connection = self.get_es_connection() File "/ws-backend/wselasticsearch/helper.py", line 386, in get_es_connection return self.get_standard_es_connection() File "/ws-backend/wselasticsearch/helper.py", line 400, in __get_standard_es_connection if config.es_use_http_auth: File "/ws-backend/lib/config.py", line 630, in es_use_http_auth return self.get_bool("Elasticsearch", "es_use_http_auth") File "/ws-backend/lib/config.py", line 58, in get_bool return True if self.__get_value(section, name) == "True" else False File "/ws-backend/lib/config.py", line 89, in get_value return self.config.get(section, name) File "/usr/local/lib/python2.7/ConfigParser.py", line 618, in get raise NoOptionError(option, section) ConfigParser.NoOptionError: No option 'es_use_http_auth' in section: 'Elasticsearch' Username:

[lavalamp-]

Hey @scalefree - this is the same problem as last time. The value can be found in tasknode.cfg.example here:

https://github.com/lavalamp-/ws-backend-community/blob/master/tasknode/tasknode.cfg.example#L69

Please update and try again.

[scalefree]

This has a bunch of Celery fields marked [[REPLACE]]. Is OK?

[scalefree]

Also has [[REPLACE]] for all ES fields. Are you sure this is right version to use?

[lavalamp-]

@scalefree - place the es_use_http_auth value into your config, rebuild, and see if that fixes the error. No reason to replace everything in your config file.

[scalefree]

The value for es_use_http_auth is True. Tried it, got the password prompt again. Tried setting it to false, also got the prompt.

[lavalamp-]

@scalefree - what password prompt?

[scalefree]

Excuse me, username prompt.

ConfigParser.NoOptionError: No option 'es_username' in section: 'Elasticsearch' Username:

That's with es_use_http_auth = True, same as in the file you pointed me to. Tried setting it to False just to see, slightly different error but again the prompt.

[lavalamp-]

Whenever you see an error being triggered by a lack of a configuration value, go ahead and add that value to the configuration file. In this case, it doesn't matter what the value of the configuration key es_username is as it won't be used since es_use_http_auth is false.

https://github.com/lavalamp-/ws-backend-community/blob/master/tasknode/tasknode.cfg.example#L70

[lavalamp-]

I just pushed to master:

https://github.com/lavalamp-/ws-docker-community/commit/d032709c5416d259060723b0b5363b7f446928db

Please pull from latest and re-do your config file again. This time around the only things you'll need to be populating are aws_key_id and aws_secret_key in tasknode.cfg. The settings.py file you have now should remain unchanged.

Let me know if that works for you.