Dockerize the project

[immanuelfodor]

Fixes #26

[marlluslustosa]

Thanks for reviewing and adding comments to the code. His recommendations were very pertinent.

The code was changed and I removed all ARGs type variables and turned them to ENV to improve their practical docker run applicability instead of writing this data statically.

Only one thing, if you are changing the application port, you have to change WEB_PORT as well as port redirects to WEB_PORT:WEB_PORT. In my case was set 8088.

This problem occurs because the politepol application uses a nginx web server running on port 80 which serves as a proxy to access two applications (the server running on port 8000 and downloader running on port 1234).

The problem is that if you only redirect any port (eg 4646) to container port 80, when the request falls into nginx (80) it cannot do this redirection within the container. But if both the external port and the internal port are the same (for example, -p 4646: 4646), it will be able to redirect traffic and access the application.

The idea I had was as follows: Inside start.sh I used a 'sed' command to replace the port when executing the container. Through the following code:

sed -i -e 's/listen\ 80/listen\ '${WEB_PORT}'/g' \
    -e 's/\[::\]:80/\[::\]:'${WEB_PORT}'/g' /etc/nginx/sites-available/default \
    && service nginx reload

So you have to specify the port with both the -e $WEB_PORT=8088 and internal redirect -p 8088:8088 variables.'

This solution is not the most elegant, as I believe it would be best to change this directly in nginx (some proxy forwarder setting).

I only tested it indoors. I didn't use it with a nginx container getting connections to this container (with .com domain).

It was open to suggestions.

Hugs

[immanuelfodor]

Fantastic news that it got merged! It's a bit weird, yes, that the web port should be hacked this way but it works fine, it's still better to have it in Docker than setting up dependencies on the host :)

However, I think I've found one bug. I renamed the dbpolitepol service and container to politepol-db and all its occurrences to match my current naming scheme, and when I started up the compose file, Django failed with an error: django.db.utils.OperationalError: (2003, "Can't connect to MySQL server on 'politepol-db' (111)"). The wait for it command contains a host:port pair of db:3306, so I needed to rename it as well, then it started up fine. I think, this is where the db should be dbpolitepol in the repo as well to make it work, as the hostname of the MySQL container is not db but dbpolitepol. I sadly did not try it with the original docker compose file, so it might work for the service prefix (db...) but I think it fails there with the same exception, so it's better to use the service name in the command nonetheless.

[marlluslustosa]

Hi, @immanuelfodor.

I fixed the bug here https://github.com/taroved/pol/pull/35.

Actually I had not removed ENTRYPOINT from the Dockerfile file so it always tried to connect to the mysql container before execution of the wait-for-it script.

On your the second try it could execute because the container was already created (and not necessarily because you renamed 'db' to 'dbpolitepol').

I had to change the paths for the start.sh script to be able to find the files in their folders.

Now it's ok, apparently. In the debug logs the wait-for-it.sh script waits for the mysql container and only then does the main politepol initiate the database connection.

Please do the test again.

Thanks. hugs

[immanuelfodor]

Yup, it looks good, works fine now indeed, good job! 🎉

[marlluslustosa]

@immanuelfodor, i have no way to test now, but I saw in a docker-compose.yaml file from another project that it is possible to replace "dbpolitepol:3306" for "$$DB_HOST:$$DB_PORT" at the line:

command: ["./wait-for-it.sh", "$$DB_HOST:$$DB_PORT", "--", "/bin/bash", "./frontend/start.sh"]

Can you test it?

[immanuelfodor]

With the double $s I get:

politepol       | wait-for-it.sh: waiting 15 seconds for $DB_HOST:$DB_PORT
politepol       | wait-for-it.sh: timeout occurred after waiting 15 seconds for $DB_HOST:$DB_PORT

But Django works then (probably because of the 15 secs passed).

With a single $ I get an error from wait for it that I should provide host and port:

politepol       | Error: you need to provide a host and port to test.
politepol       | Usage:
politepol       |     wait-for-it.sh host:port [-s] [-t timeout] [-- command args]

The only way it works as it seems is the original host:port pair.

[marlluslustosa]

I tested and worked normally with me.

command: ["./wait-for-it.sh", "$$DB_HOST:$$DB_PORT", "--", "/bin/bash", "./frontend/start.sh"]

[immanuelfodor]

Tested again, I still get this:

politepol       | wait-for-it.sh: waiting 15 seconds for $DB_HOST:$DB_PORT     
politepol       | wait-for-it.sh: timeout occurred after waiting 15 seconds for $DB_HOST:$DB_PORT

When using the hostname and port, it's fine:

politepol       | wait-for-it.sh: waiting 15 seconds for politepol-db:3306
politepol       | wait-for-it.sh: politepol-db:3306 is available after 0 seconds

[b1n4ryj4n]

if i try to run it with docker-compose up -d --build i get always a mysql_config not found error message

`Step 5/8 : RUN pip install -r requirements.txt ---> Running in 53790703b857 DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Collecting Django==1.8.6 Downloading Django-1.8.6-py2.py3-none-any.whl (6.2 MB) Collecting lxml==3.8.0 Downloading lxml-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl (6.8 MB) Collecting Scrapy==1.4.0 Downloading Scrapy-1.4.0-py2.py3-none-any.whl (248 kB) Collecting django-pipeline==1.5.4 Downloading django_pipeline-1.5.4-py2.py3-none-any.whl (29 kB) Collecting mysqlclient==1.3.7 Downloading mysqlclient-1.3.7.tar.gz (79 kB) ERROR: Command errored out with exit status 1: command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-SSDek8/mysqlclient/setup.py'"'"'; file='"'"'/tmp/pip-install-SSDek8/mysqlclient/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-SSDek8/mysqlclient/pip-egg-info cwd: /tmp/pip-install-SSDek8/mysqlclient/ Complete output (10 lines): sh: 1: mysql_config: not found Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-SSDek8/mysqlclient/setup.py", line 17, in metadata, options = get_config() File "setup_posix.py", line 44, in get_config libs = mysql_config("libs_r") File "setup_posix.py", line 26, in mysql_config raise EnvironmentError("%s not found" % (mysql_config.path,)) EnvironmentError: mysql_config not found

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ERROR: Service 'politepol' failed to build: The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1`

how can i fix that?

[marlluslustosa]

fix. https://github.com/taroved/pol/pull/42