search

GeoNode / geonode-project

A django template project for creating custom GeoNode projects.
http://geonode.org
79 stars 174 forks source link

Installing Geonode 4.x in Docker always turns into a restart of nginx #330

Closed Cornacap closed 2 years ago

Cornacap commented 2 years ago

Hello, I tried out the installation the repo of 4.x, but the nginx-container is stuck in a rebooting: 7ddd24414a93 geonode/nginx:3.x "/docker-entrypoint.…" 16 seconds ago Restarting (1) 3 seconds ago nginx4geonode4

I looked into the logs: 2022/06/07 13:07:34 [emerg] 1#1: directive "server_name" is not terminated by ";" in /etc/nginx/nginx.https.enabled.conf:11 nginx: [emerg] directive "server_name" is not terminated by ";" in /etc/nginx/nginx.https.enabled.conf:11

Where exactly do I add the missing semicolon?

frafra commented 2 years ago

https://github.com/GeoNode/geonode-project/blob/ecc97dfd0a3b5227c32a2f689f96244af370436c/docker/nginx/nginx.https.available.conf.envsubst#L11

https://github.com/GeoNode/geonode-project/blob/ecc97dfd0a3b5227c32a2f689f96244af370436c/docker-compose.yml#L55-L56

https://github.com/GeoNode/geonode-project/blob/ecc97dfd0a3b5227c32a2f689f96244af370436c/docker/nginx/docker-entrypoint.sh#L51-L53

It looks like you set HTTPS_HOST a string with just spaces instead of leave it empty:

$ HTTPS_HOST=""; if [ -z "${HTTPS_HOST}" ]; then echo disable; else echo enable; fi
disable
$ HTTPS_HOST=" "; if [ -z "${HTTPS_HOST}" ]; then echo disable; else echo enable; fi
enable

We could improve this check to avoid such kind of typos, by dropping blanks: https://unix.stackexchange.com/a/146945/41231

Cornacap commented 2 years ago

Thank you for replying so quickly, frafra! Well, I haven't touched anything and the docker-entrypoint.sh looks the same as yours (without blank).

frafra commented 2 years ago

You are welcome :) Variables are set in the .env file. Have a look there.

Cornacap commented 2 years ago

I didn't touched them either since I just wanted to run it locally at first. So did I with the Versions of 3 - without any issue.

Cornacap commented 2 years ago

One thing I did beforehand: I deactivated jenkins. Now I activated it again and get this: `ERROR: for django4geonode4 Cannot start service django: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/usr/src/{{project_name}}/entrypoint.sh": stat /usr/src/{{project_name}}/entrypoint.sh: no such file or directory: unknown

ERROR: for django Cannot start service django: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/usr/src/{{project_name}}/entrypoint.sh": stat /usr/src/{{project_name}}/entrypoint.sh: no such file or directory: unknown ERROR: Encountered errors while bringing up the project.`

frafra commented 2 years ago

Ok, so you just skipped the configuration of the geonode-project. Please read the README.md file: https://github.com/GeoNode/geonode-project#create-a-custom-project

Cornacap commented 2 years ago

No, I created (as always) a project, like described in the readme.md:

'git clone https://github.com/GeoNode/geonode-project.git -b source /usr/share/virtualenvwrapper/virtualenvwrapper.sh mkvirtualenv --python=/usr/bin/python3 {{ project_name }} pip install Django==3.2.12

django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile {{ project_name }}

cd {{ project_name }}'

frafra commented 2 years ago

I guess you followed this:

replace {{ project_name }} with whatever you would like to name your project.

If that is the case, have a look at your compose file. It looks like you are not using the one that has been generated.

Cornacap commented 2 years ago

Well, isn't it automatically placed in the .env > COMPOSE_PROJECT_NAME=geonode4 (in my case) and then used in the compose file?. All seems fine:

.env

COMPOSE_PROJECT_NAME=geonode4
DOCKER_HOST_IP=
DOCKER_ENV=production
# See https://github.com/geosolutions-it/geonode-generic/issues/28
# to see why we force API version to 1.24
DOCKER_API_VERSION="1.24"
BACKUPS_VOLUME_DRIVER=local

C_FORCE_ROOT=1
FORCE_REINIT=false
INVOKE_LOG_STDOUT=true

# LANGUAGE_CODE=pt
# LANGUAGES=(('en','English'),('pt','Portuguese'))

DJANGO_SETTINGS_MODULE=geonode4.settings
GEONODE_INSTANCE_NAME=geonode
GEONODE_LB_HOST_IP=
GEONODE_LB_PORT=

# #################
# backend
# #################
POSTGRES_USER=postgres
POSTGRES_PASSWORD={pgpwd}
GEONODE_DATABASE=geonode4
GEONODE_DATABASE_PASSWORD={dbpwd}
GEONODE_GEODATABASE=geonode4_data
GEONODE_GEODATABASE_PASSWORD={geodbpwd}
GEONODE_DATABASE_SCHEMA=public
GEONODE_GEODATABASE_SCHEMA=public
DATABASE_HOST=db
DATABASE_PORT=5432
DATABASE_URL=postgis://geonode4:{dbpwd}@db:5432/geonode4
GEODATABASE_URL=postgis://geonode4_data:{geodbpwd}@db:5432/geonode4_data
GEONODE_DB_CONN_MAX_AGE=0
GEONODE_DB_CONN_TOUT=5
DEFAULT_BACKEND_DATASTORE=datastore
BROKER_URL=amqp://guest:guest@rabbitmq:5672/
CELERY_BEAT_SCHEDULER=celery.beat:PersistentScheduler
ASYNC_SIGNALS=True

SITEURL=http://localhost/

ALLOWED_HOSTS="['django', '*', '{hostname}']"

# Data Uploader
DEFAULT_BACKEND_UPLOADER=geonode.importer
TIME_ENABLED=True
MOSAIC_ENABLED=False
HAYSTACK_SEARCH=False
HAYSTACK_ENGINE_URL=http://elasticsearch:9200/
HAYSTACK_ENGINE_INDEX_NAME=haystack
HAYSTACK_SEARCH_RESULTS_PER_PAGE=200

# #################
# Jenkins
# CI/CD Server
# #################
JENKINS_HTTP_PORT=9080
JENKINS_HTTPS_PORT=9443

# #################
# nginx
# HTTPD Server
# #################
GEONODE_LB_HOST_IP={hostname}
GEONODE_LB_PORT=80

# IP or domain name and port where the server can be reached on HTTPS (leave HOST empty if you want to use HTTP only)
# port where the server can be reached on HTTPS
HTTP_HOST={http_host}
HTTPS_HOST={https_host}

HTTP_PORT=80
HTTPS_PORT=443

# Let's Encrypt certificates for https encryption. You must have a domain name as HTTPS_HOST (doesn't work
# with an ip) and it must be reachable from the outside. This can be one of the following :
# disabled : we do not get a certificate at all (a placeholder certificate will be used)
# staging : we get staging certificates (are invalid, but allow to test the process completely and have much higher limit rates)
# production : we get a normal certificate (default)
LETSENCRYPT_MODE={letsencrypt_mode}
# LETSENCRYPT_MODE=staging
# LETSENCRYPT_MODE=production

RESOLVER=127.0.0.11

# #################
# geoserver
# #################
GEOSERVER_WEB_UI_LOCATION={geoserver_ui}/geoserver/
GEOSERVER_PUBLIC_LOCATION={geoserver_ui}/geoserver/
GEOSERVER_LOCATION=http://geoserver:8080/geoserver/
GEOSERVER_ADMIN_USER=admin
GEOSERVER_ADMIN_PASSWORD={geoserverpwd}

OGC_REQUEST_TIMEOUT=30
OGC_REQUEST_MAX_RETRIES=1
OGC_REQUEST_BACKOFF_FACTOR=0.3
OGC_REQUEST_POOL_MAXSIZE=10
OGC_REQUEST_POOL_CONNECTIONS=10

# Java Options & Memory
ENABLE_JSONP=true
outFormat=text/javascript
GEOSERVER_JAVA_OPTS=-Djava.awt.headless=true -Xms2G -Xmx4G -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=/var/log/jvm.log -XX:PerfDataSamplingInterval=500 -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT -Dorg.geotools.shapefile.datetime=false -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL={geoserver_ui}/geoserver/pdf -DALLOW_ENV_PARAMETRIZATION=true -Xbootclasspath/a:/usr/local/tomcat/webapps/geoserver/WEB-INF/lib/marlin-0.9.3-Unsafe.jar -Dsun.java2d.renderer=org.marlin.pisces.MarlinRenderingEngine

# #################
# Security
# #################
# Admin Settings
#
# ADMIN_PASSWORD is used to overwrite the GeoNode admin password **ONLY** the first time
# GeoNode is run. If you need to overwrite it again, you need to set the env var FORCE_REINIT,
# otherwise the invoke updateadmin task will be skipped and the current password already stored
# in DB will honored.

ADMIN_USERNAME=admin
ADMIN_PASSWORD={geonodepwd}
ADMIN_EMAIL={email}

# EMAIL Notifications
EMAIL_ENABLE=False
DJANGO_EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend
DJANGO_EMAIL_HOST=localhost
DJANGO_EMAIL_PORT=25
DJANGO_EMAIL_HOST_USER=
DJANGO_EMAIL_HOST_PASSWORD=
DJANGO_EMAIL_USE_TLS=False
DJANGO_EMAIL_USE_SSL=False
DEFAULT_FROM_EMAIL='{email}' # eg Company <no-reply@company.org>

# Session/Access Control
LOCKDOWN_GEONODE=False
CORS_ORIGIN_ALLOW_ALL=True
X_FRAME_OPTIONS="SAMEORIGIN"
SESSION_EXPIRED_CONTROL_ENABLED=True
DEFAULT_ANONYMOUS_VIEW_PERMISSION=True
DEFAULT_ANONYMOUS_DOWNLOAD_PERMISSION=True

# Users Registration
ACCOUNT_OPEN_SIGNUP=True
ACCOUNT_EMAIL_REQUIRED=True
ACCOUNT_APPROVAL_REQUIRED=False
ACCOUNT_CONFIRM_EMAIL_ON_GET=False
ACCOUNT_EMAIL_VERIFICATION=none
ACCOUNT_EMAIL_CONFIRMATION_EMAIL=False
ACCOUNT_EMAIL_CONFIRMATION_REQUIRED=False
ACCOUNT_AUTHENTICATION_METHOD=username_email
AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME=True

# OAuth2
OAUTH2_API_KEY=
OAUTH2_CLIENT_ID={clientid}
OAUTH2_CLIENT_SECRET={clientsecret}

# GeoNode APIs
API_LOCKDOWN=False
TASTYPIE_APIKEY=

# #################
# Production and
# Monitoring
# #################
DEBUG={debug}

SECRET_KEY="{secret_key}"

STATIC_ROOT=/mnt/volumes/statics/static/
MEDIA_ROOT=/mnt/volumes/statics/uploaded/
GEOIP_PATH=/mnt/volumes/statics/geoip.db

CACHE_BUSTING_STATIC_ENABLED=False

MEMCACHED_ENABLED=False
MEMCACHED_BACKEND=django.core.cache.backends.memcached.MemcachedCache
MEMCACHED_LOCATION=127.0.0.1:11211
MEMCACHED_LOCK_EXPIRE=3600
MEMCACHED_LOCK_TIMEOUT=10

MAX_DOCUMENT_SIZE=2
CLIENT_RESULTS_LIMIT=5
API_LIMIT_PER_PAGE=1000

# GIS Client
GEONODE_CLIENT_LAYER_PREVIEW_LIBRARY=mapstore
MAPBOX_ACCESS_TOKEN=
BING_API_KEY=
GOOGLE_API_KEY=

# Monitoring
MONITORING_ENABLED=True
MONITORING_DATA_TTL=365
USER_ANALYTICS_ENABLED=True
USER_ANALYTICS_GZIP=True
CENTRALIZED_DASHBOARD_ENABLED=False
MONITORING_SERVICE_NAME=local-geonode
MONITORING_HOST_NAME=geonode

# Other Options/Contribs
MODIFY_TOPICCATEGORY=True
AVATAR_GRAVATAR_SSL=True
EXIF_ENABLED=True
CREATE_LAYER=True
FAVORITE_ENABLED=True

# LDAP
LDAP_ENABLED=False
LDAP_SERVER_URL=ldap://<the_ldap_server>
LDAP_BIND_DN=uid=ldapinfo,cn=users,dc=ad,dc=example,dc=org
LDAP_BIND_PASSWORD=<something_secret>
LDAP_USER_SEARCH_DN=dc=ad,dc=example,dc=org
LDAP_USER_SEARCH_FILTERSTR=(&(uid=%(user)s)(objectClass=person))
LDAP_GROUP_SEARCH_DN=cn=groups,dc=ad,dc=example,dc=org
LDAP_GROUP_SEARCH_FILTERSTR=(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6))
LDAP_GROUP_PROFILE_MEMBER_ATTR=uniqueMember

# CELERY

# expressed in KB
# CELERY__MAX_MEMORY_PER_CHILD="200000"
# ##
# Note right autoscale value must coincide with worker concurrency value
# CELERY__AUTOSCALE_VALUES="1,4"
# CELERY__WORKER_CONCURRENCY="4"
# ##
# CELERY__OPTS="--without-gossip --without-mingle -Ofair -B -E"
# CELERY__BEAT_SCHEDULE="/mnt/volumes/statics/celerybeat-schedule"
# CELERY__LOG_LEVEL="INFO"
# CELERY__LOG_FILE="/var/log/celery.log"
# CELERY__WORKER_NAME="worker1@%h"

# PostgreSQL
POSTGRESQL_MAX_CONNECTIONS=200

# Common containers restart policy
RESTART_POLICY_CONDITION="on-failure"      
RESTART_POLICY_DELAY="5s"
RESTART_POLICY_MAX_ATTEMPTS="3"
RESTART_POLICY_WINDOW=120s

DEFAULT_MAX_UPLOAD_SIZE=5368709120
DEFAULT_MAX_PARALLEL_UPLOADS_PER_USER=5

docker-compose.yml

version: '3.4'

# Common Django template for GeoNode and Celery services below
x-common-django:
  &default-common-django
  image: ${COMPOSE_PROJECT_NAME}_django:latest
  restart: unless-stopped
  env_file:
    - .env
  volumes:
    # - '.:/usr/src/{{project_name}}'
    - statics:/mnt/volumes/statics
    - geoserver-data-dir:/geoserver_data/data
    - backup-restore:/backup_restore
    - data:/data
    - tmp:/tmp

services:

  # Our custom django application. It includes Geonode.
  django:
    << : *default-common-django
    build:
      context: ./
      dockerfile: Dockerfile
    container_name: django4${COMPOSE_PROJECT_NAME}
    healthcheck:
      test: "curl --fail --silent --write-out 'HTTP CODE : %{http_code}\n' --output /dev/null http://127.0.0.1:8001/"
      interval: 60s
      timeout: 10s
      retries: 1
      start_period: 60s
    environment:
      - IS_CELERY=False
    entrypoint: ["/usr/src/{{project_name}}/entrypoint.sh"]
    command: "uwsgi --ini /usr/src/{{project_name}}/uwsgi.ini"

  # Celery worker that executes celery tasks created by Django.
  celery:
    << : *default-common-django
    image: ${COMPOSE_PROJECT_NAME}_django:latest
    container_name: celery4${COMPOSE_PROJECT_NAME}
    depends_on:
      - django
    environment:
      - IS_CELERY=True
    entrypoint: ["/usr/src/{{project_name}}/entrypoint.sh"]
    command: "celery-cmd"

  # Nginx is serving django static and media files and proxies to django and geonode
  geonode:
    image: geonode/nginx:3.x
    build: ./docker/nginx/
    container_name: nginx4${COMPOSE_PROJECT_NAME}
    environment:
      - HTTPS_HOST=${HTTPS_HOST}
      - HTTP_HOST=${HTTP_HOST}
      - HTTPS_PORT=${HTTPS_PORT}
      - HTTP_PORT=${HTTP_PORT}
      - LETSENCRYPT_MODE=${LETSENCRYPT_MODE}
      - RESOLVER=127.0.0.11
    ports:
      - "${HTTP_PORT}:80"
      - "${HTTPS_PORT}:443"
    volumes:
      - nginx-certificates:/geonode-certificates
      - statics:/mnt/volumes/statics
    restart: unless-stopped

  # Gets and installs letsencrypt certificates
  letsencrypt:
    image: geonode/letsencrypt:latest
    build: ./docker/letsencrypt/
    container_name: letsencrypt4${COMPOSE_PROJECT_NAME}
    environment:
      - HTTPS_HOST=${HTTPS_HOST}
      - HTTP_HOST=${HTTP_HOST}
      - ADMIN_EMAIL=${ADMIN_EMAIL}
      - LETSENCRYPT_MODE=${LETSENCRYPT_MODE}
    volumes:
      - nginx-certificates:/geonode-certificates
    restart: unless-stopped

  # Geoserver backend
  geoserver:
    image: geonode/geoserver:2.19.6
    build: ./docker/geoserver/
    container_name: geoserver4${COMPOSE_PROJECT_NAME}
    healthcheck:
      test: "curl --fail --silent --write-out 'HTTP CODE : %{http_code}\n' --output /dev/null http://127.0.0.1:8080/geoserver/rest/workspaces/geonode.html"
      interval: 60s
      timeout: 10s
      retries: 1
      start_period: 60s
    env_file:
      - .env
    volumes:
      - statics:/mnt/volumes/statics
      - geoserver-data-dir:/geoserver_data/data
      - backup-restore:/backup_restore
      - data:/data
      - tmp:/tmp
    restart: unless-stopped
  data-dir-conf:
    image: geonode/geoserver_data:2.19.6
    container_name: gsconf4${COMPOSE_PROJECT_NAME}
    command: /bin/true
    volumes:
      - geoserver-data-dir:/geoserver_data/data

  # PostGIS database.
  db:
    # use geonode official postgis 13 image
    image: geonode/postgis:13
    command: postgres -c "max_connections=${POSTGRESQL_MAX_CONNECTIONS}"
    container_name: db4${COMPOSE_PROJECT_NAME}
    env_file:
      - .env
    volumes:
      - dbdata:/var/lib/postgresql/data
      - dbbackups:/pg_backups
    restart: unless-stopped
    # uncomment to enable remote connections to postgres
    #ports:
    #  - "5432:5432"

  # Vanilla RabbitMQ service. This is needed by celery
  rabbitmq:
    image: rabbitmq:3.7-alpine
    container_name: rabbitmq4${COMPOSE_PROJECT_NAME}
    volumes:
      - rabbitmq:/var/lib/rabbitmq
    restart: unless-stopped

  jenkins:
    image: jenkins/jenkins:lts
    # image: istresearch/jenkins:latest
    container_name: jenkins4${COMPOSE_PROJECT_NAME}
    user: jenkins
    ports:
      - '${JENKINS_HTTP_PORT}:${JENKINS_HTTP_PORT}'
      - '${JENKINS_HTTPS_PORT}:${JENKINS_HTTPS_PORT}'
      - '50000:50000'
    # network_mode: "host"
    volumes:
      - jenkins_data:/var/jenkins_home
      - backup-restore:/backup_restore
      - data:/data
    environment:
      - 'JENKINS_OPTS=--httpPort=${JENKINS_HTTP_PORT} --httpsPort=${JENKINS_HTTPS_PORT} --prefix=/jenkins'
    restart: unless-stopped
volumes:
  jenkins_data:
    driver: local
  statics:
    name: ${COMPOSE_PROJECT_NAME}-statics
  nginx-certificates:
    name: ${COMPOSE_PROJECT_NAME}-nginxcerts
  geoserver-data-dir:
    name: ${COMPOSE_PROJECT_NAME}-gsdatadir
  dbdata:
    name: ${COMPOSE_PROJECT_NAME}-dbdata
  dbbackups:
    name: ${COMPOSE_PROJECT_NAME}-dbbackups
  backup-restore:
    name: ${COMPOSE_PROJECT_NAME}-backup-restore
  data:
    name: ${COMPOSE_PROJECT_NAME}-data
  tmp:
    name: ${COMPOSE_PROJECT_NAME}-tmp
  rabbitmq:
    name: ${COMPOSE_PROJECT_NAME}-rabbitmq
frafra commented 2 years ago

As you can see, you still have {{project_name}} around, which means that the docker-compose.yml you are using has not been generated using django-admin startproject.

Cornacap commented 2 years ago

ok, so I'll have closer look again. Was sure I followed the routine as always (since it's not my first time installing Geonode that way). I give a feedback.

And again: thanks for your patience.

Cornacap commented 2 years ago

Now I installed everything new and the project_name was taken (geonode4):

docker-compose.yml:


services:

  # Our custom django application. It includes Geonode.
  django:
    << : *default-common-django
    build:
      context: ./
      dockerfile: Dockerfile
    container_name: django4${COMPOSE_PROJECT_NAME}
    healthcheck:
      test: "curl --fail --silent --write-out 'HTTP CODE : %{http_code}\n' --output /dev/null http://127.0.0.1:8001/"
      interval: 60s
      timeout: 10s
      retries: 1
      start_period: 60s
    environment:
      - IS_CELERY=False
    entrypoint: ["/usr/src/geonode4/entrypoint.sh"]
    command: "uwsgi --ini /usr/src/geonode4/uwsgi.ini"

  # Celery worker that executes celery tasks created by Django.
  celery:
    << : *default-common-django
    image: ${COMPOSE_PROJECT_NAME}_django:latest
    container_name: celery4${COMPOSE_PROJECT_NAME}
    depends_on:
      - django
    environment:
      - IS_CELERY=True
    entrypoint: ["/usr/src/geonode4/entrypoint.sh"]
    command: "celery-cmd"

  # Nginx is serving django static and media files and proxies to django and geonode
  geonode:
    image: geonode/nginx:3.x
    build: ./docker/nginx/
    container_name: nginx4${COMPOSE_PROJECT_NAME}
    environment:
      - HTTPS_HOST=${HTTPS_HOST}
      - HTTP_HOST=${HTTP_HOST}
      - HTTPS_PORT=${HTTPS_PORT}
      - HTTP_PORT=${HTTP_PORT}
      - LETSENCRYPT_MODE=${LETSENCRYPT_MODE}
      - RESOLVER=127.0.0.11
    ports:
      - "${HTTP_PORT}:80"
      - "${HTTPS_PORT}:443"
    volumes:
      - nginx-certificates:/geonode-certificates
      - statics:/mnt/volumes/statics
    restart: unless-stopped

Still the container continues to restart:

935d4061e919   geonode/nginx:3.x            "/docker-entrypoint.…"   3 minutes ago   Restarting (1) 9 seconds ago     
nginx4geonode4   | 2022/06/08 12:04:39 [emerg] 1#1: directive "server_name" is not terminated by ";" in /etc/nginx/nginx.https.enabled.conf:11
Cornacap commented 2 years ago

This time I did not try to configure the .env manually. Instead I used the script create-envfile.py. Now the nginx-Problem is gone. But I got a new one: Now django is stuck with restarts.

The log says: 

Can not delete the $HOME/.override_env file as it doesn't exists
Traceback (most recent call last):
  File "/usr/local/bin/invoke", line 8, in <module>
    sys.exit(program.run())
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 384, in run
    self.execute()
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 566, in execute
    executor.execute(*self.tasks)
  File "/usr/local/lib/python3.10/site-packages/invoke/executor.py", line 129, in execute
    result = call.task(*args, **call.kwargs)
  File "/usr/local/lib/python3.10/site-packages/invoke/tasks.py", line 127, in __call__
    result = self.body(*args, **kwargs)
  File "/usr/src/huff_geonode_4/tasks.py", line 121, in update
    current_allowed.extend([str(pub_ip), f'{pub_ip}:{pub_port}'])
AttributeError: 'str' object has no attribute 'extend'
aucampr commented 2 years ago

This time I did not try to configure the .env manually. Instead I used the script create-envfile.py. Now the nginx-Problem is gone. But I got a new one: Now django is stuck with restarts.

The log says:

Can not delete the $HOME/.override_env file as it doesn't exists
Traceback (most recent call last):
  File "/usr/local/bin/invoke", line 8, in <module>
    sys.exit(program.run())
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 384, in run
    self.execute()
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 566, in execute
    executor.execute(*self.tasks)
  File "/usr/local/lib/python3.10/site-packages/invoke/executor.py", line 129, in execute
    result = call.task(*args, **call.kwargs)
  File "/usr/local/lib/python3.10/site-packages/invoke/tasks.py", line 127, in __call__
    result = self.body(*args, **kwargs)
  File "/usr/src/huff_geonode_4/tasks.py", line 121, in update
    current_allowed.extend([str(pub_ip), f'{pub_ip}:{pub_port}'])
AttributeError: 'str' object has no attribute 'extend'

I have this exact same error message. This is the first time that I tried to install.

Cornacap commented 2 years ago

Still hanging on this, working behind a proxy (in an unprotected area it works). But the output of the error has changed: (As a reminder nginx is fine now, but Django restarts)

GeoNode databases are up - executing command
waitfordbs tasks done
DOCKER_ENV=production
**************************migrations*******************************
Operations to perform:
  Apply all migrations: account, actstream, admin, announcements, auth, avatar, base, br, contenttypes, django_celery_results, documents, favorite, geoapps, geonode_client, geonode_mapstore_client, geonode_resource_processing, geonode_themes, groups, guardian, harvesting, invitations, layers, management_commands_http, maps, monitoring, oauth2_provider, people, pinax_notifications, processing, ratings, resource, sequences, services, sessions, sites, socialaccount, taggit, tastypie, upload, user_messages
Running migrations:
  No migrations to apply.
  Your models in app(s): 'ratings', 'upload' have changes that are not yet reflected in a migration, and so won't be applied.
  Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
Unknown command: 'rebuild_index'
Type 'manage.py help' for usage.
migrations tasks done
**********************prepare fixture***************************
Public Hostname or IP is 192.168.211.43
Public PORT is None
prepare tasks done
**************************update geoip*******************************
/usr/local/bin/django-admin.py:17: RemovedInDjango40Warning: django-admin.py is deprecated in favor of django-admin.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f86f011f5b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='build.geo-solutions.it', port=443): Max retries exceeded with url: /geonode/geoserver/latest/GeoLite2-City.tar.gz (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f86f011f5b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/django-admin.py", line 21, in <module>
    management.execute_from_command_line()
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.10/site-packages/geonode/monitoring/management/commands/updategeoip.py", line 61, in handle
    r = requests.get(options['url'], stream=True, timeout=10, verify=False)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='build.geo-solutions.it', port=443): Max retries exceeded with url: /geonode/geoserver/latest/GeoLite2-City.tar.gz (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f86f011f5b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
Restarting periodic command scheduler: cronStopping periodic command scheduler: cron.
Starting periodic command scheduler: cron.
start-stop-daemon: warning: failed to kill 39: No such process
Restarting memcached: memcached.
\n\n\n
Cornacap commented 2 years ago

After a long journey not being able to install Version 4 in our Proxy environment, I had success today.

This was what I had always prepared to make docker work with our Proxy:

/etc/apt/apt.conf.d/proxy.conf
Acquire::http::Proxy "http://xxx.xxx.xxx.xx:8000";
#Acquire::https::Proxy "https://xxx.xxx.xxx.xx:8000";
Acquire::https::Proxy "http://xxx.xxx.xxx.xx:8000";

---------------------------------------------------
/etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
export http_proxy=http://xxx.xxx.xxx.xx:8000
export https_proxy=http://xxx.xxx.xxx.xx:8000
export HTTP_PROXY=http://xxx.xxx.xxx.xx:8000
export HTTPS_PROXY=http://xxx.xxx.xxx.xx:8000

---------------------------------------------------
/etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="http_proxy=http://xxx.xxx.xxx.xx:8000"
Environment="no_proxy=localhost,127.0.0.1"
#Environment=https_proxy=https://xxx.xxx.xxx.xx:8000"
Environment="https_proxy=http://xxx.xxx.xxx.xx:8000"

---------------------------------------------------
/etc/docker/config.json
{
 "proxies":
 {
   "default":
   {
     "httpProxy": "http://xxx.xxx.xxx.xx:8000",
     "httpsProxy": "http://xxx.xxx.xxx.xx:8000",
     "noProxy": "127.0.0.1,localhost"
   }
 }
}

The build has to be like this:

docker-compose -f docker-compose.yml build --build-arg http_proxy=http://xxx.xxx.xxx.xx:8000 --build-arg https_proxy=http://xxx.xxx.xxx.xx:8000

This was the new one of today that made it work: an addition in the .env:

HTTP_PROXY=http://xxx.xxx.xxx.xx:8000
HTTPS_PROXY=http://xxx.xxx.xxx.xx:8000
NO_PROXY=127.0.0.1,localhost,geoserver

Only thing now (what I had before and which vanished after an update (?)): Can't upload shp-Files nor create a dataset. :-(

Cornacap commented 2 years ago

Maybe it would be worth to make a note aboute the handling of proxies in the documentation?

afabiani commented 2 years ago

@Cornacap nice, yes, it would be worth having a section on the documentation.

I guess you will need to enable the http_proxy variable on the GeoServer JAVA_OPTS too probably?

Do you have any stacktrace or log of some sort that allows you to understand the reason you can't upload shp-files?

Cornacap commented 2 years ago

@afabiani this time it was more simple and my mistake. Like last time I forgot to adjust the password for geoserver.