search

geopython / GeoHealthCheck

Service Status and QoS Checker for OGC Web Services
https://geohealthcheck.org
MIT License
84 stars 71 forks source link

Type error regarding timezone on startup of GHC #389

Closed fsteggink closed 2 years ago

fsteggink commented 3 years ago

GHC can't start when freshly built as a Docker container, even when you let GHC initialize a DB. This happens with the current master (#61511405fbeb318caa2133f901b5c43c1266524b)

To reproduce:

version: "3"

services:
  ghc_web:
    image: mytag:latest
    container_name: ghc_web
    restart: unless-stopped
    ports:
      - 127.0.0.1:8083:80
    env_file:
      - ghc.env
    volumes:
      - ghc_sqlitedb:/GeoHealthCheck/DB
      - ghc_settings:/GeoHealthCheck/settings

volumes:
  ghc_sqlitedb:
    driver: local
    driver_opts:
      type: none
      device: "./db"
      o: bind
  ghc_settings:
    driver: local
    driver_opts:
      type: none
      device: "./settings"
      o: bind

For testing purposes I'm running the runner in the web container. This used to work a few weeks ago. The env file doesn't contain anything special. It doesn't matter whether I keep the DB (sqlite) or not.

Output of docker logs ghc_web after a few seconds (sqlite db removed, so it gets initialized):

START /run-web.sh
Container timezone not modified
Container timezone set to: Europe/Amsterdam
START /configure.sh
Using DB_TYPE=sqlite
Creating SQLite DB tables...
/GeoHealthCheck /
---> pavement.create
python GeoHealthCheck/models.py create admintest admintest #REDACTED#
2021-08-17 11:47:22,262 - init - INFO - created GHC App instance #1
Creating database objects
Creating superuser account
/
END /configure.sh
---> pavement.upgrade
Upgrading database...
cd /GeoHealthCheck/GeoHealthCheck
python manage.py db upgrade
2021-08-17 11:47:23,060 - init - INFO - created GHC App instance #1
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 496427d03f87, empty message
INFO  [alembic.runtime.migration] Running upgrade 496427d03f87 -> 992013af402f, empty message
INFO  [alembic.runtime.migration] Running upgrade 992013af402f -> 2638c2a40625, empty message
INFO  [alembic.runtime.migration] Running upgrade 2638c2a40625 -> bb91fb332c36, empty message
INFO  [alembic.runtime.migration] Running upgrade bb91fb332c36 -> 34531bfd7cab, empty message
INFO  [alembic.runtime.migration] Running upgrade 34531bfd7cab -> 90e1c865a561, empty message
INFO  [alembic.runtime.migration] Running upgrade 90e1c865a561 -> f72ff1ac3967, empty message
Before: DB tables: ['alembic_version', 'check_vars', 'probe_vars', 'recipient', 'resource', 'resource_lock', 'resource_tags', 'resourcenotification', 'run', 'tag', 'user']
Tables for tag-support already present, will not create them
Tables for Probe-support already present, will not create them
Column report already present in run table
Column active already present in resource table
Tables for recipient support already present, will not create them
Column run_frequency already present in resource table
Table for Resource-locking already present, will not create
NOT Encrypting for: admintest
Column auth already present in resource table
cd /GeoHealthCheck
Running GHC WSGI on 0.0.0.0:80 with 1 workers and SCRIPT_NAME=
[2021-08-17 11:47:23 +0200] [1] [INFO] Starting gunicorn 20.1.0
[2021-08-17 11:47:23 +0200] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
[2021-08-17 11:47:23 +0200] [1] [INFO] Using worker: eventlet
[2021-08-17 11:47:23 +0200] [18] [INFO] Booting worker with pid: 18
2021-08-17 11:47:24,215 - init - INFO - created GHC App instance #1
2021-08-17 11:47:24,330 - GeoHealthCheck.app - INFO - Running GHC Scheduler in WebApp
2021-08-17 11:47:24,419 - scheduler - INFO - Starting scheduler
2021-08-17 11:47:24,420 - apscheduler.scheduler - INFO - Scheduler started
2021-08-17 11:47:24,420 - scheduler - INFO - lifecycle_listener: EVENT_SCHEDULER_STARTED - <SchedulerEvent (code=1)>
2021-08-17 11:47:24,420 - apscheduler.scheduler - DEBUG - Looking for jobs to run
2021-08-17 11:47:24,420 - apscheduler.scheduler - DEBUG - No jobs; waiting until a job is added
[2021-08-17 11:47:24 +0200] [18] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/venv/lib/python3.7/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/venv/lib/python3.7/site-packages/gunicorn/workers/geventlet.py", line 134, in init_process
    super().init_process()
  File "/venv/lib/python3.7/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/venv/lib/python3.7/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/venv/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/venv/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/venv/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/venv/lib/python3.7/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/GeoHealthCheck/GeoHealthCheck/app.py", line 82, in <module>
    start_schedule()
  File "/GeoHealthCheck/GeoHealthCheck/scheduler.py", line 189, in start_schedule
    scheduler.add_job(flush_runs, 'interval', minutes=150)
  File "/venv/lib/python3.7/site-packages/apscheduler/schedulers/base.py", line 420, in add_job
    'trigger': self._create_trigger(trigger, trigger_args),
  File "/venv/lib/python3.7/site-packages/apscheduler/schedulers/base.py", line 921, in _create_trigger
    return self._create_plugin_instance('trigger', trigger, trigger_args)
  File "/venv/lib/python3.7/site-packages/apscheduler/schedulers/base.py", line 906, in _create_plugin_instance
    return plugin_cls(**constructor_kwargs)
  File "/venv/lib/python3.7/site-packages/apscheduler/triggers/interval.py", line 38, in __init__
    self.timezone = astimezone(timezone)
  File "/venv/lib/python3.7/site-packages/apscheduler/util.py", line 84, in astimezone
    raise TypeError('Only timezones from the pytz library are supported')
TypeError: Only timezones from the pytz library are supported
[2021-08-17 11:47:24 +0200] [18] [INFO] Worker exiting (pid: 18)
2021-08-17 11:47:24,551 - scheduler - INFO - Stopping Scheduler
2021-08-17 11:47:24,551 - apscheduler.scheduler - INFO - Scheduler has been shut down
2021-08-17 11:47:24,551 - scheduler - INFO - lifecycle_listener: EVENT_SCHEDULER_SHUTDOWN - <SchedulerEvent (code=2)>
2021-08-17 11:47:24,551 - apscheduler.scheduler - DEBUG - Looking for jobs to run
2021-08-17 11:47:24,552 - apscheduler.scheduler - DEBUG - No jobs; waiting until a job is added
[2021-08-17 11:47:24 +0200] [1] [INFO] Shutting down: Master
[2021-08-17 11:47:24 +0200] [1] [INFO] Reason: Worker failed to boot.
START /run-web.sh
Container timezone not modified
Container timezone set to: Europe/Amsterdam
START /configure.sh
Using DB_TYPE=sqlite
NOT creating SQLite DB tables...
END /configure.sh
---> pavement.upgrade
Upgrading database...
cd /GeoHealthCheck/GeoHealthCheck
python manage.py db upgrade
2021-08-17 11:47:25,797 - init - INFO - created GHC App instance #1
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
cd /GeoHealthCheck
Running GHC WSGI on 0.0.0.0:80 with 1 workers and SCRIPT_NAME=
[2021-08-17 11:47:26 +0200] [1] [INFO] Starting gunicorn 20.1.0
# SEQUENCE REPEATS #

GHC version: current master (see above) Docker version:

d:\Temp\ghc>docker --version
Docker version 20.10.7, build f0df350

d:\Temp\ghc>docker-compose --version
docker-compose version 1.29.2, build 5becea4c
fsteggink commented 3 years ago

This issue seems to be related: https://github.com/Yelp/elastalert/issues/2968

fsteggink commented 3 years ago

The suggested fix (namely to add tzlocal<3.0 as a requirement) works.

fsteggink commented 3 years ago

The package tzlocal version 3.0 has been released four days ago (Aug. 13th): https://pypi.org/project/tzlocal/. I'll submit a PR for the quick fix, but only for Docker. I don't know if this is needed when GHC is run locally, since I'm only able to run the Docker version on Windows. Perhaps it isn't even needed. It is possible that the Docker time zone is used, and deemed to be incompatible. But for the long term a fix should be made so that tzlocal 3+ works as well.

justb4 commented 3 years ago

Ok, yes pytz is not required currently for non-Docker GHC (unless one runs with gunicorn/eventlet). So it comes in as a dependency there, which is always tricky, so often I do pip freeze and have those deps as requirements.txt, so builds are repeatable without surprises. But the fix in #390 looks ok for now.

justb4 commented 3 years ago

Ok #390 fix works, pip freeze gives:

alabaster==0.7.12
alembic==1.6.5
APScheduler==3.6.1
attrs==21.2.0
Babel==2.9.1
certifi==2021.5.30
charset-normalizer==2.0.4
click==8.0.1
dnspython==1.16.0
docutils==0.16
eventlet==0.30.2
Flask==1.1.1
Flask-Babel==0.12.2
Flask-Login==0.4.1
Flask-Migrate==2.5.2
Flask-Script==2.0.6
Flask-SQLAlchemy==2.4.0
greenlet==1.1.1
gunicorn==20.1.0
idna==3.2
imagesize==1.2.0
importlib-metadata==4.7.1
itsdangerous==1.1.0
Jinja2==3.0.1
jsonschema==3.0.2
lxml==4.6.3
Mako==1.1.5
MarkupSafe==2.0.1
openapi-spec-validator==0.2.8
OWSLib==0.20.0
packaging==21.0
passlib==1.7.1
Paver==1.3.4
psycopg2==2.7.5
Pygments==2.10.0
pyparsing==2.4.7
pyproj==2.6.1.post1
pyrsistent==0.18.0
python-dateutil==2.8.2
python-editor==1.0.4
pytz==2021.1
PyYAML==5.4.1
requests==2.26.0
six==1.16.0
snowballstemmer==2.1.0
Sphinx==4.1.1
sphinx-rtd-theme==0.5.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
SQLAlchemy==1.3.8
typing-extensions==3.10.0.0
tzlocal==2.1
urllib3==1.26.6
Werkzeug==0.16.1
WTForms==2.2.1
zipp==3.5.0

(nb one needs to activate the /venv/bin/activate when bashing into the Docker GHC (web) Container)

We need to revisit/freeze all deps: I propose to close this issue and open a new one for "Freeze dependencies". Ok?

justb4 commented 2 years ago

@fsteggink think we should add tzlocal<3.0 to requirements.txt for regular installs. It is used by APScheduler, see also this issue. .... I see that fix is also included in recent PR #403 ....So will review/merge that PR first...

fsteggink commented 2 years ago

Fine with me. I'm not able to test it locally.

justb4 commented 2 years ago

With #403 now merged, local install/run works (tzlocal is 2.1) , let's close this.