Debian packages for GeoNode and OSGeoLive

gannebamm commented 3 years ago

make Debian packages great again

kalxas commented 3 years ago

Debian packaging spreadsheet: https://docs.google.com/spreadsheets/d/1P5iU6UGraR1vlhx1idLv3wQDQ-SE3w2sXCw9f55ZhFE/edit?usp=sharing

kalxas commented 3 years ago

https://twitter.com/launchpadstatus/status/1336103011454771202

kalxas commented 3 years ago

Progress can be seen here:

https://launchpad.net/~gcpp-kalxas/+archive/ubuntu/geonode/+packages?field.name_filter=&field.status_filter=published&field.series_filter=focal

kalxas commented 3 years ago

All dependencies are now packaged as well:

https://launchpad.net/%7Egcpp-kalxas/+archive/ubuntu/geonode/+packages?field.name_filter=&field.status_filter=published&field.series_filter=focal

I will test and then move to the geonode ppa.

kalxas commented 3 years ago

@afabiani @gannebamm I just did a first test to install and got some errors:

django-jsonfield 1.1.0 conflicts with jsonfield 3.1.0 dpkg: error processing archive /tmp/apt-dpkg-install-p2gZhM/125-python3-jsonfield_3.1.0-0~focal0_all.deb (--unpack): trying to overwrite '/usr/lib/python3/dist-packages/jsonfield/__init__.py', which is also in package python3-django-jsonfield 1.1.0-2
pinax-ratings conflicts with pinax-notifications dpkg: error processing archive /tmp/apt-dpkg-install-p2gZhM/128-python3-pinax-ratings_4.0.0-0~focal0_all.deb (--unpack): trying to overwrite '/usr/lib/python3/dist-packages/pinax/__init__.py', which is also in package python3-pinax-notifications 5.0.3-0~focal0

Any ideas?

kalxas commented 3 years ago

I have remove jsonfield 3.1.0 from dependency list until further notice. Installing both packages with pip results in one in dist-packages (the last one installed)

afabiani commented 3 years ago

I guess we can safely remove jsonfield

No sure for pinax ratings, I need to double check.

kalxas commented 3 years ago

the problem with pinax-ratings and pinax-notifications is that they both include an identical __init__.py file and they both place it under the dist-packages/pinax folder...

kalxas commented 3 years ago

I have removed the file from ratings package, will test again

kalxas commented 3 years ago

Current status from OSGeoLive build logs:

Unpacking python3-geonode (3.1.0-0~focal5) ...
Setting up python3-elasticsearch (7.1.0-2) ...
Setting up python3-glob2 (0.5-3) ...
Setting up python3-openid (3.1.0-1) ...
Setting up python3-mercantile (1.1.6-1~focal1) ...
Setting up python3-django-uuid-upload-path (1.0.0-0~focal0) ...
Setting up python3-timeout-decorator (0.4.1-0~focal0) ...
Setting up python3-billiard (3.6.1.0-1) ...
Setting up python3-requests-toolbelt (0.8.0-1.1) ...
Setting up python3-geoserver-restconfig (2.0.4.8-0~focal0) ...
Setting up python3-geonode-avatar (5.0.3-1~focal2) ...
Setting up python3-py (1.8.1-1) ...
Setting up python3-jdcal (1.0-1.3) ...
Setting up python3-django-storages (1.9.1-1) ...
Setting up python3-coreschema (0.0.4-3) ...
Setting up python3-django-autocomplete-light (3.5.1-1~focal0) ...
Setting up python3-django-polymorphic (2.1.2-2) ...
Setting up python3-zope.interface (4.7.1-1) ...
Setting up python3-pbr (5.4.5-0ubuntu1) ...
update-alternatives: using /usr/bin/python3-pbr to provide /usr/bin/pbr (pbr) in auto mode
Setting up python3-geonode-oauth-toolkit (2.0.0-1~focal0) ...
/usr/lib/python3/dist-packages/oauth2_provider/oauth2_validators.py:749: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if (request.grant_type is "authorization_code" and "access_token" in token) or request.response_type == "code id_token token" or (request.response_type == "id_token token" and "access_token" in token):
Setting up python3-openssl (19.0.0-1build1) ...
Setting up python3-ephem (3.7.7.1-1) ...
Setting up python3-automat (0.8.0-1ubuntu1) ...
Setting up python3-pyflakes (2.1.1-2) ...
Setting up python3-twisted-bin:amd64 (18.9.0-11) ...
Setting up python3-memcache (1.59-4) ...
Setting up python3-geonode-announcements (2.0.2-1~focal0) ...
Setting up python3-django-appconf (1.0.3-1) ...
Setting up python3-coverage (4.5.2+dfsg.1-4ubuntu1) ...
Setting up python3-pilkit (2.0-3) ...
Setting up python3-hamcrest (1.9.0-3) ...
Setting up fonts-glewlwyd (1.4.9-3) ...
Setting up python3-django-forms-bootstrap (3.1.0-1~focal0) ...
Setting up python3-django-downloadview (2.1.1-1) ...
Setting up python3-inflection (0.3.1-2) ...
Setting up python3-wheel (0.34.2-1) ...
Setting up python3-dj-database-url (0.5.0-2) ...
Setting up python3-psutil (5.5.1-1ubuntu4) ...
Setting up python3-django-multi-email-field (0.5.1-1~focal0) ...
Setting up python3-django-floppyforms (1.9.0-1~focal0) ...
Setting up python3-vine (1.3.0+dfsg-1) ...
Setting up python3-django-taggit (0.24.0-2) ...
Setting up python3-django-modeltranslation (0.13.3-0.1) ...
Setting up python3-django-ipware (2.1.0-1) ...
Setting up python3-djangorestframework-writable-nested (0.6.2-0~focal0) ...
Setting up python3-atomicwrites (1.1.5-2build1) ...
Setting up python3-uritemplate (0.6-4) ...
Setting up python3-fake-factory (0.7.7-3) ...
Setting up python3-mock (3.0.5-1build1) ...
Setting up python3-djangorestframework (3.10.2-1) ...
Setting up python3-itypes (1.1.0-4) ...
Setting up python3-django-tinymce (3.1.0-0~focal0) ...
Setting up python3-pycodestyle (2.5.0-2) ...
Setting up python3-packaging (20.3-1) ...
Setting up python3-flaky (3.6.1-1build1) ...
Setting up python3-django-recaptcha (2.0.6-0~focal0) ...
Setting up python3-geonode-django-activity-stream (0.9.0-0~focal0) ...
Setting up python3-incremental (16.10.1-3.2) ...
Setting up python3-django-js-asset (1.2.2-2) ...
Setting up python3-pycountry (19.8.18+ds1-2) ...
Setting up python3-hyperlink (19.0.0-1) ...
Setting up python3-service-identity (18.1.0-5build1) ...
Setting up python3-django-leaflet (0.27.1-0~focal0) ...
Setting up python3-django-imagekit (4.0.2-3) ...
Setting up python3-django-invitations (1.9.3-0~focal0) ...
Setting up python3-markdown (3.1.1-3) ...
Setting up python3-django-jsonfield (1.1.0-2) ...
Setting up python3-resize-image (1.1.19-0~focal0) ...
Setting up python3-dj-pagination (2.4.0-0~focal0) ...
Setting up python3-pluggy (0.13.0-2) ...
Setting up python3-ruamel.yaml (0.15.89-3build1) ...
Setting up python3-django-geonode-client (1.0.9-1~focal0) ...
Setting up python3-pinax-ratings (4.0.0-0~focal1) ...
Setting up python3-django-treebeard (4.3.1+dfsg-1) ...
Setting up python3-geonode-user-messages (2.0.1-1~focal1) ...
Setting up python3-parse (1.6.6-0.2build1) ...
Setting up python3-amqp (2.5.2-0ubuntu1) ...
Setting up python3-gn-gsimporter (2.0.1-0~focal0) ...
Setting up python3-django-grappelli (2.14.2-0~focal0) ...
Setting up python3-factory-boy (2.11.1-3) ...
Setting up python3-mccabe (0.6.1-3) ...
Setting up python3-dynamic-rest (2.0.0-0~focal0) ...
Setting up python3-constantly (15.1.0-1build1) ...
Setting up python-pip-whl (20.0.2-5ubuntu1.1) ...
Setting up python3-anyjson (0.3.3-2) ...
Setting up uap-core (20190213-2) ...
Setting up python3-django-guardian (2.0.0-2) ...
Setting up python3-django-jsonfield-compat (0.4.4+geonode-2~focal1) ...
Setting up python3-maxminddb (1.5.2-0ubuntu1) ...
Setting up python3-invoke (1.3.0+ds-0.1) ...
Setting up python3-soupsieve (1.9.5+dfsg-1) ...
Setting up python3-unidecode (1.1.1-3build1) ...
Setting up python3-mimeparse (1.6.0-3build1) ...
Setting up python3-websocket (0.53.0-2ubuntu1) ...
update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode
Setting up python3-django-haystack (2.8.1-3) ...
/usr/lib/python3/dist-packages/haystack/backends/whoosh_backend.py:614: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if value is None or len(value) is 0:
Setting up python3-xmljson (0.1.9-1~focal0) ...
Setting up python3-requests-oauthlib (1.0.0-1.1build1) ...
Setting up python3-mako (1.1.0+ds1-1ubuntu2) ...
Setting up python3-django-bootstrap3-datetimepicker-2 (2.8.2-0~focal0) ...
Setting up python3-geonode-dialogos (3.0.1-1~focal1) ...
Setting up python3-django-filters (2.1.0-1) ...
Setting up python3-django-cors-headers (2.2.0-2) ...
Setting up python3-arcrest (10.5.3-1~focal0) ...
Setting up python3-pinax-notifications (5.0.3-0~focal0) ...
Setting up python3-django-mapstore-adapter (2.0.6-1~focal1) ...
Setting up python3-django-mptt (0.10.0-1) ...
Setting up python3-geoip2 (2.9.0+dfsg1-2) ...
Setting up python3-djangorestframework-gis (0.14-3) ...
Setting up python3-django-tastypie (0.13.3-1.1) ...
Setting up python3-pytest (4.6.9-1) ...
Setting up python3-docker (4.1.0-1) ...
Setting up python3-coreapi (2.3.3-4) ...
Setting up python3-pytest-django (3.5.1-1) ...
Setting up python3-kombu (4.6.7-1ubuntu3) ...
Setting up python3-ua-parser (0.9.0-1) ...
Setting up python3-bs4 (4.8.2-1) ...
Setting up python3-slugify (4.0.0-1) ...
Setting up python3-flake8 (3.7.9-2) ...
Setting up python3-djangorestframework-extensions (0.6.0-1) ...
Setting up python3-django-markdownify (0.8.1-0~focal0) ...
Setting up python3-parse-type (0.3.4-3) ...
Setting up python3-twisted (18.9.0-11) ...
Setting up python3-django-allauth (0.41.0+ds-1) ...
Setting up python3-pip (20.0.2-5ubuntu1.1) ...
Setting up python3-user-agents (2.0-3) ...
Setting up python3-django-geonode-mapstore-client (2.0.9-0~focal0) ...
Setting up python3-djangorestframework-guardian (0.3.0-0~focal0) ...
Setting up python3-djangorestframework-yasg (1.20.0-0~focal0) ...
Setting up python3-pytest-bdd (3.2.1-1) ...
Setting up python3-celery (4.2.1-5ubuntu1) ...
Setting up python3-geonode (3.1.0-0~focal5) ...
Processing triggers for man-db (2.9.1-1) ...
127.0.0.1 geonode
Deploying geonode demonstration instance
Done
create geonode_app database with PostGIS
CREATE EXTENSION
Done
create geonode_data database with PostGIS
CREATE EXTENSION
Done
Copying settings files
Done
Configuring GeoNode
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 118, in create
    cls = getattr(mod, cls_name)
AttributeError: module 'pinax' has no attribute 'ratings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 136, in create
    import_module(entry)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pinax.ratings'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 118, in create
    cls = getattr(mod, cls_name)
AttributeError: module 'pinax' has no attribute 'ratings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 136, in create
    import_module(entry)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pinax.ratings'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 118, in create
    cls = getattr(mod, cls_name)
AttributeError: module 'pinax' has no attribute 'ratings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 136, in create
    import_module(entry)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pinax.ratings'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 118, in create
    cls = getattr(mod, cls_name)
AttributeError: module 'pinax' has no attribute 'ratings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 136, in create
    import_module(entry)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pinax.ratings'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 118, in create
    cls = getattr(mod, cls_name)
AttributeError: module 'pinax' has no attribute 'ratings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 136, in create
    import_module(entry)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pinax.ratings'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 118, in create
    cls = getattr(mod, cls_name)
AttributeError: module 'pinax' has no attribute 'ratings'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/lib/python3/dist-packages/django/apps/config.py", line 136, in create
    import_module(entry)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pinax.ratings'

kalxas commented 3 years ago

Pinax ratings packaging now fixed and does not conflict with pinax notifications. I also fixed the django-jsonfield 1.x conflict with jsonfield 3.x: both packages install similar files at the same location, I think we should review this dependency in our requirements.txt

kalxas commented 3 years ago

Great progress, migrations now work in the build

Deploying geonode demonstration instance
Done
create geonode_app database with PostGIS
CREATE EXTENSION
Done
create geonode_data database with PostGIS
CREATE EXTENSION
Done
Copying settings files
Done
Configuring GeoNode
Migrations for 'base':
  /usr/lib/python3/dist-packages/geonode/base/migrations/0047_auto_20210107_0943.py
    - Alter field name on hierarchicalkeyword
    - Alter field slug on hierarchicalkeyword
Migrations for 'ratings':
  /usr/lib/python3/dist-packages/pinax/ratings/migrations/0004_auto_20210107_0943.py
    - Alter field category on overallrating
    - Alter field category on rating
Operations to perform:
  Apply all migrations: account, actstream, admin, announcements, auth, avatar, base, br, contenttypes, dialogos, documents, favorite, geonode_client, geonode_themes, groups, guardian, invitations, layers, maps, mapstore2_adapter, monitoring, oauth2_provider, people, pinax_notifications, ratings, services, sessions, sites, socialaccount, taggit, tastypie, upload, user_messages
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying taggit.0001_initial... OK
  Applying taggit.0002_auto_20150616_2121... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0001_initial... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying people.24_initial... OK
  Applying account.0001_initial... OK
  Applying account.0002_email_max_length... OK
  Applying actstream.0001_initial... OK
  Applying actstream.0002_remove_action_data... OK
  Applying actstream.0003_add_follow_flag... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying announcements.0001_initial... OK
  Applying announcements.0002_auto_20200311_0959... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying avatar.0001_initial... OK
  Applying avatar.0002_auto_20200210_1421... OK
  Applying base.24_initial... OK
  Applying layers.24_initial... OK
  Applying services.24_initial... OK
  Applying services.0025_harvestjob... OK
  Applying services.0026_auto_20171130_0600_squashed_0041_auto_20190404_0820... OK
  Applying services.0027_auto_20190429_0831... OK
  Applying services.0028_remove_service_last_updated... OK
  Applying services.0029_remove_service_created... OK
  Applying base.24_to_26... OK
  Applying base.26_to_27... OK
  Applying base.0027_auto_20170801_1228_squashed_0037_auto_20190222_1347... OK
  Applying base.0028_curatedthumbnail... OK
  Applying base.0029_resourcebase_last_updated... OK
  Applying base.0030_resourcebase_created... OK
  Applying base.0031_auto_20200114_1651... OK
  Applying base.0032_auto_20200115_1121... OK
  Applying base.0033_auto_20200115_1145... OK
  Applying base.0034_auto_20200212_0410... OK
  Applying base.0035_auto_20200212_0428... OK
  Applying base.0036_resourcebase_doi... OK
  Applying base.0037_auto_20200305_1520... OK
  Applying base.0038_configuration... OK
  Applying base.0039_auto_20200321_1338... OK
  Applying base.0038_delete_backup... OK
  Applying base.0040_merge_20200321_2245... OK
  Applying groups.24_initial... OK
  Applying groups.26_to_27... OK
  Applying groups.0027_auto_20180105_1631_squashed_0028_auto_20180606_1543... OK
  Applying groups.0028_auto_20200114_1651... OK
  Applying groups.0029_auto_20200115_1121... OK
  Applying groups.0030_auto_20200221_1107... OK
  Applying groups.0031_auto_20200305_1522... OK
  Applying groups.0031_auto_20200304_1434... OK
  Applying groups.0032_merge_20200306_1153... OK
  Applying base.0038_auto_20200318_0953... OK
  Applying base.0041_merge_20200323_1119... OK
  Applying base.0042_auto_20200414_1554... OK
  Applying base.0043_auto_20200527_0833... OK
  Applying base.0044_auto_20201001_1514... OK
  Applying base.0045_resourcebase_attribution... OK
  Applying base.0046_auto_20201116_0929... OK
  Applying base.0047_auto_20210107_0943... OK
  Applying br.0001_initial... OK
  Applying dialogos.0001_initial... OK
  Applying documents.24_initial... OK
  Applying documents.25_add_documentresourcelink_table... OK
  Applying documents.26_move_data_to_documentresourcelink_table... OK
  Applying documents.27_drop_resource_columns_from_document_table... OK
  Applying documents.0028_auto_20170801_1228_squashed_0035_auto_20190404_0820... OK
  Applying documents.0029_auto_20190429_0831... OK
  Applying documents.0030_auto_20200305_1520... OK
  Applying documents.0031_auto_20201107_2241... OK
  Applying favorite.0001_initial... OK
  Applying geonode_client.0001_initial... OK
  Applying geonode_client.0002_auto_20180412_1039_squashed_0005_auto_20181015_1201... OK
  Applying geonode_themes.0001_initial... OK
  Applying geonode_themes.0002_auto_20181015_1208_squashed_0003_remove_geonodethemecustomization_identifier... OK
  Applying geonode_themes.0003_auto_20190503_1742... OK
  Applying geonode_themes.0004_auto_20190503_1817... OK
  Applying geonode_themes.0005_auto_20190510_1335... OK
  Applying geonode_themes.0006_geonodethemecustomization_body_text_color... OK
  Applying geonode_themes.0007_auto_20190510_1545... OK
  Applying geonode_themes.0008_auto_20191122_1714... OK
  Applying geonode_themes.0009_geonodethemecustomization_footer_bg_color... OK
  Applying geonode_themes.0010_auto_20200115_1121... OK
  Applying geonode_themes.0011_auto_20200727_1451... OK
  Applying groups.0033_auto_20200321_1349... OK
  Applying groups.0034_auto_20200512_1431... OK
  Applying guardian.0001_initial... OK
  Applying invitations.0001_initial... OK
  Applying invitations.0002_auto_20151126_0426... OK
  Applying invitations.0003_auto_20151126_1523... OK
  Applying layers.24_to_26... OK
  Applying layers.26_to_27... OK
  Applying layers.0027_auto_20170801_1228_squashed_0033_auto_20180606_1543... OK
  Applying layers.0028_auto_20200610_0759... OK
  Applying layers.0029_auto_20200610_0803... OK
  Applying layers.0030_auto_20200610_0825... OK
  Applying layers.0031_auto_20200612_1533... OK
  Applying layers.0032_auto_20200612_1544... OK
  Applying maps.24_initial... OK
  Applying maps.0025_auto_20170801_1228_squashed_0032_auto_20190404_0820... OK
  Applying maps.0026_auto_20190429_0831... OK
  Applying maps.0027_auto_20190516_0909... OK
  Applying maps.0028_maplayer_store... OK
  Applying maps.0029_delete_mapsnapshot... OK
  Applying mapstore2_adapter.0001_initial... OK
  Applying mapstore2_adapter.0002_auto_20190618_1236... OK
  Applying mapstore2_adapter.0003_auto_20200310_0848... OK
  Applying monitoring.0001_monitoring_init... OK
  Applying monitoring.0002_monitoring_update... OK
  Applying monitoring.0003_monitoring_resources... OK
  Applying monitoring.0004_monitoring_metric_type... OK
  Applying monitoring.0005_monitoring_ows_service... OK
  Applying monitoring.0006_monitoring_path_text... OK
  Applying monitoring.0007_monitoring_exception_message... OK
  Applying monitoring.0008_monitoring_notifications_check... OK
  Applying monitoring.0009_sample_count... OK
  Applying monitoring.0010_metric_unit... OK
  Applying monitoring.0011_notification_def... OK
  Applying monitoring.0012_notifications_service... OK
  Applying monitoring.0013_notifications_def_description... OK
  Applying monitoring.0014_notifications_emails... OK
  Applying monitoring.0015_notification_grace_period... OK
  Applying monitoring.0016_notification_severity... OK
  Applying monitoring.0017_monitoring_notification_active... OK
  Applying monitoring.0018_notification_check_def... OK
  Applying monitoring.0019_notification_check_def_link... OK
  Applying monitoring.0020_metric_description... OK
  Applying monitoring.0021_auto_20180301_0932... OK
  Applying monitoring.0022_service_last_check_default... OK
  Applying monitoring.0023_auto_20190528_0906... OK
  Applying monitoring.0024_auto_20190605_1619... OK
  Applying monitoring.0025_auto_20190813_0808... OK
  Applying monitoring.0026_auto_20190821_0736... OK
  Applying monitoring.0027_monitoredresource_resource_id... OK
  Applying monitoring.0028_auto_20190830_1018... OK
  Applying monitoring.0029_auto_20200115_1121... OK
  Applying monitoring.0030_auto_20200305_1520... OK
  Applying monitoring.0031_auto_20201012_0931... OK
  Applying oauth2_provider.0001_initial... OK
  Applying oauth2_provider.0002_08_updates... OK
  Applying oauth2_provider.0003_auto_20160316_1503... OK
/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField Avatar.date_uploaded received a naive datetime (2021-01-07 09:43:21.080594) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"
  Applying oauth2_provider.0004_auto_20160525_1623... OK
  Applying oauth2_provider.0005_auto_20170514_1141... OK
  Applying oauth2_provider.0006_auto_20170903_1632... OK
  Applying oauth2_provider.0007_application_algorithm... OK
  Applying oauth2_provider.0008_idtoken... OK
  Applying oauth2_provider.0006_auto_20171214_2232... OK
  Applying oauth2_provider.0009_merge_20180606_1314... OK
  Applying oauth2_provider.0010_auto_20190406_1805... OK
  Applying oauth2_provider.0011_auto_20190413_2007... OK
  Applying oauth2_provider.0012_accesstoken_id_token... OK
  Applying oauth2_provider.0013_auto_20190816_1714... OK
  Applying people.0025_auto_20170924_0932_squashed_0028_auto_20180606_1543... OK
  Applying people.0026_auto_20200109_1600... OK
  Applying people.0027_auto_20200114_1651... OK
  Applying people.0028_auto_20200115_1121... OK
  Applying pinax_notifications.0001_initial... OK
  Applying pinax_notifications.0002_auto_20171003_2006... OK
  Applying ratings.0001_initial... OK
  Applying ratings.0002_auto_20170819_1121... OK
  Applying ratings.0003_auto_20170819_1127... OK
  Applying ratings.0004_auto_20210107_0943... OK
  Applying services.0030_auto_20200115_1121... OK
  Applying services.0031_service_probe... OK
  Applying sessions.0001_initial... OK
  Applying sites.0001_initial... OK
  Applying sites.0002_alter_domain_unique... OK
  Applying socialaccount.0001_initial... OK
  Applying socialaccount.0002_token_max_lengths... OK
  Applying socialaccount.0003_extra_data_default_dict... OK
  Applying tastypie.0001_initial... OK
  Applying tastypie.0002_api_access_url_length... OK
  Applying upload.24_initial... OK
  Applying upload.0025_auto_20200115_1836... OK
  Applying upload.0026_auto_20200321_1349... OK
  Applying user_messages.0001_initial... OK
  Applying user_messages.0002_auto_20171107_1128... OK
  Applying user_messages.0003_auto_20171108_1037... OK
  Applying user_messages.0004_auto_20171108_1101... OK
/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField Profile.last_login received a naive datetime (2011-06-09 15:45:34) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"
/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField Profile.date_joined received a naive datetime (2011-06-09 15:15:27) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"
Installed 1 object(s) from 1 fixture(s)
Installed 1 object(s) from 1 fixture(s)
Installed 301 object(s) from 1 fixture(s)
Done

Still get 500 error in Apache in the iso. Can someone please download and have a look? http://aiolos.survey.ntua.gr/gisvm/dev/osgeolive-nightly-build207-amd64-6279979-master.iso

kalxas commented 3 years ago

basically we need to review this file:

https://github.com/OSGeo/OSGeoLive/blob/master/app-conf/geonode/local_settings.py.sample

afabiani commented 3 years ago

@kalxas cool, will try to have a look later today

kalxas commented 3 years ago

The build logs look clean, except this:

Inspecting the available layers in GeoServer ...
Found 19 layers, starting processing
[failed] Layer giant_polygon (1/19)
[failed] Layer poi (2/19)
[failed] Layer poly_landmarks (3/19)
[failed] Layer tiger_roads (4/19)
[failed] Layer Arc_Sample (5/19)
[failed] Layer Pk50095 (6/19)
[failed] Layer mosaic (7/19)
[failed] Layer Img_Sample (8/19)
[failed] Layer states (9/19)
[failed] Layer tasmania_cities (10/19)
[failed] Layer tasmania_roads (11/19)
[failed] Layer tasmania_state_boundaries (12/19)
[failed] Layer tasmania_water_bodies (13/19)
[failed] Layer archsites (14/19)
[failed] Layer bugsites (15/19)
[failed] Layer restricted (16/19)
[failed] Layer roads (17/19)
[failed] Layer streams (18/19)
[failed] Layer sfdem (19/19)

kalxas commented 3 years ago

Once the setup is confirmed to work, I will push the packages to the GeoNode ppa

afabiani commented 3 years ago

@kalxas I did a test with the iso you provided. Actually those are the issues I have found:

Apache2 won't startup because the folder /var/log/apache2 is missing
In order for GeoNode to startup correctly, we need to either: a. Put into local_settings.py the option MONITORING_ENABLED = False b. Delete from /usr/lib/python3/dist-packages/geonode/monitoring/models.py the line 49: import user_agents
With the changes (1.) and (2.) GeoNode starts and we can login as admin / admin, nevertheless it won't be possible to upload layers into GeoServer because the 2.18.0 version installed on the iso is missing the plugins and data_dir configurations needed by GeoNode

kalxas commented 3 years ago

@afabiani many thanks for the feedback, I am going to apply the above fixes and wait for the GeoServer 2.18.0 package.

kalxas commented 3 years ago

https://github.com/OSGeo/OSGeoLive/commit/5b8ecad29016456d04aae1d4c5f6be3395b4b461

kalxas commented 3 years ago

Debian packages moved to https://launchpad.net/~geonode/+archive/ubuntu/osgeolive/+packages?field.name_filter=&field.status_filter=published&field.series_filter=focal

afabiani commented 3 years ago

@kalxas please find the GeoServer bin here

kalxas commented 3 years ago

Thanks @afabiani Wow, 190MB larger than the vanilla zip? Is there any way to make it smaller?

afabiani commented 3 years ago

Uhm I'll check but I don't think so. I guess most of the space is taken by the JARS plugins inside WEB-INF/lib

kalxas commented 3 years ago

Ouch, we currently have ~10MB left in the disk, this will break the build

kalxas commented 3 years ago

@afabiani this is the install log for the new package of geoserver:

2021-01-23 13:34:17 (1.35 MB/s) - 'geoserver-2.18.1-bin.zip' saved [306363983/306363983]

Unpacking GeoServer in /usr/local/lib/geoserver-2.18.1
Configuring GeoServer script
sed: can't read /usr/local/lib/geoserver-2.18.1/bin/startup.sh: No such file or directory
sed: can't read /usr/local/lib/geoserver-2.18.1/bin/shutdown.sh: No such file or directory
sed: can't read /usr/local/lib/geoserver-2.18.1/start.ini: No such file or directory
./install_geoserver.sh: 96: cannot create /usr/local/lib/geoserver-2.18.1/bin/start_admin.sh: Directory nonexistent
./install_geoserver.sh: 121: cannot create /usr/local/lib/geoserver-2.18.1/bin/stop_notify.sh: Directory nonexistent
chmod: cannot access '/usr/local/lib/geoserver-2.18.1/bin/startup.sh': No such file or directory
chmod: cannot access '/usr/local/lib/geoserver-2.18.1/bin/start_admin.sh': No such file or directory
chmod: cannot access '/usr/local/lib/geoserver-2.18.1/bin/shutdown.sh': No such file or directory
chmod: cannot access '/usr/local/lib/geoserver-2.18.1/bin/stop_notify.sh': No such file or directory
The user `user' is already a member of `users'.
chgrp: cannot access '/usr/local/lib/geoserver-2.18.1/data_dir': No such file or directory
chgrp: cannot access '/usr/local/lib/geoserver-2.18.1/logs': No such file or directory
chmod: cannot access '/usr/local/lib/geoserver-2.18.1/data_dir': No such file or directory
chmod: cannot access '/usr/local/lib/geoserver-2.18.1/logs': No such file or directory

I guess there is some folder structure change compared to upstream 2.18.1?

kalxas commented 3 years ago

I also see that this jar is 156 MB gs-web-app-2.18-SNAPSHOT.jar

kalxas commented 3 years ago

ok, folder structure fixed and zip uploaded again

kalxas commented 3 years ago

I have removed the huge jar, now testing if geonode still works in the iso

kalxas commented 3 years ago

Fixed

afabiani commented 3 years ago

@kalxas cool, please let me know if I can help you testing or fixing something!

Thanks for your effort

kalxas commented 2 years ago

Hi @afabiani @gannebamm

I am doing the packaging for OSGeoLive 15.0 and I get the following error on setup:

Configuring GeoNode
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'GeoNode Project Templates'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 121, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/markdownify/templatetags/markdownify.py", line 5, in <module>
    from bleach.css_sanitizer import ALLOWED_CSS_PROPERTIES
ModuleNotFoundError: No module named 'bleach.css_sanitizer'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 361, in execute
    self.check()
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 387, in check
    all_issues = self._run_checks(
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 377, in _run_checks
    return checks.run_checks(**kwargs)
  File "/usr/lib/python3/dist-packages/django/core/checks/registry.py", line 72, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/usr/lib/python3/dist-packages/django/contrib/admin/checks.py", line 79, in check_dependencies
    for engine in engines.all():
  File "/usr/lib/python3/dist-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/usr/lib/python3/dist-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/usr/lib/python3/dist-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 25, in __init__
    options['libraries'] = self.get_templatetag_libraries(libraries)
  File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 108, in get_installed_libraries
    for name in get_package_libraries(pkg):
  File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 123, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'markdownify.templatetags.markdownify': No module named 'bleach.css_sanitizer'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'GeoNode Project Templates'

Any ideas?

afabiani commented 2 years ago

@kalxas see https://github.com/GeoNode/geonode/issues/9263

Fix: pip install django-markdownify==0.9.1

kalxas commented 2 years ago

ouch, I did not expect 0.9.2 to cause this :)

kalxas commented 2 years ago

@afabiani I am now getting this:

Configuring GeoNode
Migrations for 'ratings':
  /usr/lib/python3/dist-packages/pinax/ratings/migrations/0004_auto_20220713_1025.py
    - Alter field category on overallrating
    - Alter field category on rating
/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField Avatar.date_uploaded received a naive datetime (2022-07-13 10:25:11.074608) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"
Operations to perform:
  Apply all migrations: account, actstream, admin, announcements, auth, avatar, base, br, contenttypes, dialogos, django_celery_beat, django_celery_results, documents, favorite, geoapp_dashboards, geoapp_geostories, geoapps, geonode_client, geonode_themes, groups, guardian, invitations, layers, management_commands_http, maps, mapstore2_adapter, monitoring, oauth2_provider, people, pinax_notifications, ratings, services, sessions, sites, socialaccount, taggit, tastypie, upload, user_messages
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying taggit.0001_initial... OK
  Applying taggit.0002_auto_20150616_2121... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0001_initial... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying people.24_initial... OK
  Applying account.0001_initial... OK
  Applying account.0002_email_max_length... OK
  Applying actstream.0001_initial... OK
  Applying actstream.0002_remove_action_data... OK
  Applying actstream.0003_add_follow_flag... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying announcements.0001_initial... OK
  Applying announcements.0002_auto_20200311_0959... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying avatar.0001_initial... OK
  Applying avatar.0002_auto_20200210_1421... OK
  Applying base.24_initial... OK
  Applying layers.24_initial... OK
  Applying services.24_initial... OK
  Applying services.0025_harvestjob... OK
  Applying services.0026_auto_20171130_0600_squashed_0041_auto_20190404_0820... OK
  Applying layers.24_to_26... OK
  Applying layers.26_to_27... OK
  Applying base.24_to_26... OK
  Applying base.26_to_27... OK
  Applying base.0027_auto_20170801_1228_squashed_0037_auto_20190222_1347... OK
  Applying layers.0027_auto_20170801_1228_squashed_0033_auto_20180606_1543... OK
  Applying services.0027_auto_20190429_0831... OK
  Applying services.0028_remove_service_last_updated... OK
  Applying services.0029_remove_service_created... OK
  Applying base.0028_curatedthumbnail... OK
  Applying base.0029_resourcebase_last_updated... OK
  Applying base.0030_resourcebase_created... OK
  Applying base.0031_auto_20200114_1651... OK
  Applying base.0032_auto_20200115_1121... OK
  Applying base.0033_auto_20200115_1145... OK
  Applying base.0034_auto_20200212_0410... OK
  Applying base.0035_auto_20200212_0428... OK
  Applying base.0036_resourcebase_doi... OK
  Applying base.0037_auto_20200305_1520... OK
  Applying base.0038_configuration... OK
  Applying base.0039_auto_20200321_1338... OK
  Applying base.0038_delete_backup... OK
  Applying base.0040_merge_20200321_2245... OK
  Applying groups.24_initial... OK
  Applying groups.26_to_27... OK
  Applying groups.0027_auto_20180105_1631_squashed_0028_auto_20180606_1543... OK
  Applying groups.0028_auto_20200114_1651... OK
  Applying groups.0029_auto_20200115_1121... OK
  Applying groups.0030_auto_20200221_1107... OK
  Applying groups.0031_auto_20200305_1522... OK
  Applying groups.0031_auto_20200304_1434... OK
  Applying groups.0032_merge_20200306_1153... OK
  Applying base.0038_auto_20200318_0953... OK
  Applying base.0041_merge_20200323_1119... OK
  Applying base.0042_auto_20200414_1554... OK
  Applying base.0043_auto_20200527_0833... OK
  Applying base.0044_resourcebase_bbox_polygon...Traceback (most recent call last):
  File "/usr/bin/django-admin", line 5, in <module>
    management.execute_from_command_line()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/core/management/commands/migrate.py", line 232, in handle
    post_migrate_state = executor.migrate(
  File "/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/lib/python3/dist-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/lib/python3/dist-packages/django/db/migrations/operations/fields.py", line 110, in database_forwards
    schema_editor.add_field(
  File "/usr/lib/python3/dist-packages/django/db/backends/base/schema.py", line 433, in add_field
    definition, params = self.column_sql(model, field, include_default=True)
  File "/usr/lib/python3/dist-packages/django/db/backends/base/schema.py", line 150, in column_sql
    db_params = field.db_parameters(connection=self.connection)
  File "/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py", line 696, in db_parameters
    type_string = self.db_type(connection)
  File "/usr/lib/python3/dist-packages/django/contrib/gis/db/models/fields.py", line 105, in db_type
    return connection.ops.geo_db_type(self)
AttributeError: 'DatabaseOperations' object has no attribute 'geo_db_type'
/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField Profile.last_login received a naive datetime (2011-06-09 15:45:34) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"
/usr/lib/python3/dist-packages/django/db/models/fields/__init__.py:1424: RuntimeWarning: DateTimeField Profile.date_joined received a naive datetime (2011-06-09 15:15:27) while time zone support is active.
  warnings.warn("DateTimeField %s received a naive datetime (%s)"
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedColumn: column "language" of relation "people_profile" does not exist
LINE 1: ...area" = NULL, "zipcode" = NULL, "country" = NULL, "language"...

kalxas commented 2 years ago

Some more context: GeoNode 3.3.x Django 2.2.x installer: https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_geonode.sh local_settings.py: https://github.com/OSGeo/OSGeoLive/blob/master/app-conf/geonode/local_settings.py.sample geonode ppa: https://launchpad.net/%7Egcpp-kalxas/+archive/ubuntu/geonode/+packages?field.name_filter=&field.status_filter=published&field.series_filter=jammy osgeolive ppa: https://launchpad.net/%7Eosgeolive/+archive/ubuntu/nightly/+packages?field.name_filter=&field.status_filter=published&field.series_filter=jammy Ubuntu Version: 22.04 Python: 3.10

kalxas commented 2 years ago

One issue I can think of is the idna version which is 3.x instead of <2.11 in ubuntu 22.04

kalxas commented 2 years ago

I hope this will solve this issue: https://github.com/OSGeo/OSGeoLive/commit/9aacbdc2e668294f010cfb2ac45e07af04562800

kalxas commented 2 years ago

The migrations finally worked after downgrading psycopg2 version but when I try the application I get the following:

@afabiani any ideas?

Current iso is available: http://download.osgeo.org/livedvd/nightly/osgeolive-nightly-build123-amd64-9aacbdc-master.iso and build logs: http://download.osgeo.org/livedvd/nightly/osgeolive-nightly-build123-amd64-9aacbdc-master-log.tar.gz

afabiani commented 2 years ago

@kalxas currently not, I will need to try the iso myself. Usually that is an issue with the ubuntu UWSGI service. I often choose to create a service myself and allow it starting uwsgi via the command line directly: uwsgi --ini <path_to_geonode.ini>

kalxas commented 2 years ago

GeoNode is deployed directly on Apache in the iso, not uwsgi

afabiani commented 2 years ago

@kalxas mod_wsgi ? In that case I don't know, most probably an issue with the virtualenv paths

kalxas commented 2 years ago

@afabiani yes, I will also give a deeper look on that iso setup.

kalxas commented 2 years ago

@afabiani update: please use 15.0beta1 iso to reproduce the issue: http://download.osgeo.org/livedvd/releases/15.0/osgeolive-15.0beta1-amd64.iso http://download.osgeo.org/livedvd/releases/15.0/osgeolive-15.0beta1-amd64-log.tar.gz

afabiani commented 2 years ago

@kalxas so far I managed to make GeoNode starting on OSGEOLive, it looks like the user_agents package and middleware are causing some issues with mod_wsgi (if you need I can send you the updated settings.py).

However the system overall needs to be further configured in order to work well.

As an instance there's no connection to a DB (which is now mandatory) and the GeoServer proxy_pass does not work as expected. Moreover the Vanilla version of GeoServer cannot work without the plugins needed for GeoNode.

Long story short... there's still a lot of configuration stuff to do here to make GeoNode working correctly on this VM.

kalxas commented 2 years ago

Thank you @afabiani for the update. Please share the settings.py changes, we have to start from somewhere :)

afabiani commented 2 years ago

@kalxas here below the changes you have to apply in order to make Apache HTTPD working on the OSGEOLive machine

diff --git a/geonode/monitoring/models.py b/geonode/monitoring/models.py
index e17e2d189..9a2c31d65 100644
--- a/geonode/monitoring/models.py
+++ b/geonode/monitoring/models.py
@@ -44,7 +44,6 @@ except ImportError:
     except ImportError:
         pass

-import user_agents
 from ipware import get_client_ip
 import pycountry
 from geonode.monitoring.forms import MultiEmailField
@@ -543,6 +542,7 @@ class RequestEvent(models.Model):

     @staticmethod
     def _get_ua_family(ua):
+        import user_agents
         return str(user_agents.parse(ua))

     @classmethod

diff --git a/geonode/settings.py b/geonode/settings.py
index 3622d787d..732944fb4 100644
--- a/geonode/settings.py
+++ b/geonode/settings.py
@@ -493,7 +493,7 @@ INSTALLED_APPS = (
     'django_celery_beat',
     'django_celery_results',
     'markdownify',
-    'django_user_agents',
+    # 'django_user_agents',

     # REST APIs
     'rest_framework',
@@ -770,7 +770,7 @@ MIDDLEWARE = (
     'django.middleware.clickjacking.XFrameOptionsMiddleware',
     'django.middleware.security.SecurityMiddleware',
     'oauth2_provider.middleware.OAuth2TokenMiddleware',
-    'django_user_agents.middleware.UserAgentMiddleware',
+    # 'django_user_agents.middleware.UserAgentMiddleware',
     'geonode.base.middleware.MaintenanceMiddleware',
     'geonode.base.middleware.ReadOnlyMiddleware',   # a Middleware enabling Read Only mode of Geonode
 )

kalxas commented 2 years ago

Thank you @afabiani I am going to patch and test now.

kalxas commented 2 years ago

The above patch worked fine, thanks @afabiani ! Next? How to fix proxy_pass?

afabiani commented 2 years ago

Hi @kalxas next step would be to use a geoserver WAR from here https://artifacts.geonode.org/index.html which contains already the plugins we need. The data_dir should be also aligned accordingly, you can use as an example the following one

https://github.com/GeoNode/geoserver-geonode-ext/tree/2.20.5/geoserver/data/geonode_v2205_osgeolive

kalxas commented 1 year ago

Hi @afabiani FYI I am now preparing GeoNode 4.0.3 packages for OSGeoLive 16.0

GeoNode / geonode

Debian packages for GeoNode and OSGeoLive #6703