django.gis: cannot use GDAL in python project

pasqui23 commented 5 years ago

Issue description

I have a django project which uses the gis contribution module.

python ./manage.py check                            
Traceback (most recent call last):
  File "./manage.py", line 16, in <module>
    execute_from_command_line(sys.argv)
  File "~/.local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "~/.local/lib/python3.7/site-packages/django/core/management/__init__.py", line 357, in execute
    django.setup()
  File "~/.local/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "~/.local/lib/python3.7/site-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "~/.local/lib/python3.7/site-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/nix/store/5ml275njn06n151a4yvjc385xkxmjl61-python3-3.7.3/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "~/.local/lib/python3.7/site-packages/django/contrib/auth/models.py", line 2, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "~/.local/lib/python3.7/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
    class AbstractBaseUser(models.Model):
  File "~/.local/lib/python3.7/site-packages/django/db/models/base.py", line 117, in __new__
    new_class.add_to_class('_meta', Options(meta, app_label))
  File "~/.local/lib/python3.7/site-packages/django/db/models/base.py", line 321, in add_to_class
    value.contribute_to_class(cls, name)
  File "~/.local/lib/python3.7/site-packages/django/db/models/options.py", line 204, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "~/.local/lib/python3.7/site-packages/django/db/__init__.py", line 28, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "~/.local/lib/python3.7/site-packages/django/db/utils.py", line 201, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "~/.local/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
    return import_module('%s.base' % backend_name)
  File "/nix/store/5ml275njn06n151a4yvjc385xkxmjl61-python3-3.7.3/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 6, in <module>
    from .features import DatabaseFeatures
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/db/backends/postgis/features.py", line 1, in <module>
    from django.contrib.gis.db.backends.base.features import BaseSpatialFeatures
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/db/backends/base/features.py", line 3, in <module>
    from django.contrib.gis.db.models import aggregates
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/db/models/__init__.py", line 3, in <module>
    import django.contrib.gis.db.models.functions  # NOQA
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/db/models/functions.py", line 4, in <module>
    from django.contrib.gis.db.models.fields import BaseSpatialField, GeometryField
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/db/models/fields.py", line 3, in <module>
    from django.contrib.gis import forms, gdal
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/forms/__init__.py", line 3, in <module>
    from .fields import (  # NOQA
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/forms/fields.py", line 2, in <module>
    from django.contrib.gis.gdal import GDALException
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/gdal/__init__.py", line 28, in <module>
    from django.contrib.gis.gdal.datasource import DataSource
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/gdal/datasource.py", line 39, in <module>
    from django.contrib.gis.gdal.driver import Driver
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as vcapi, raster as rcapi
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 9, in <module>
    from django.contrib.gis.gdal.libgdal import GDAL_VERSION, lgdal
  File "~/.local/lib/python3.7/site-packages/django/contrib/gis/gdal/libgdal.py", line 43, in <module>
    % '", "'.join(lib_names)
django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0", "gdal1.11.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.

Steps to reproduce

Install Django
django-admin startproject test
Add a class with a field from django.contrib.gis
python ./manage.py check

Technical details

system: "x86_64-linux"
host os: Linux 5.1.15, NixOS, 19.09pre184359.73392e79aa6 (Loris)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.2.2
channels(root): "nixos-19.09pre184820.aa2a7e49b82"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

FRidh commented 5 years ago

Install Django

How? I take it you pip-installed something. How is that supposed to find GDAL?

pasqui23 commented 5 years ago

The description of pygdal says it is "virtulalenv friendly".I thought that it would find automatically GDAL if it were installed on the system.

pasqui23 commented 5 years ago

I have also installed gdal( not the python package) globally,but still gives this error even with GEOS_LIBRARY_PATH ="/run/current-system/sw/lib/" or GEOS_LIBRARY_PATH ="/run/current-system/sw/lib/libgdal.so"

lsix commented 4 years ago

Hi,

Sorry I only see this issue now. I hope that my answer is not too late and can still be usefull.

In nixpkgs, django's gdal support is optional in order to limit the closure size when using django.

To use gdal support, you need to override django (locally or globally) and use the useGdal parameter set to true:

nix-shell -p 'python3.withPackages (ps: [ (ps.django.override { withGdal = true; }) ps.ipython])' --command ipython
Python 3.7.5 (default, Oct 14 2019, 23:08:55) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import django.contrib.gis.gdal                                         

In [2]:

If you do the same without the override, the import will fail.

The override is available for all supported django versions.

stale[bot] commented 4 years ago

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

Search for maintainers and people that previously touched the related code and @ mention them in a comment.
Ask on the NixOS Discourse.
Ask on the #nixos channel on irc.freenode.net.

pasqui23 commented 4 years ago

I've eventually found a workaround by adding

GDAL_LIBRARY_PATH = "/run/current-system/sw/lib/libgdal.so"
GEOS_LIBRARY_PATH = "/run/current-system/sw/lib/libgeos_c.so"

to settings.py and creating an entire container,but it feels like a workaround and even with the above override still does not work without the above workaround

pasqui23 commented 3 years ago

@lsix to be clear without the above workaoround it won't work,even with the override,hence why I've left this issue open.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

pasqui23 commented 3 years ago

Still needs the workaround.

stale[bot] commented 2 years ago

I marked this as stale due to inactivity. → More info

pasqui23 commented 2 years ago

Nope,still needed

stale[bot] commented 2 years ago

I marked this as stale due to inactivity. → More info

henriod commented 1 year ago

This issue still persist

euporos commented 1 year ago

I can confirm this; workaround is still needed.

FRidh commented 1 year ago

If the override does not function then the patch needs to be fixed.

henriod commented 1 year ago

For deployment in Railways i just went to Dockerfile road. Solve the gdal issue for geodjango application

ThibaultLemaire commented 1 year ago

Hi,

Sorry I only see this issue now. I hope that my answer is not too late and can still be usefull.

In nixpkgs, django's gdal support is optional in order to limit the closure size when using django.

To use gdal support, you need to override django (locally or globally) and use the useGdal parameter set to true:
nix-shell -p 'python3.withPackages (ps: [ (ps.django.override { withGdal = true; }) ps.ipython])' --command ipython
Python 3.7.5 (default, Oct 14 2019, 23:08:55) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import django.contrib.gis.gdal                                         

In [2]:  
If you do the same without the override, the import will fail.

The override is available for all supported django versions.

Thank

Could this be documented somewhere?

NixOS / nixpkgs