GDAL 'not available'

[ajinkya933]

I have installed GDAL through conda:

$ conda install -c conda-forge gdal==2.4.4

I get :

Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

After that I installed:

python3 -m pip install pyrasterframes

But, when I run :

from pyrasterframes.utils import gdal_version
print(gdal_version())

I get response:

not available

Why do I get this error. I actually have gdal installed and also import gdal works . Is there a way I can fix tis error ?

[vpipkt]

One clarifying question, when you run python3 -m pip install pyrasterframes is the conda environment with gdal 2.4.4 activated? e.g.

$ conda install -n some_conda_env -c conda-forge gdal==2.4.4
$ conda activate some_conda_env
(some_conda_env) $ python3 -m pip install pyrasterframes
(some_conda_env) $ python3 -c "from pyrasterframes.utils import
gdal_version; print(gdal_version())

On Wed, Sep 16, 2020 at 1:35 PM sleeping barber notifications@github.com wrote:

I have installed GDAL through conda:

$ conda install -c conda-forge gdal==2.4.4

I get :

Collecting package metadata (current_repodata.json): done Solving environment: done

All requested packages already installed.

After that I installed:

python3 -m pip install pyrasterframes

But, when I run :

from pyrasterframes.utils import gdal_version print(gdal_version())

I get response:

not available

Why do I get this error. I actually have gdal installed and also import gdal works . Is there a way I can fix tis error ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/locationtech/rasterframes/issues/507, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3P4L26CXQILCJFH7GT6JTSGDZPPANCNFSM4RPFP3VQ .

[ajinkya933]

@vpipkt I do:

$ conda info --envs

# conda environments:
#
base                     /opt/conda
some_conda            *  /opt/conda/envs/some_conda

Then I install GDAL

conda install -n some_conda -c conda-forge gdal==2.4.4

After that I activated :

conda activate some_conda

$ python3
Python 3.6.11 | packaged by conda-forge | (default, Jul 23 2020, 22:18:32)
>>> from pyrasterframes.utils import gdal_version
>>> print(gdal_version())

I get this error :

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/some_conda/lib/python3.6/site-packages/pyrasterframes/utils.py", line 106, in gdal_version
    fcn = RFContext.active().lookup("buildInfo")
  File "/opt/conda/envs/some_conda/lib/python3.6/site-packages/pyrasterframes/rf_context.py", line 84, in active
    "RasterFrames have not been enabled for the active session. Call 'SparkSession.withRasterFrames()'.")
AttributeError: RasterFrames have not been enabled for the active session. Call 'SparkSession.withRasterFrames()'.

In order to resolve this error I did:

>>> from pyrasterframes.utils import create_rf_spark_session
>>> from pyrasterframes.utils import gdal_version
>>> spark = create_rf_spark_session()

20/09/17 07:04:40 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

>>> print(gdal_version())
20/09/17 07:04:54 WARN GDALRasterSource$: GDAL native bindings are not available. Falling back to JVM-based reader for GeoTIFF format.

not available

But still I get the error:

GDAL not available. Why is this ?

[vpipkt]

Apologies it should be:

(some_conda_env) $ python3 -c "from pyrasterframes.utils import
create_rf_spark_session, gdal_version; spark = create_rf_spark_session();
print(gdal_version());"

On Thu, Sep 17, 2020 at 3:07 AM sleeping barber notifications@github.com wrote:

@vpipkt https://github.com/vpipkt I do:

$ conda info --envs

conda environments:

# base /opt/conda some_conda * /opt/conda/envs/some_conda

Then I install GDAL

conda install -n some_conda -c conda-forge gdal==2.4.4

After that I activated :

conda activate some_conda

$ python3 Python 3.6.11 | packaged by conda-forge | (default, Jul 23 2020, 22:18:32)

from pyrasterframes.utils import gdal_version print(gdal_version())

I get this error :

Traceback (most recent call last): File "", line 1, in File "/opt/conda/envs/some_conda/lib/python3.6/site-packages/pyrasterframes/utils.py", line 106, in gdal_version fcn = RFContext.active().lookup("buildInfo") File "/opt/conda/envs/some_conda/lib/python3.6/site-packages/pyrasterframes/rf_context.py", line 84, in active "RasterFrames have not been enabled for the active session. Call 'SparkSession.withRasterFrames()'.") AttributeError: RasterFrames have not been enabled for the active session. Call 'SparkSession.withRasterFrames()'.

In order to resolve this error I did:

from pyrasterframes.utils import create_rf_spark_session from pyrasterframes.utils import gdal_version spark = create_rf_spark_session()

20/09/17 07:04:40 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

print(gdal_version()) 20/09/17 07:04:54 WARN GDALRasterSource$: GDAL native bindings are not available. Falling back to JVM-based reader for GeoTIFF format. not available

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/locationtech/rasterframes/issues/507#issuecomment-694008950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB3P4L2W3YSDJVOE62L4NXLSGGYRVANCNFSM4RPFP3VQ .

[vpipkt]

marking as question until we can show there is some unexpected behavior going on

[vpipkt]

@ajinkya933 did the above resolve your issue/question?

[ajinkya933]

No

[vpipkt]

Happy to engage on our gitter chat, we can try to work through the details. Feel free to update the issue here with more details about your environment etc. Which OS are you using? And for reference what pyrasterframes version?

My next guess here is that perhaps providing the gdal library path as a JVM option may help. Refer to this thread: https://github.com/locationtech/rasterframes/issues/356#issuecomment-554503657

[tosen1990]

Why @ajinkya933 closed this issue? Did you fix your problem? I've got the same error. Install GDAL 2.4.4 through conda 4.9.2 and macos Catalina version 10.15.6.

[pomadchin]

@tosen1990 since you're using MacOS your issue can not be that easily resolved. We fixed that only in gdal-warp bindings 3.1.1 that is only compatible with GDAL 3.1.x.

Try to depend on GDAL 3.1.x and gdal-warp-bindings 1.1.1, it should resolve your issues.