databrickslabs / mosaic

An extension to the Apache Spark framework that allows easy and fast processing of very large geospatial datasets.
https://databrickslabs.github.io/mosaic/
Other
269 stars 66 forks source link

Bug: `st_geomfromwkb` no attribute `config.mosaic_context` #338

Closed aw-west-defra closed 1 year ago

aw-west-defra commented 1 year ago

Describe the bug I cannot load my data from WKB in python. AttributeError: module 'mosaic.config.config' has no attribute 'mosaic_context'

To Reproduce

import mosaic
filepath = ...
df = (spark.read.parquet(filepath)
  .withColumn('geometry', st_geomfromwkb('geometry'))
)

Additional context The parquet file loads correctly as WKB. Briefly looking at mosaic.config.config, I think you use : instead of =.

Screenshots image

Trackback

AttributeError                            Traceback (most recent call last)
<command-2659424542065347> in <module>
      2 f = 'dbfs:/tmp/osm_water.parquet'  # GeoParquet single file
      3 df = (spark.read.parquet(f)
----> 4   .withColumn('geometry', mosaic.st_geomfromwkb('geometry'))
      5 )
      6 display(df)

/local_disk0/.ephemeral_nfs/envs/pythonEnv-070f49e2-6a19-484d-9c62-f41e6c4ea63a/lib/python3.8/site-packages/mosaic/api/constructors.py in st_geomfromwkb(geom)
    126 
    127     """
--> 128     return config.mosaic_context.invoke_function(
    129         "st_geomfromwkb", pyspark_to_java_column(geom)
    130     )

AttributeError: module 'mosaic.config.config' has no attribute 'mosaic_context'
aw-west-defra commented 1 year ago

mosaic.__version__ == 0.3.9

aw-west-defra commented 1 year ago

The whole api is failing.

image

aw-west-defra commented 1 year ago

Ops, my bad.

mosaic.enable_mosaic(spark, dbutils)