Automatic SQL registration doesn't work

[edobrynin-dodo]

Describe the bug

I have mosaic 0.3.7.jar installed on my databricks cluster.

I also put this jar into these locations:

• /databricks/jars
• /FileStore/mosaic/jars
• /FileStore/jars/mosaic/jars

I put these spark configs to my cluster's "Advanced options" section:

• spark.sql.extensions com.databricks.labs.mosaic.sql.extensions.MosaicSQL
• spark.databricks.labs.mosaic.geometry.api JTS
• spark.databricks.labs.mosaic.index.system H3

However, when I'm trying to check if Mosaic functions were registered:...

spark.sql("""show functions""").where("startswith(function, 'st_')").display()

...the result is zero unfortunately.

Expected behavior display() call shows functions from Mosaic library titled with "st_"

Additional context

• Databricks Runtime == 12.1 (includes Apache Spark 3.3.1, Scala 2.12), Photon OFF, Pricing tier == Standard

May I ask for help? Probably I do something wrong:( Thank you in advance!

[edurdevic]

Hi @edobrynin-dodo, I see that you are installing the mosaic library from the Cluster UI. This procedure installs the library after the cluster is already started, after the SQL registration trigger. In order to make it work, you need to install the Mosaic library via init script like described here.

Please let us know if that works.

[edobrynin-dodo]

Hi @edurdevic!

Got it, thank you.

I think the issue can be closed then.

May I create a PR to add this info in documentation? For example in Apache Sedona they specifically write about it: