search

voltrondata / spark-substrait-gateway

Implements a gateway that speaks the SparkConnect protocol and drives a backend using Substrait (over ADBC Flight SQL).
Apache License 2.0
15 stars 8 forks source link

fix: update reference to find_tpch #52

Closed amoeba closed 1 month ago

amoeba commented 1 month ago

Resolves bug introduced by 555495152ebc008018bc866fa47c9e158bb9d634.

amoeba commented 1 month ago

Before this PR,

spark-substrait-gateway-env ❯ python src/gateway/demo/client_demo.py
Traceback (most recent call last):
  File "/Users/bryce/src/voltrondata/spark-substrait-gateway/src/gateway/demo/client_demo.py", line 58, in <module>
    execute_query(spark)
  File "/Users/bryce/src/voltrondata/spark-substrait-gateway/src/gateway/demo/client_demo.py", line 35, in execute_query
    df_customer = get_customer_database(spark_session)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bryce/src/voltrondata/spark-substrait-gateway/src/gateway/demo/client_demo.py", line 26, in get_customer_database
    location_customer = str(Backend.find_tpch() / 'customer')
                            ^^^^^^^^^^^^^^^^^
AttributeError: type object 'Backend' has no attribute 'find_tpch'

after:

spark-substrait-gateway-env ❯ python src/gateway/demo/client_demo.py
+---------+------------------+--------------------+-----------+---------------+---------+------------+--------------------+
|c_custkey|            c_name|           c_address|c_nationkey|        c_phone|c_acctbal|c_mktsegment|           c_comment|
+---------+------------------+--------------------+-----------+---------------+---------+------------+--------------------+
|        9|Customer#000000009|xKiAFTjUsCuxfeleN...|          8|18-338-906-3675|  8324.07|   FURNITURE|r theodolites acc...|
|       14|Customer#000000014|     KXkletMlL2JQEA |          1|11-845-129-3851|   5266.3|   FURNITURE|, ironic packages...|
|       16|Customer#000000016| cYiaeMLZSMAOQ2 d0W,|         10|20-781-609-3107|  4681.03|   FURNITURE|kly silent courts...|
|       20|Customer#000000020|       JrPk8Pqplj4Ne|         22|32-957-234-8742|   7603.4|   FURNITURE|g alongside of th...|
|       25|Customer#000000025|Hp8GyFQgGHFYSilH5...|         12|22-603-468-3533|   7133.7|   FURNITURE|y. accounts sleep...|
|       28|Customer#000000028|iVyg0daQ,Tha8x2WP...|          8|18-774-241-1462|  1007.18|   FURNITURE| along the regula...|
|       29|Customer#000000029|sJ5adtfyAkCK63df2...|          0|10-773-203-7342|  7618.27|   FURNITURE|its after the car...|
|       37|Customer#000000037|   7EV4Pwh,3SboctTWt|          8|18-385-235-7162|  -917.75|   FURNITURE|ilent packages ar...|
|       49|Customer#000000049|cNgAeX7Fqrdf7HQN9...|         10|20-908-631-4424|  4573.94|   FURNITURE|nusual foxes! flu...|
|       51|Customer#000000051|         uR,wEaiTvo4|         12|22-344-885-4251|   855.87|   FURNITURE|eposits. furiousl...|
+---------+------------------+--------------------+-----------+---------------+---------+------------+--------------------+

[Row(c_custkey=2, c_phone='23-768-687-3665', c_mktsegment='AUTOMOBILE'), Row(c_custkey=3, c_phone='11-719-748-3364', c_mktsegment='AUTOMOBILE'), Row(c_custkey=4, c_phone='14-128-190-5944', c_mktsegment='MACHINERY'), Row(c_custkey=5, c_phone='13-750-942-6364', c_mktsegment='HOUSEHOLD'), Row(c_custkey=6, c_phone='30-114-968-4951', c_mktsegment='AUTOMOBILE')]
EpsilonPrime commented 1 month ago

Thanks for the fix!