-
### Description:
I deployed the radanalytics/spark-operator on OKD 4.6 (using OpenDataHub, you can find the full ODH manifests we are using here: https://github.com/MaastrichtU-IDS/odh-manifest…
-
I have tried the [PySpark Shell](https://docs.delta.io/latest/quick-start.html#pyspark-shell).
I executed `pip install pyspark==3.4.1`. However, I missed the instruction of `pip install delta-spark…
-
I'm tried to use spark-excel in Azure Databricks but I seem to be be running into an error. I earlier tried the same using **SQLServer Big Data Cluster** but I was unable [to](https://github.com/creal…
-
Running,
`spark.read.table('database.table').limit(10).show()`
is a lot faster than running,
`spark.sql('SELECT * from database.table limit 10')`
Intuitively, we would expect both of the…
-
The UNIX time conversion depends on a system time zone. For example, PySpark's ```F.unix_timestamp``` can produce different results for users in different time zones. One can achieve reproducible resu…
-
I've enabled iceberg.engine.hive.enabled=true in hive-site.xml.
1. in spark sql, create iceberg tables with hive catalog
spark3-shell --jars iceberg-spark3-runtime-0.11.1.jar \
--conf spark.sql.cat…
-
**Is your feature request related to a problem? Please describe.**
We have copied some of the code from `Object Canonicalize` .
Need to evaluate if this change should be pulled in https://github.co…
-
@icexelloss
(even though the "timeColumn" argument error can be bypassed by renaming the column in question to `time`) the `joinLeft` is not working for me:
```
print( sc.version )
print( tm )
…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
I'm trying to read an excel file in databricks that has some very large text fields and I'm g…
-
### Describe the issue
When running `databricks bundle deploy` I get the following error
*Potenitally sensitive info replaced with ``*
```bash
Updating deployment state...
Error: terraform ap…