-
### Query engine
- Spark 3.5
- Iceberg 1.5.2
- Azure storage account ABFSS
### Question
I’m trying to use the `MERGE INTO` statement within a Spark streaming application that runs several s…
-
**Describe the bug**
```
%dpl
| makeresults count=1 | teragrep exec kafka save myTopic
```
gives error from spark:
```
Failed to find data source: kafka. Please deploy the application as pe…
-
Is it planned to add an integration with Spark Streaming ? It could be useful to be able to apply some lineage for batch and streaming data
-
Hi,
So I was wondering if this valid to say use in Spark Streaming to maintain a global state lookup table in a batch, that can be updated in a Streaming Batch and refreshed periodically.
Help grea…
-
**Describe the problem you faced**
Hi Folks. I'm trying to get some advice here on how to better deal with a large upsert dataset.
The data has a very wide key space and no great/obvious partiti…
-
As peer of flink, we use Spark for transformation
-
Hi Team,
I would like to understand whether there is any plan for Spark Structured Streaming support. Currently, I could not see Spark Streaming dependency for this codebase.
Kindly request you …
-
It would be good to have an example of Cloudant data being analysed using Streaming KMeans - http://spark.apache.org/docs/latest/mllib-clustering.html#streaming-k-means
I can't think of an example of…
-
Sparklens seem to be waiting for the job to complete, however it doesn't seem to be working with the
Spark streaming jobs. Do you have any plan to fix it ?
-
Changes in JsonMRFSource.scala
```
case Some("abfss") =>
val serviceId = dbutils.secrets.get(scope="your-secrets-scope", key="azure-sp-id")
val serviceCredential = dbutils.secret…