-
Hello,
I am trying to update my application to spark 3.5.1 (Scala version 2.12.18, OpenJDK 64-Bit Server VM, 17.0.10) but the scala 2.12 connector keeps throwing this error.
While trying to str…
-
We are using the latest release of the spark operator: `v1beta2-1.1.2-2.4.5`.
I believe that this is related to #537. I am adding an initContainer onto the driver pod like this:
```
apiVersion: "…
-
Hello, I tried to find about how to use spark streaming in this project but got nothing.(not structured streaming)
Even I tried "StreamingContext" and nothing be found.
So I want to ask if we ha…
-
Description: Develop a real-time streaming word count application leveraging Apache Spark Streaming's DStream API. Utilize Python to ingest streaming text data from a chosen source, tokenize the wor…
-
Description: Craft Python code within a notebook environment to orchestrate time-based aggregations, leveraging tumbling or sliding windows for granular insights into your streaming data. Refine your …
-
### Apache Iceberg version
1.5.0 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
When using spark readStream the option `stream-from-timestamp` is used to position…
-
I am using streamnative's Apache Pulsar connector version `3.4.4.1` with spark dependencies version `3.4.1`. When submitting the spark app on console, I am getting below error.
```java
Exception…
-
Difficulty: 2
Technology: Apache Spark Streaming
Description: Perform a join operation on two streams of data using Apache Spark Streaming's DStream API. Write Python code to define join logic b…
-
Hi, when i want to run the app
.PHONY:
streaming-app-run: # Run Spark Structured streaming app locally.
poetry run spark-submit \
$(SPARK_ARGS) \
movie_ratings_streaming/entrypoint.py
if…
-
1.streaming-socket.py
```python
import json
import socket
import time
import pandas as pd
def send_data_over_socket(file_path,host='127.0.0.1',port=9999,chunk_size=2):
s = socket.socket…