-
try if Spark 3.4.0 image will work
-
```
Python 2.7
Pyspark 2.2.1
JDBC format for MySQL->Spark DF
For writing Spark DF-> AWS Redshift i am using the `Spark-Redshift` driver from Databricks.
```
I am reading data into Spark from M…
-
Hello everyone!
I am trying to convert an instance of HashingTF sparkml transformer. When I invoke convert_sparkml function I get an error that says that 'pyspark.ml.feature.HashingTF' is not suppo…
sansr updated
3 years ago
-
I am curious if there are issues calling boto3 client from AppMaster when the network is isolated. Or, maybe my network config is off... not sure. I am running pyspark script using the PySparkProces…
-
Hi, maybe it's not a bug and it's just that I am missing something when using this library. I am trying to load a pretrained DistilBert model from the hugginface-transformer project. This model is sup…
-
Currently, Ubuntu users cannot simply `docker run` following the suggested setup in the main readme.
Common error messages were:
```
rico@laptop$ sudo docker run -p 8888:8888 -v $(pwd):/home/jovyan…
-
- Ibis
- R dplyr
- polars
- pandas 2.0
- pyspark
-
Hi,
In my CentOS Linux, I installed:
`1- ! pip install --upgrade -q pyspark==3.4.1 spark-nlp==5.2.2`
`2- ! pip install --upgrade spark-nlp-jsl==5.2.1 --user --extra-index-url https://pypi.johnsn…
-
use `pyspark` to write these alg should be meaningful
-
The UNIX time conversion depends on a system time zone. For example, PySpark's ```F.unix_timestamp``` can produce different results for users in different time zones. One can achieve reproducible resu…