Closed avivshafir closed 2 months ago
No response
Sentence embedding in pyspark
The model is not being found
Model should be found
document_assembler = DocumentAssembler().setInputCol("text").setOutputCol("document") sentencerDL = ( SentenceDetectorDLModel.pretrained("sentence_detector_dl", "xx") .setInputCols(["document"]) .setOutputCol("sentence") ) embeddings = ( XlmRoBertaSentenceEmbeddings.pretrained("bge_m3 ","xx") .setInputCols(["sentence"]) .setOutputCol("embeddings") ) pipeline = Pipeline().setStages([document_assembler, sentencerDL, embeddings]) df = spark.createDataFrame([["John Doe", "Jane Doe"]]).toDF("text")
sparknlp=='5.3.3' jar version=5.3.3 spark_version='3.5.0'
databricks runtime version: 14.3.x-gpu-ml-scala2.12
spark-submit
This is our fault, the code snippet has an extra space in the name. Thanks for reporting it, I'll fix the model's card.
space
Is there an existing issue for this?
Who can help?
No response
What are you working on?
Sentence embedding in pyspark
Current Behavior
The model is not being found
Expected Behavior
Model should be found
Steps To Reproduce
Spark NLP version and Apache Spark
sparknlp=='5.3.3' jar version=5.3.3 spark_version='3.5.0'
databricks runtime version: 14.3.x-gpu-ml-scala2.12
Type of Spark Application
spark-submit
Java Version
No response
Java Home Directory
No response
Setup and installation
No response
Operating System and Version
No response
Link to your project (if available)
No response
Additional Information
No response