search

JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing
https://sparknlp.org/
Apache License 2.0
3.76k stars 703 forks source link

A fatal error has been detected by the Java Runtime Environment when I load model from hdfs #14272

Closed xueyuan1990 closed 1 month ago

xueyuan1990 commented 1 month ago

Is there an existing issue for this?

I find it here(I'm not sure it's the same error) : https://github.com/JohnSnowLabs/spark-nlp/issues/5918

Current Behavior

I got error below:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00007fa16bcb5260, pid=23030, tid=0x00007fa2af75b700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_144-b01) (build 1.8.0_144-b01)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.144-b01 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libtensorflow_framework.so.2+0x16bc260]  nsync::nsync_mu_init(nsync::nsync_mu_s_*)+0x0
#
# Core dump written. Default location: /home/xueyuan.zhang/article_recom/core or core.23030
#
# An error report file with more information is saved as:
# /home/xueyuan.zhang/article_recom/hs_err_pid23030.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
./test_spark_nlp.sh: line 20: 23030 Aborted                 (core dumped) /opt/spark3/bin/spark-submit --master yarn --name action_sequence_123 --deploy-mode client --files /opt/spark3/conf/hive-site.xml --class $class_name --jars hdfs:///apps/recommend/models/jars/xueyuan/mzreader/spark-nlp_2.12-5.3.3.jar,hdfs:///apps/recommend/models/jars/xueyuan/mzreader/aws-java-sdk-core-1.12.722.jar,hdfs:///apps/recommend/models/jars/xueyuan/mzreader/tensorflow-cpu_2.12-0.4.4.jar $jar_file $today

hs_err_pid23030.log

Steps To Reproduce

I load model from https://sparknlp.org/2022/04/11/bert_embeddings_chinese_roberta_wwm_ext_zh_3_0.html and put it onto hdfs_path. But when I load it, the error comes.

import com.johnsnowlabs.nlp.embeddings.BertEmbeddings
val hdfs_path="...../bert_embeddings_chinese_roberta_wwm_ext_zh"
val embeddings = BertEmbeddings.load(hdfs_path)

spark-submit :

#!/bin/bash
today=$1
jar_file="./article_recommender_spark3-2.0-SNAPSHOT.jar"
class_name="com.algo.recom.article_recommender.v20240511.test_spark_nlp"
/opt/spark3/bin/spark-submit \
--master yarn \
--name action_sequence_123 \
--deploy-mode client \
--files /opt/spark3/conf/hive-site.xml \
--class $class_name \
--jars hdfs:///apps/recommend/models/jars/xueyuan/mzreader/spark-nlp_2.12-5.3.3.jar,hdfs:///apps/recommend/models/jars/xueyuan/mzreader/aws-java-sdk-core-1.12.722.jar,hdfs:///apps/recommend/models/jars/xueyuan/mzreader/tensorflow-cpu_2.12-0.4.4.jar \
$jar_file $today

Spark NLP version and Apache Spark

com.johnsnowlabs.nlp spark-nlp_2.12 5.3.3

spark version 3.3.2 Scala version 2.12.15 Java HotSpot(TM) 64-Bit Server VM, 1.8.0_144

maziyarpanahi commented 1 month ago

hi, could you please share some details regarding your:

xueyuan1990 commented 1 month ago

CentOS Linux 7 CPU spark-nlp_2.12-5.3.3.jar is provided in this link : https://sparknlp.org/docs/en/install#scala-and-java

<!-- https://mvnrepository.com/artifact/com.johnsnowlabs.nlp/spark-nlp -->
<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp_2.12</artifactId>
    <version>5.3.3</version>
</dependency>

I don't know where is the Fat JAR.

maziyarpanahi commented 1 month ago

I suspect you are either missing some dependencies or your OS is too old.

Please use the Fat JAR instead and try BertEmbeddings.pretrained() for this test.

xueyuan1990 commented 1 month ago

I use the Fat JAR downloaded from https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/jars/spark-nlp-assembly-5.3.3.jar. And try BertEmbeddings.pretrained("bert_embeddings_chinese_roberta_wwm_ext","zh"). But the error still comes. So you think my OS is too old. What's your suggest version?

maziyarpanahi commented 1 month ago

Thanks, but not to rule out the model's issue, I need you to just use BertEmbeddings.pretrained() no model inside. I just need to see if it's the model that has the problem or the OS. (from my experience, CentOS 7 is just way too old to support new TensorFlow/ONNX requirements, but let's see if the default model works)

please just run this code:

bert = BertEmbeddings.pretrained()

The default model is ONNX, so I want to see what happens when you use ONNX instead of TF.

maziyarpanahi commented 1 month ago

So this is what I was worried about, your Operating System is old. Both TensorFlow and ONNX use a much newer libstdc++. So you either have to upgrade the machines to a newer release that comes with newer libstdc++ or somehow manually update them.

xueyuan1990 commented 1 month ago

I upgrade my OS to CentOS Linux release 7.9.2009 (Core) and gcc to gcc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3). BertEmbeddings.pretrained() can load model successfully. But BertEmbeddings.pretrained("bert_embeddings_chinese_roberta_wwm_ext","zh") still can not. The error still comes:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00007f5cd2387260, pid=6533, tid=0x00007f5e08838700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_144-b01) (build 1.8.0_144-b01)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.144-b01 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libtensorflow_framework.so.2+0x16bc260]  nsync::nsync_mu_init(nsync::nsync_mu_s_*)+0x0
#
# Core dump written. Default location: /home/xueyuan.zhang/article_recom/core or core.6533
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x00007f5e00017800):  JavaThread "main" [_thread_in_native, id=6613, stack(0x00007f5e08739000,0x00007f5e08839000)]

siginfo: si_signo: 4 (SIGILL), si_code: 2 (ILL_ILLOPN), si_addr: 0x00007f5cd2387260

Registers:
RAX=0x00007f5ccc0d0a80, RBX=0x00007f5cd2b89aa0, RCX=0x00007f5e00000020, RDX=0x00007f5ccc0d0a80
RSP=0x00007f5e08832c58, RBP=0x00007f5e08832d60, RSI=0x00007f5ccc0d0b00, RDI=0x00007f5ccc0d0a80
R8 =0x00000000000001b1, R9 =0x00007f5e00000740, R10=0x00007f5e000008d0, R11=0x0000000000000002
R12=0x00007f5ccc0d0a80, R13=0x00007f5e08832cd8, R14=0x00007f5e08832ce0, R15=0x00007f5e08832ca0
RIP=0x00007f5cd2387260, EFLAGS=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000000
  TRAPNO=0x0000000000000006

Top of Stack: (sp=0x00007f5e08832c58)
0x00007f5e08832c58:   00007f5cd14357a7 d86fca2c22619800
0x00007f5e08832c68:   00007f5e08832cb0 00007f5ccc0d0c38
0x00007f5e08832c78:   00007f5ccc0d0c50 00007f5e08832c9f
0x00007f5e08832c88:   00007f5ccc0d0c40 61636f6c3674534e
0x00007f5e08832c98:   746563616635656c 00007f5e01785c08
0x00007f5e08832ca8:   00007f5e01785c38 00007f5ccc0d0c30
0x00007f5e08832cb8:   00007f5ccc0d0c40 00007f5ccc0d0c40
0x00007f5e08832cc8:   00007f5cd2b7b440 0000000000000001
0x00007f5e08832cd8:   00007f5e017f2f28 00007f5ccc042ab8
0x00007f5e08832ce8:   00007f5ccc0d0c50 00007f5ccc0d0c60
0x00007f5e08832cf8:   00007f5ccc0d0c60 00007f5cd2b87c80
0x00007f5e08832d08:   00007f5cd2ac0f98 00007f5cd2b886e0
0x00007f5e08832d18:   00007f5e08832d28 0000000000ff0000
0x00007f5e08832d28:   4744c89227432000 00007f5cd2acad00
0x00007f5e08832d38:   00007f5cd28759b0 00007fffd55e7958
0x00007f5e08832d48:   00007f5e0009c720 00007f5cd2876ab0
0x00007f5e08832d58:   0000000000000000 00007f5e08832e30
0x00007f5e08832d68:   00007f5cd140039a 00007f5cd2b874a8
0x00007f5e08832d78:   00007f5cd2afb9ed 00007f5cd2b87db8
0x00007f5e08832d88:   00007f5cd2b874a8 00007f5cd2b86f60
0x00007f5e08832d98:   00007f5cd2afbc25 00007f5cd2b809d8
0x00007f5e08832da8:   00007f5cd2b874a0 00007f5cd2b875c0
0x00007f5e08832db8:   00007f5cd2b0b09d 00007f5e07fd96a0
0x00007f5e08832dc8:   00007f5cd2b86f60 00007f5cd2b89654
0x00007f5e08832dd8:   00007f5cd2b875c0 00007f5cd2b87b60
0x00007f5e08832de8:   00007f5cd2acaccd 00007f5e0009c720
0x00007f5e08832df8:   4744c89227432000 00007f5cd2acad00
0x00007f5e08832e08:   00007f5cd28759b0 00007fffd55e7958
0x00007f5e08832e18:   00007f5e0009c720 00007f5cd2876ab0
0x00007f5e08832e28:   0000000000000000 0000000000000021
0x00007f5e08832e38:   00007f5e086260fa 00000000ffffffff
0x00007f5e08832e48:   0000000000000021 00007fffd55e7958 

Instructions: (pc=0x00007f5cd2387260)
0x00007f5cd2387240:   31 c0 c3 66 66 66 66 2e 0f 1f 84 00 00 00 00 00
0x00007f5cd2387250:   8b 07 89 c2 83 e2 fd f0 0f b1 17 75 f3 c3 66 90
0x00007f5cd2387260:   c5 f9 ef c0 c5 f8 11 07 c3 0f 1f 80 00 00 00 00
0x00007f5cd2387270:   55 48 89 e5 41 57 4c 8d 7e 08 41 56 41 55 49 89 

Register to memory mapping:

RAX=0x00007f5ccc0d0a80 is an unknown value
RBX=0x00007f5cd2b89aa0: _ZNSs4_Rep20_S_empty_rep_storageE+0 in /lib64/libstdc++.so.6 at 0x00007f5cd2a16000
RCX=0x00007f5e00000020 is an unknown value
RDX=0x00007f5ccc0d0a80 is an unknown value
RSP=0x00007f5e08832c58 is pointing into the stack for thread: 0x00007f5e00017800
RBP=0x00007f5e08832d60 is pointing into the stack for thread: 0x00007f5e00017800
RSI=0x00007f5ccc0d0b00 is an unknown value
RDI=0x00007f5ccc0d0a80 is an unknown value
R8 =0x00000000000001b1 is an unknown value
R9 =0x00007f5e00000740 is an unknown value
R10=0x00007f5e000008d0 is an unknown value
R11=0x0000000000000002 is an unknown value
R12=0x00007f5ccc0d0a80 is an unknown value
R13=0x00007f5e08832cd8 is pointing into the stack for thread: 0x00007f5e00017800
R14=0x00007f5e08832ce0 is pointing into the stack for thread: 0x00007f5e00017800
R15=0x00007f5e08832ca0 is pointing into the stack for thread: 0x00007f5e00017800

Stack: [0x00007f5e08739000,0x00007f5e08839000],  sp=0x00007f5e08832c58,  free space=999k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libtensorflow_framework.so.2+0x16bc260]  nsync::nsync_mu_init(nsync::nsync_mu_s_*)+0x0
C  [libtensorflow_framework.so.2+0x73539a]  _GLOBAL__sub_I_loader.cc+0x6a
C  [ld-linux-x86-64.so.2+0xf0fa]  call_init.part.0+0x9a

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.lang.ClassLoader$NativeLibrary.load(Ljava/lang/String;Z)V+0
j  java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+328
j  java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+48
j  java.lang.Runtime.load0(Ljava/lang/Class;Ljava/lang/String;)V+57
j  java.lang.System.load(Ljava/lang/String;)V+7
j  org.bytedeco.javacpp.Loader.loadLibrary(Ljava/lang/Class;[Ljava/net/URL;Ljava/lang/String;[Ljava/lang/String;)Ljava/lang/String;+953
j  org.bytedeco.javacpp.Loader.load(Ljava/lang/Class;Ljava/util/Properties;ZLjava/lang/String;)Ljava/lang/String;+597
j  org.bytedeco.javacpp.Loader.load(Ljava/lang/Class;Ljava/util/Properties;Z)Ljava/lang/String;+4
j  org.bytedeco.javacpp.Loader.load()Ljava/lang/String;+10
j  org.tensorflow.internal.c_api.global.tensorflow.<clinit>()V+0
v  ~StubRoutines::call_stub
J 1053  java.lang.Class.forName0(Ljava/lang/String;ZLjava/lang/ClassLoader;Ljava/lang/Class;)Ljava/lang/Class; (0 bytes) @ 0x00007f5df13cc2ba [0x00007f5df13cc240+0x7a]
J 5231 C2 java.lang.Class.forName(Ljava/lang/String;ZLjava/lang/ClassLoader;)Ljava/lang/Class; (53 bytes) @ 0x00007f5df1e476c8 [0x00007f5df1e476a0+0x28]
j  org.bytedeco.javacpp.Loader.load(Ljava/lang/Class;Ljava/util/Properties;ZLjava/lang/String;)Ljava/lang/String;+286
j  org.bytedeco.javacpp.Loader.load(Ljava/lang/Class;Ljava/util/Properties;Z)Ljava/lang/String;+4
j  org.bytedeco.javacpp.Loader.load(Ljava/lang/Class;)Ljava/lang/String;+7
j  org.tensorflow.NativeLibrary.load()V+2
j  org.tensorflow.TensorFlow.<clinit>()V+0
v  ~StubRoutines::call_stub
J 1053  java.lang.Class.forName0(Ljava/lang/String;ZLjava/lang/ClassLoader;Ljava/lang/Class;)Ljava/lang/Class; (0 bytes) @ 0x00007f5df13cc2ba [0x00007f5df13cc240+0x7a]
J 2238 C1 java.lang.Class.forName(Ljava/lang/String;)Ljava/lang/Class; (15 bytes) @ 0x00007f5df172774c [0x00007f5df17275a0+0x1ac]
j  org.tensorflow.Graph.<clinit>()V+19
v  ~StubRoutines::call_stub
j  com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.readGraph(Ljava/lang/String;)Lorg/tensorflow/Graph;+12
j  com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.unpackWithoutBundle(Ljava/lang/String;)Lscala/Tuple4;+19
j  com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.read(Ljava/lang/String;ZZ[Ljava/lang/String;ZLscala/Option;)Lscala/Tuple2;+329
j  com.johnsnowlabs.ml.tensorflow.ReadTensorflowModel.readTensorflowModel(Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;Ljava/lang/String;ZZ[Ljava/lang/String;ZLscala/Option;)Lcom/johnsnowlabs/ml/tensorflow/TensorflowWrapper;+197
j  com.johnsnowlabs.ml.tensorflow.ReadTensorflowModel.readTensorflowModel$(Lcom/johnsnowlabs/ml/tensorflow/ReadTensorflowModel;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;Ljava/lang/String;ZZ[Ljava/lang/String;ZLscala/Option;)Lcom/johnsnowlabs/ml/tensorflow/TensorflowWrapper;+14
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.readTensorflowModel(Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;Ljava/lang/String;ZZ[Ljava/lang/String;ZLscala/Option;)Lcom/johnsnowlabs/ml/tensorflow/TensorflowWrapper;+14
j  com.johnsnowlabs.nlp.embeddings.ReadBertDLModel.readModel(Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)V+99
j  com.johnsnowlabs.nlp.embeddings.ReadBertDLModel.readModel$(Lcom/johnsnowlabs/nlp/embeddings/ReadBertDLModel;Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)V+4
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.readModel(Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)V+4
j  com.johnsnowlabs.nlp.embeddings.ReadBertDLModel.$anonfun$$init$$1(Lcom/johnsnowlabs/nlp/embeddings/ReadBertDLModel;Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)V+4
j  com.johnsnowlabs.nlp.embeddings.ReadBertDLModel.$anonfun$$init$$1$adapted(Lcom/johnsnowlabs/nlp/embeddings/ReadBertDLModel;Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)Ljava/lang/Object;+4
j  com.johnsnowlabs.nlp.embeddings.ReadBertDLModel$$Lambda$276.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+16
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$onRead$1(Lcom/johnsnowlabs/nlp/HasFeatures;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;Lscala/Function3;)V+4
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$onRead$1$adapted(Lcom/johnsnowlabs/nlp/HasFeatures;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;Lscala/Function3;)Ljava/lang/Object;+4
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$Lambda$2416.apply(Ljava/lang/Object;)Ljava/lang/Object;+16
J 8508 C2 scala.collection.mutable.ResizableArray.foreach(Lscala/Function1;)V (37 bytes) @ 0x00007f5df1f72a78 [0x00007f5df1f729e0+0x98]
J 6935 C2 scala.collection.mutable.ArrayBuffer.foreach(Lscala/Function1;)V (6 bytes) @ 0x00007f5df1ea3794 [0x00007f5df1ea3760+0x34]
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.onRead(Lcom/johnsnowlabs/nlp/HasFeatures;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)V+14
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$read$1(Lcom/johnsnowlabs/nlp/ParamsAndFeaturesReadable;Lcom/johnsnowlabs/nlp/HasFeatures;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)V+4
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$read$1$adapted(Lcom/johnsnowlabs/nlp/ParamsAndFeaturesReadable;Lcom/johnsnowlabs/nlp/HasFeatures;Ljava/lang/String;Lorg/apache/spark/sql/SparkSession;)Ljava/lang/Object;+4
j  com.johnsnowlabs.nlp.ParamsAndFeaturesReadable$$Lambda$1633.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+16
j  com.johnsnowlabs.nlp.FeaturesReader.load(Ljava/lang/String;)Lcom/johnsnowlabs/nlp/HasFeatures;+38
j  com.johnsnowlabs.nlp.FeaturesReader.load(Ljava/lang/String;)Ljava/lang/Object;+2
j  com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(Lorg/apache/spark/ml/util/DefaultParamsReadable;Lcom/johnsnowlabs/nlp/pretrained/ResourceRequest;)Lorg/apache/spark/ml/PipelineStage;+26
j  com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(Lorg/apache/spark/ml/util/DefaultParamsReadable;Ljava/lang/String;Lscala/Option;Ljava/lang/String;)Lorg/apache/spark/ml/PipelineStage;+25
j  com.johnsnowlabs.nlp.HasPretrained.pretrained(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/apache/spark/ml/PipelineStage;+45
j  com.johnsnowlabs.nlp.HasPretrained.pretrained$(Lcom/johnsnowlabs/nlp/HasPretrained;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/apache/spark/ml/PipelineStage;+4
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.com$johnsnowlabs$nlp$embeddings$ReadablePretrainedBertModel$$super$pretrained(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+4
j  com.johnsnowlabs.nlp.embeddings.ReadablePretrainedBertModel.pretrained(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+4
j  com.johnsnowlabs.nlp.embeddings.ReadablePretrainedBertModel.pretrained$(Lcom/johnsnowlabs/nlp/embeddings/ReadablePretrainedBertModel;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+4
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.pretrained(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+4
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.pretrained(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/apache/spark/ml/PipelineStage;+4
j  com.johnsnowlabs.nlp.HasPretrained.pretrained(Ljava/lang/String;Ljava/lang/String;)Lorg/apache/spark/ml/PipelineStage;+9
j  com.johnsnowlabs.nlp.HasPretrained.pretrained$(Lcom/johnsnowlabs/nlp/HasPretrained;Ljava/lang/String;Ljava/lang/String;)Lorg/apache/spark/ml/PipelineStage;+3
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.com$johnsnowlabs$nlp$embeddings$ReadablePretrainedBertModel$$super$pretrained(Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+3
j  com.johnsnowlabs.nlp.embeddings.ReadablePretrainedBertModel.pretrained(Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+3
j  com.johnsnowlabs.nlp.embeddings.ReadablePretrainedBertModel.pretrained$(Lcom/johnsnowlabs/nlp/embeddings/ReadablePretrainedBertModel;Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+3
j  com.johnsnowlabs.nlp.embeddings.BertEmbeddings$.pretrained(Ljava/lang/String;Ljava/lang/String;)Lcom/johnsnowlabs/nlp/embeddings/BertEmbeddings;+3
j  com.algo.recom.article_recommender.v20240511.test_spark_nlp$.main([Ljava/lang/String;)V+7
j  com.algo.recom.article_recommender.v20240511.test_spark_nlp.main([Ljava/lang/String;)V+4
v  ~StubRoutines::call_stub
j  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
j  sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+100
J 2140 C1 sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (10 bytes) @ 0x00007f5df16bb94c [0x00007f5df16bb840+0x10c]
J 2139 C1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x00007f5df16bb35c [0x00007f5df16baf60+0x3fc]
j  org.apache.spark.deploy.JavaMainApplication.start([Ljava/lang/String;Lorg/apache/spark/SparkConf;)V+97
j  org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(Lorg/apache/spark/deploy/SparkSubmitArguments;Z)V+449
j  org.apache.spark.deploy.SparkSubmit.doRunMain$1(Lorg/apache/spark/deploy/SparkSubmitArguments;Z)V+111
j  org.apache.spark.deploy.SparkSubmit.submit(Lorg/apache/spark/deploy/SparkSubmitArguments;Z)V+65
j  org.apache.spark.deploy.SparkSubmit.doSubmit([Ljava/lang/String;)V+78
j  org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit([Ljava/lang/String;)V+2
j  org.apache.spark.deploy.SparkSubmit$.main([Ljava/lang/String;)V+10
j  org.apache.spark.deploy.SparkSubmit.main([Ljava/lang/String;)V+4
v  ~StubRoutines::call_stub
maziyarpanahi commented 1 month ago

Great, there is something wrong with the model. suggest you either found another model with Chinese support, try to import the same model yourself with ONNX, or import another model yourself:

Import new model(s):