Closed sarathsurpur closed 3 years ago
@sarathsurpur This repo only supports public and open-source features. What you are using is licensed (we have no info), please either contact them directly via email or on Slack you can ask them inside the #healthcare
channel.
Description
I am trying to use the Licensed version of SPARK NLP in health care domain to extract the ICD codes from clinical notes and I was following this notebook https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/3.Clinical_Entity_Resolvers.ipynb I was able to download all the models except icd_resolver and clinical_ner. But the same model like clinical_ner is working when I try to use PretrainedPipeline('explain_clinical_doc_era', 'en', 'clinical/models').
The error am getting is posted below for your reference
Expected Behavior
Current Behavior
Couldn't download the model clinical ner and ICD 10 model
Steps to Reproduce
Followed this notebook with a licensed version>> https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Healthcare/3.Clinical_Entity_Resolvers.ipynb
Context
I am trying to extract ICD code from notes I couldnt create a pipeline
Your Environment
Spark NLP version
sparknlp.version()
: 3.2.3Apache NLP version
spark.version
: 3.2.3 pyspark 3.1.2spark-nlp 3.2.3
spark-nlp-jsl 3.2.3
Java version
java -version
: 1.8.3 openjdk version "1.8.0_292" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)Setup and installation (Pypi, Conda, Maven, etc.): pypi
Operating System and version: Mac os
Link to your project (if any):
Error Text -
Approximate size to download 367.3 KB [ | ]sentence_detector_dl_healthcare download started this may take some time. Approximate size to download 367.3 KB [ / ]Download done! Loading the resource. [ — ] 2021-09-28 18:22:15.930532: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. [OK!] embeddings_clinical download started this may take some time. Approximate size to download 1.6 GB [ / ]embeddings_clinical download started this may take some time. Approximate size to download 1.6 GB Download done! Loading the resource. [OK!] ner_clinical download started this may take some time. Approximate size to download 13.9 MB [ | ] An error occurred while calling z:com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader.downloadModel. : com.johnsnowlabs.license.exceptions.JslInvalidLicenseException: Wrong symbol in license key: [77aaa362]. at com.johnsnowlabs.license.LicenseValidator$.$anonfun$getLicense$4(LicenseValidator.scala:96) at scala.Option.flatMap(Option.scala:271) at com.johnsnowlabs.license.LicenseValidator$.$anonfun$getLicense$2(LicenseValidator.scala:82) at scala.Option.getOrElse(Option.scala:189) at com.johnsnowlabs.license.LicenseValidator$.getLicense(LicenseValidator.scala:75) at com.johnsnowlabs.license.LicenseValidator$.checkValidLicenseIsPresent(LicenseValidator.scala:164) at com.johnsnowlabs.license.CheckLicense.checkValidEnvironment(CheckLicense.scala:19) at com.johnsnowlabs.license.CheckLicense.checkValidEnvironment$(CheckLicense.scala:18) at com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader$.checkValidEnvironment(InternalsPythonResourceDownloader.scala:13) at com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader$.downloadModel(InternalsPythonResourceDownloader.scala:37) at com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader.downloadModel(InternalsPythonResourceDownloader.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) [OK!]
Py4JJavaError Traceback (most recent call last) /var/folders/8n/zt5x88910bx47vs2xyhwl4nw0000gn/T/ipykernel_48112/1081320759.py in
20
21 # Named Entity Recognition for clinical concepts.
---> 22 clinical_ner = MedicalNerModel.pretrained("ner_clinical", "en", "clinical/models") \
23 .setInputCols(["sentence", "token", "word_embeddings"]) \
24 .setOutputCol("ner")
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/sparknlp_jsl/annotator.py in pretrained(name, lang, remote_loc) 1901 from sparknlp.pretrained import ResourceDownloader 1902 return ResourceDownloader.downloadModel(MedicalNerModel, name, lang, remote_loc, -> 1903 j_dwn='InternalsPythonResourceDownloader') 1904 1905
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/sparknlp/pretrained.py in downloadModel(reader, name, language, remote_loc, j_dwn) 60 except Py4JJavaError as e: 61 sys.stdout.write("\n" + str(e)) ---> 62 raise e 63 finally: 64 stop_threads = True
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/sparknlp/pretrained.py in downloadModel(reader, name, language, remote_loc, j_dwn) 57 t1.start() 58 try: ---> 59 j_obj = _internal._DownloadModel(reader.name, name, language, remote_loc, j_dwn).apply() 60 except Py4JJavaError as e: 61 sys.stdout.write("\n" + str(e))
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/sparknlp/internal.py in init(self, reader, name, language, remote_loc, validator) 212 def init(self, reader, name, language, remote_loc, validator): 213 super(_DownloadModel, self).init("com.johnsnowlabs.nlp.pretrained." + validator + ".downloadModel", reader, --> 214 name, language, remote_loc) 215 216
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/sparknlp/internal.py in init(self, java_obj, args) 163 super(ExtendedJavaWrapper, self).init(java_obj) 164 self.sc = SparkContext._active_spark_context --> 165 self._java_obj = self.new_java_obj(java_obj, args) 166 self.java_obj = self._java_obj 167
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/sparknlp/internal.py in new_java_obj(self, java_class, args) 173 174 def new_java_obj(self, java_class, args): --> 175 return self._new_java_obj(java_class, *args) 176 177 def new_java_array(self, pylist, java_class):
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/pyspark/ml/wrapper.py in _new_java_obj(java_class, args) 64 java_obj = getattr(java_obj, name) 65 java_args = [_py2java(sc, arg) for arg in args] ---> 66 return java_obj(java_args) 67 68 @staticmethod
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/py4j/java_gateway.py in call(self, *args) 1303 answer = self.gateway_client.send_command(command) 1304 return_value = get_return_value( -> 1305 answer, self.gateway_client, self.target_id, self.name) 1306 1307 for temp_arg in temp_args:
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/pyspark/sql/utils.py in deco(*a, kw) 109 def deco(*a, *kw): 110 try: --> 111 return f(a, kw) 112 except py4j.protocol.Py4JJavaError as e: 113 converted = convert_exception(e.java_exception)
~/.pyenv/versions/entity_extraction/lib/python3.7/site-packages/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name) 326 raise Py4JJavaError( 327 "An error occurred while calling {0}{1}{2}.\n". --> 328 format(target_id, ".", name), value) 329 else: 330 raise Py4JError(
Py4JJavaError: An error occurred while calling z:com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader.downloadModel. : com.johnsnowlabs.license.exceptions.JslInvalidLicenseException: Wrong symbol in license key: [77aaa362]. at com.johnsnowlabs.license.LicenseValidator$.$anonfun$getLicense$4(LicenseValidator.scala:96) at scala.Option.flatMap(Option.scala:271) at com.johnsnowlabs.license.LicenseValidator$.$anonfun$getLicense$2(LicenseValidator.scala:82) at scala.Option.getOrElse(Option.scala:189) at com.johnsnowlabs.license.LicenseValidator$.getLicense(LicenseValidator.scala:75) at com.johnsnowlabs.license.LicenseValidator$.checkValidLicenseIsPresent(LicenseValidator.scala:164) at com.johnsnowlabs.license.CheckLicense.checkValidEnvironment(CheckLicense.scala:19) at com.johnsnowlabs.license.CheckLicense.checkValidEnvironment$(CheckLicense.scala:18) at com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader$.checkValidEnvironment(InternalsPythonResourceDownloader.scala:13) at com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader$.downloadModel(InternalsPythonResourceDownloader.scala:37) at com.johnsnowlabs.nlp.pretrained.InternalsPythonResourceDownloader.downloadModel(InternalsPythonResourceDownloader.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748)