TF Exception : UniversalSentenceEncoder.pretrained("tfhub_use_multi_lg", "xx")

C-K-Loan commented 3 years ago

Description

Loading UniversalSentenceEncoder.pretrained("tfhub_use_multi_lg", "xx") causes TF exceptions beeing thrown


Py4JJavaError: An error occurred while calling z:com.johnsnowlabs.nlp.pretrained.PythonResourceDownloader.downloadModel.
: org.tensorflow.exceptions.TensorFlowException: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /tmp/6a7e53ae5d72_ner1548179176805096808/variables
     [[{{node save/RestoreV2}}]]
    at org.tensorflow.internal.c_api.AbstractTF_Status.throwExceptionIfNotOK(AbstractTF_Status.java:101)
    at org.tensorflow.Session.run(Session.java:666)
    at org.tensorflow.Session.access$100(Session.java:72)
    at org.tensorflow.Session$Runner.runHelper(Session.java:381)
    at org.tensorflow.Session$Runner.run(Session.java:329)
    at com.johnsnowlabs.ml.tensorflow.TensorflowWrapper$.readWithSP(TensorflowWrapper.scala:391)
    at com.johnsnowlabs.ml.tensorflow.ReadTensorflowModel.readTensorflowWithSPModel(TensorflowSerializeModel.scala:162)
    at com.johnsnowlabs.ml.tensorflow.ReadTensorflowModel.readTensorflowWithSPModel$(TensorflowSerializeModel.scala:135)
    at com.johnsnowlabs.nlp.embeddings.UniversalSentenceEncoder$.readTensorflowWithSPModel(UniversalSentenceEncoder.scala:214)
    at com.johnsnowlabs.nlp.embeddings.ReadUSETensorflowModel.readTensorflow(UniversalSentenceEncoder.scala:188)
    at com.johnsnowlabs.nlp.embeddings.ReadUSETensorflowModel.readTensorflow$(UniversalSentenceEncoder.scala:186)
    at com.johnsnowlabs.nlp.embeddings.UniversalSentenceEncoder$.readTensorflow(UniversalSentenceEncoder.scala:214)
    at com.johnsnowlabs.nlp.embeddings.ReadUSETensorflowModel.$anonfun$$init$$1(UniversalSentenceEncoder.scala:192)
    at com.johnsnowlabs.nlp.embeddings.ReadUSETensorflowModel.$anonfun$$init$$1$adapted(UniversalSentenceEncoder.scala:192)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$onRead$1(ParamsAndFeaturesReadable.scala:31)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$onRead$1$adapted(ParamsAndFeaturesReadable.scala:30)
    at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
    at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.onRead(ParamsAndFeaturesReadable.scala:30)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$read$1(ParamsAndFeaturesReadable.scala:41)
    at com.johnsnowlabs.nlp.ParamsAndFeaturesReadable.$anonfun$read$1$adapted(ParamsAndFeaturesReadable.scala:41)
    at com.johnsnowlabs.nlp.FeaturesReader.load(ParamsAndFeaturesReadable.scala:19)
    at com.johnsnowlabs.nlp.FeaturesReader.load(ParamsAndFeaturesReadable.scala:8)
    at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(ResourceDownloader.scala:363)
    at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(ResourceDownloader.scala:357)
    at com.johnsnowlabs.nlp.pretrained.PythonResourceDownloader$.downloadModel(ResourceDownloader.scala:474)
    at com.johnsnowlabs.nlp.pretrained.PythonResourceDownloader.downloadModel(ResourceDownloader.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.lang.Thread.run(Thread.java:748)

Steps to Reproduce

Use this notebook
- Or run the following in a colab cell

!wget http://setup.johnsnowlabs.com/colab.sh -O - | bash
import sparknlp
from sparknlp.annotator import *
sparknlp.start()
# crash
use = UniversalSentenceEncoder.pretrained("tfhub_use_multi_lg", "xx")

Other Multi Lingual Models tested, that work fine

WordEmbeddingsModel.pretrained('glove_840B_300','xx')\
BertSentenceEmbeddings.pretrained("labse", "xx")
BertSentenceEmbeddings.pretrained("sent_bert_multi_cased", "xx")

Your Environment

Spark NLP version sparknlp.version(): 3.0.1
Apache NLP version spark.version: 3.0.2
Java version java -version: 1.8
Setup and installation (Pypi, Conda, Maven, etc.): !wget http://setup.johnsnowlabs.com/colab.sh -O - | bash
Operating System and version: Google Collab

maziyarpanahi commented 3 years ago

I have more info on this:

Deprecated Models in Spark NLP 3.0.x These don't have TF2.0 SavedModel on TF Hub yet

tfhub_use_xling_many
tfhub_use_xling_en_es
tfhub_use_xling_en_fr
tfhub_use_xling_en_de

It was already reported here and we are waiting: https://github.com/tensorflow/hub/issues/735

I'll keep this issue open for other users. Thanks @C-K-Loan

maziyarpanahi commented 3 years ago

@C-K-Loan to extend the last comment, the following models however should work. I have found the issue and it will be fixed in the next release:

tfhub_use_multi
tfhub_use_multi_lg

JohnSnowLabs / spark-nlp