search

JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing
https://sparknlp.org/
Apache License 2.0
3.86k stars 711 forks source link

Could you tell me how to use latest pretrained tensorFlow USE data in Spark? #2402

Closed franklydog closed 3 years ago

franklydog commented 3 years ago

I get google's USE pretrained latest data in https://tfhub.dev/google/universal-sentence-encoder-multilingual-large/3. So I want to use it in class of UniversalSentenceEncoder.As far as I know, Spark NLP also use Tensorflow-hub version of USE that is wrapped in a way. Could you tell me what is the "Way"? Thank you for any help you can offer.

maziyarpanahi commented 3 years ago

Hi, Since we are still on TensorFlow 1.15.x in Java, the process of doing that at the moment is long and outside of the scope of Spark NLP. In future releases, we'll see if we can make it easier once we are migrated to TensorFlow 2.x.

That's being said, we already have those multi-lingual models available: https://nlp.johnsnowlabs.com/2020/12/08/tfhub_use_multi_lg_xx.html