JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing
https://sparknlp.org/
Apache License 2.0
3.76k stars 703 forks source link

Sparknlp 1035 test all notebooks to import tensor flow models to spark nlp #14238

Closed ahmedlone127 closed 1 month ago

ahmedlone127 commented 2 months ago

Updating transformers and tensorflow versions for all notebooks

Description

This PR upgrades transformers version to 4.39.3 and tensorflow version to 2.11.0 for all tensorflow export notebooks It also fixes various minor bugs, colab links, typos and other small improvements.

Known issues: HuggingFace in Spark NLP - XlmRoBertaForQuestionAnswering.ipynb (Missing spp file when trying to import ) HuggingFace in Spark NLP - XlmRoBertaForTokenClassification.ipynb (Missing spp file when trying to import ) HuggingFace in Spark NLP - XlmRoBertaForSequenceClassification.ipynb (Missing spp file when trying to import ) HuggingFace in Spark NLP - XLNet.ipynb( int 32/64 tensorflow error ) HuggingFace in Spark NLP - SwinForImageClassification.ipynb(Py4JJavaError: An error occurred while calling z:com.johnsnowlabs.nlp.annotators.cv.SwinForImageClassification.loadSavedModel.) HuggingFace in Spark NLP - LongformerForQuestionAnswering.ipynb( Kept running then RAM error, try chanigng model ) HuggingFace in Spark NLP - Longformer.ipynb( Kept running then RAM error, try chanigng model ) CamemBertForTokenClassification( int 32/64 tensorflow error ) HuggingFace in Spark NLP - BERT Sentence.ipynb( int 32/64 tensorflow error ) Fine_Tuned_Sentence_Bert_in_Spark_NLP.ipynb