Difference between JSL’s “nlu” and “spark-nlp” packages?

[falconair]

Hi, can you please add a comment on the description page about the difference between nlu and spark-nlp libraries?

[C-K-Loan]

HI @falconair ,

thank you for the suggestion, we will be adding such a description page.

In a nutshell, you could say the relationship between NLU and Spark NLP is like Keras and Tensorflow. NLU is a high-level wrapper around Spark NLP, which is the core computation and feature engine.

It gives you the following things :

• Access almost every of Spark NLP/ OCR and Healthcare features in exactly 1 line of code, reducing boilerplate code
• Automates a lot of Data-Engineering tasks with the parameters you have in the .predict(), reducing boilerplate code because you don't have to deal with complex Spark Dataframe Schemas
• Works on Pythonic Datastructures like Pandas DF, Strings, Lists of Strings etc.. Spark -NLP mainly works on Spark Dataframes.
• Access to dozens of visualizations & Streamlit features in 1 line of code https://nlu.johnsnowlabs.com/docs/en/viz_examples https://nlu.johnsnowlabs.com/docs/en/streamlit_viz_examples
• Various utilities to make working with Spark NLP easier https://nlu.johnsnowlabs.com/docs/en/utils_for_spark_nlp

Hope this clears up all questions,

best greetings,

Christian